Maintained with ☕️ by
IcePanel logo

Dataproc - July 10th, 2020 [Change, Feature, Fix]

Share

Services

## Feature Added `--temp-bucket` flag to `gcloud dataproc clusters create` and `gcloud dataproc workflow-templates set-managed-cluster` to allow users to configure a Cloud Storage bucket that stores ephemeral cluster and jobs data, such as Spark and MapReduce history files. ## Feature Extended Jupyter to support notebooks stored on VM persistent disk. This change modifies the Jupyter contents manager to create two virtual top-level directories, named `GCS`, and `Local Disk`. The `GCS` directory points to the Cloud Storage location used by previous versions, and the `Local Disk` directory points to the persistent disk of the VM running Jupyter. ## Feature Dataproc images now include the [oauth2l](https://github.com/google/oauth2l) command line tool. The tool is installed in `/usr/local/bin`, which is available to all users in the VM. ## Change New [sub-minor versions](https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#supported%5Fdataproc%5Fversions) of Dataproc images: 1.2.102-debian9, 1.3.62-debian9, 1.4.33-debian9, 1.3.62-debian10, 1.4.33-debian10, 1.5.8-debian10, 1.3.62-ubuntu18, 1.4.33-ubuntu18, 1.5.8-ubuntu18, 2.0.0-RC4-debian10, 2.0.0-RC4-ubuntu18 ## Fix * Images 1.3 - 1.5: * Fixed [HIVE-11920](https://issues.apache.org/jira/browse/HIVE-11920): ADD JAR failing with URL schemes other than file/ivy/hdfs. * Images 1.3 - 2.0 preview: * Fixed [TEZ-4108](https://issues.apache.org/jira/browse/TEZ-4108): NullPointerException during speculative execution race condition. ## Fix Fixed a race condition that could nondeterministically cause Hive-WebHCat to fail at startup when HBase is not enabled.