Dataproc - July 10th, 2020 [Change, Feature, Fix]
Share
Services
## Feature
Added `--temp-bucket` flag to `gcloud dataproc clusters create` and `gcloud dataproc workflow-templates set-managed-cluster` to allow users to configure a Cloud Storage bucket that stores ephemeral cluster and jobs data, such as Spark and MapReduce history files.
## Feature
Extended Jupyter to support notebooks stored on VM persistent disk. This change modifies the Jupyter contents manager to create two virtual top-level directories, named `GCS`, and `Local Disk`. The `GCS` directory points to the Cloud Storage location used by previous versions, and the `Local Disk` directory points to the persistent disk of the VM running Jupyter.
## Feature
Dataproc images now include the [oauth2l](https://github.com/google/oauth2l) command line tool. The tool is installed in `/usr/local/bin`, which is available to all users in the VM.
## Change
New [sub-minor versions](https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#supported%5Fdataproc%5Fversions) of Dataproc images: 1.2.102-debian9, 1.3.62-debian9, 1.4.33-debian9, 1.3.62-debian10, 1.4.33-debian10, 1.5.8-debian10, 1.3.62-ubuntu18, 1.4.33-ubuntu18, 1.5.8-ubuntu18, 2.0.0-RC4-debian10, 2.0.0-RC4-ubuntu18
## Fix
* Images 1.3 - 1.5:
* Fixed [HIVE-11920](https://issues.apache.org/jira/browse/HIVE-11920): ADD JAR failing with URL schemes other than file/ivy/hdfs.
* Images 1.3 - 2.0 preview:
* Fixed [TEZ-4108](https://issues.apache.org/jira/browse/TEZ-4108): NullPointerException during speculative execution race condition.
## Fix
Fixed a race condition that could nondeterministically cause Hive-WebHCat to fail at startup when HBase is not enabled.
What else is happening at Google Cloud Platform?
Toxic combination findings are generally available. This includes the following updates
October 16th, 2024
Services
Share