Maintained with ☕️ by
IcePanel logo

Dataproc - February 26th, 2021 [Change, Fix]

Share

Services

## Change New [sub-minor versions](https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions) of Dataproc images: `1.3.85-debian10`, `1.3.85-ubuntu18`, `1.4.56-debian10`, `1.4.56-ubuntu18`, `1.5.31-centos8`, `1.5.31-debian10`, `1.5.31-ubuntu18`, `2.0.4-debian10`, and `2.0.4-ubuntu18` ## Change **Image 2.0:** Upgraded Spark to [3.1.1 RC2 version](https://github.com/apache/spark/releases/tag/v3.1.1-rc2) ## Change Allow stopping clusters that have autoscaling enabled, and allow enabling autoscaling on clusters that are `STOPPED`, `STOPPING`, or `STARTING`. If you stop a cluster that has autoscaling enabled, the Dataproc autoscaler will stop scaling the cluster. It will resume scaling the cluster once it has been started again. If you enable autoscaling on a stopped cluster, the autoscaling policy will only take effect once the cluster has been started (see [Starting and stopping clusters](https://cloud.google.com/dataproc/docs/guides/dataproc-start-stop.md)). ## Change Deactivated `mysql` and `hive-metastore` components for clusters created with a [Dataproc Metastore](https://cloud.google.com/dataproc-metastore) service on an image that has the `DISABLE_COMPONENT_HIVE_METASTORE` and `DISABLE_COMPONENT_MYSQL` capabilities. ## Fix **Image 1.3 - 1.5**: [HIVE-18871](https://issues.apache.org/jira/browse/HIVE-18871): hive on Tez execution error due to set `hive.aux.jars.path` to `hdfs://`