New Managed Service for Apache Spark (formerly Dataproc on Compute Engine) subminor cluster image versions
Share
Services
## Announcement
Announcement
New [**Managed Service for Apache Spark** (formerly Dataproc on Compute Engine) subminor cluster image versions](https://docs.cloud.google.com/managed-spark/docs/concepts/versioning/image-version-lists#supported-dataproc-image-versions):
* 2.1.114-debian11, 2.1.114-rocky8, 2.1.114-ubuntu20, 2.1.114-ubuntu20-arm
* 2.2.82-debian12, 2.2.82-rocky9, 2.2.82-ubuntu22, 2.2.82-ubuntu22-arm
* 2.3.30-debian12, 2.3.30-ml-ubuntu22, 2.3.30-rocky9, 2.3.30-ubuntu22, 2.3.30-ubuntu22-arm
## Breaking
Breaking
**Managed Service for Apache Spark** (formerly Dataproc on Compute Engine): The configuration for Spark shuffle partitions (`spark.sql.shuffle.partitions`) has changed from an integer to a string type.
This change impacts image versions `2.3.30` and later in version `2.3`, and `2.2.82` and later in version `2.2`.
* **Impact:** This change only affects users who are programmatically setting the configuration in code using `spark.conf.set()` with an integer literal.
* **Impacted example:** `spark.conf.set("spark.sql.shuffle.partitions", 100)`
* **User action:** Update your code to pass a string literal instead of an integer.
* **Example fix:** `spark.conf.set("spark.sql.shuffle.partitions", "100")`
* **Not impacted:** Setting the configuration via command-line arguments (e.g., `spark-submit --conf spark.sql.shuffle.partitions=100`), properties files, or Spark SQL commands (`spark.sql("SET spark.sql.shuffle.partitions=100")`) remains unaffected, as these methods naturally parse the input as strings.
What else is happening at Google Cloud Platform?
Read update
Services
Share
Cloud Data Fusion version 6.11.1.3 is generally available (GA)
about 22 hours ago
Services
Share
Starting June 1, 2026, the Data Catalog service begins a phased shutdown
about 22 hours ago
Services
Share
Read update
Services
Share