Using Dataproc version 2.2 in your Cloud Data Fusion pipeline can fail in some cases with the following error
Share
Services
## Issue
Using Dataproc version 2.2 in your Cloud Data Fusion pipeline can fail in some cases with the following error: `ERROR [Driver:o.a.s.d.y.ApplicationMaster@97] - User class threw exception: java.lang.NoSuchMethodError: 'org.apache.spark.sql.catalyst.encoders.ExpressionEncoder org.apache.spark.sql.catalyst.encoders.RowEncoder.apply(org.apache.spark.sql.types.StructType)' at io.cdap.cdap.etl.spark.batch.OpaqueDatasetCollection.toDataframeCollection(OpaqueDatasetCollection.java:111)`.
To avoid this issue, [change the Dataproc image to 2.1](https://cloud.google.com/data-fusion/docs/how-to/change-dataproc-image#change-to-dataproc-21) ([CDAP-21075](https://cdap.atlassian.net/browse/CDAP-21075)).
What else is happening at Google Cloud Platform?
The CPU allocation setting has been renamed to Billing in the Google Cloud console for Cloud Run services
December 13th, 2024
Services
Share
Google Kubernetes Engine (GKE) - December 13th, 2024 [Feature]
December 13th, 2024
Services
Share