Cloud Data Fusion version 6.11.0 is available in Preview. You can view instance metrics and pipeline metrics in Cloud Monitoring and in the dashboard provided by Cloud Data Fusion

## Feature Cloud Data Fusion version 6.11.0 is available in [Preview](https://cloud.google.com/products#product-launch-stages). ## Feature You can view instance metrics and pipeline metrics in Cloud Monitoring and in the dashboard provided by Cloud Data Fusion. For more information, see [Metrics overview](https://cloud.google.com/data-fusion/docs/concepts/metrics-overview) and [Monitor Cloud Data Fusion system, instance, and pipeline health](https://cloud.google.com/data-fusion/docs/how-to/metrics-views). ## Feature You can view instance logs and pipeline logs in Cloud Logging, and in the dashboard provided by Cloud Data Fusion. For more information, see [View Cloud Data Fusion logs](https://cloud.google.com/data-fusion/docs/how-to/view-datafusion-logs). ## Feature When a pipeline run fails, you can retrieve detailed error information on the pipeline details page of the Cloud Data Fusion web interface. Cloud Data Fusion classifies pipeline errors by category, reason, and message. This classification speeds up resolution and reduces the need to examine complex logs. For more information, see [Retrieve error information for a failed pipeline run](https://cloud.google.com/data-fusion/docs/troubleshoot#pipeline). ## Feature Cloud Data Fusion 6.11.0 offers high availability with reduced upgrade downtime. ## Change Changes in Cloud Data Fusion 6.11.0: * To create ephemeral clusters, Cloud Data Fusion uses the Dataproc 2.2 image by default. For more information about its limitations in Cloud Data Fusion, see [Change the Dataproc image to version 2.1](https://cloud.google.com/data-fusion/docs/how-to/change-dataproc-image#change-to-dataproc-21). * The maximum concurrent runs limit for triggers is displayed in the console ([CDAP-21072](https://cdap.atlassian.net/browse/CDAP-21072)). * Added support for destination table write preference in the BigQuery Execute plugin ([PLUGIN-1438](https://cdap.atlassian.net/browse/PLUGIN-1438)). ## Fix Fixed in Cloud Data Fusion 6.11.0: * Fixed a null pointer exception in the BigQuery multi-sink plugin when used without a reference name ([PLUGIN-1843](https://cdap.atlassian.net/browse/PLUGIN-1843)). * Fixed Joiner plugin failures observed on Dataproc 2.2-debian12 instances ([CDAP-21075](https://cdap.atlassian.net/browse/CDAP-21075)). * Fixed an issue that prevented pipelines from accepting empty input in Amazon S3 and Google Cloud Storage source plugins ([PLUGIN-1742](https://cdap.atlassian.net/browse/PLUGIN-1742)). * Fixed an issue in RBAC-enabled instances where the pipeline details page displayed an incorrect author name ([CDAP-21069](https://cdap.atlassian.net/browse/CDAP-21069)). ## Breaking A soft limit of 2 MB for pipeline JSON size is introduced in 6.11.0\. Pipelines exceeding this size might encounter deployment failures. ## Deprecate The following APIs for searching and querying metrics are deprecated in 6.11.0: * `POST v3/metrics/query` * `POST v3/metrics/search` ## Deprecate The following APIs for downloading system service and pipeline run logs are deprecated in 6.11.0: * `GET /v3/namespaces/<NAMESPACE_ID>/apps/<APP_ID>/<PROGRAM_TYPE>/<PROGRAM_ID>/logs` * `GET /v3/system/services/<SERVICE_ID>/logs` ## Deprecate The ability to retrieve all applications without pagination using the `GET /v3/namespaces/<NAMESPACE_ID>/apps` endpoint is deprecated in 6.11.0.