Maintained with ☕️ by
IcePanel logo

Dataflow supports Apache Beam SDK version 2.78.0 for Java, Python,

Share

Services

## Feature Feature Dataflow supports Apache Beam SDK version 2.78.0 for Java, Python, and Go ([SDK version support status](https://cloud.google.com/dataflow/docs/support/sdk-version-support-status)). This version offers new features such as: * BigQuery API quota and billing project attribution across the Java, Python, and Go SDKs * Managed I/O support for Apache Iceberg `AddFiles` in Java and Python, and dynamic multi-table file registration in Java * Write ramp-up throttling opt-out for Java `FirestoreIO` * `SplunkIO` write support in Beam YAML, and full source metadata support for `DebeziumIO` * Secret Manager resolution for AWS credentials in Python pipelines For more information, see the[Apache Beam SDK release notes](https://github.com/apache/beam/releases/tag/v2.78.0). ## Feature Feature Eager watermark advancement is generally available (GA) for[Streaming Engine](https://cloud.google.com/dataflow/docs/streaming-engine) jobs. When you enable eager watermark advancement, records that arrive behind the current watermark no longer stall the watermark, so windowed results and event-time timers run on schedule. Delayed records are still processed concurrently with on-time data. If your pipeline uses windowed aggregations or stateful processing, configure allowed lateness to cover expected delays before you enable this feature. For more information, see[Eager watermark advancement](https://cloud.google.com/dataflow/docs/guides/eager-watermark-advancement).