We released an updated version of the Apigee hybrid software, v1.14.0

## Announcement ### hybrid v1.14.0 On December 16, 2024 we released an updated version of the Apigee hybrid software, v1.14.0. * For information on upgrading, see [Upgrading Apigee hybrid to version v1.14](https://cloud.google.com/apigee/docs/hybrid/v1.14/upgrade). * For information on new installations, see [The big picture](https://cloud.google.com/apigee/docs/hybrid/v1.14/big-picture). ## Feature **Enhanced Proxy Limits in Hybrid environments** Starting in version v1.14, new Apigee hybrid organizations can be provisioned with the ability to deploy more than 50 proxies per environment enabled. This feature is already available for [Apigee X](https://cloud.google.com/apigee/docs/api-platform/reference/limits#api-proxies). Starting with Apigee hybrid version 1.14, the limits for Apigee hybrid organizations have increased: * The maximum number of deployed API proxies and shared flows per organization is 6000. * The maximum number of proxy deployment units per Apigee instance is 6000. * The maximum number of API base paths per Apigee organization is 3000. When more than 50 proxies are deployed in an environment, Apigee will automatically partition the environment into several distinct replica sets, each containing a subset of proxies deployed in the environment. These replica subsets are equivalent in behavior to a single environment in the way it loads and runs a set of proxies and other environment resources. This will be transparent to the user, and you can continue to use the environment as you would a single environment. ## Feature **Cassandra credential rotation** Starting in version v1.14, you can rotate Cassandra credentials in Kubernetes secrets. In addition, you can now roll back credential rotation before the cleanup job is initiated in both Vault and Kubernetes secrets. See: * [Rotating Cassandra credentials in Kubernetes secrets](https://cloud.google.com/apigee/docs/hybrid/v1.14/rotating-cassandra-credentials-non-ess) * [Rotating Cassandra credentials in Vault: Rolling back a rotation](https://cloud.google.com/apigee/docs/hybrid/v1.14/rotating-cassandra-credentials-in-vault#rollback-rotation) * [Rotating Cassandra credentials in Kubernetes secrets: Rolling back a rotation](https://cloud.google.com/apigee/docs/hybrid/v1.14/rotating-cassandra-credentials-non-ess#rollback-rotation) ## Feature **Enable and disable metrics-based scaling with `customAutoscaling.enabled`** Starting in version v1.14, you can enable and disable metrics-based auto-scaling with the `customAutoscaling.enabled` configuration property. See: * [Scale and autoscale runtime services: Metrics-based scaling](https://cloud.google.com/apigee/docs/hybrid/v1.14/scale-and-autoscale#metrics-scaling) * [customAutoscaling.enabled](https://cloud.google.com/apigee/docs/hybrid/v1.14/config-prop-ref#customautoscaling-enabled) ## Feature **New analytics and debug data pipeline for hybrid orgs** Starting with version 1.14, all newly created Apigee hybrid orgs created can use a new data pipeline to collect analytics and debug data and allow various runtime components to write data directly to our control plane. See: * [Enable Control Plane access](https://cloud.google.com/apigee/docs/hybrid/v1.14/install-enable-control-plane-access) * [Using data residency with Apigee hybrid](https://cloud.google.com/apigee/docs/hybrid/v1.14/using-data-residency-with-apigee-hybrid) ## Feature **Forward Proxy allowlist access** Starting in version v1.14, forward proxies pass through access to allowlisted URLs. Therefore you only need to configure allowlists to googleapis.com URLs on the server on which the forward proxy is configured. See: * [Google Cloud URLs to allow for Hybrid](https://cloud.google.com/apigee/docs/hybrid/v1.14/allow-gcp-urls) * [Using Data Residency with Apigee hybrid: URL allowlisting](https://cloud.google.com/apigee/docs/hybrid/v1.14/using-data-residency-with-apigee-hybrid#url-allowlisting) * [Configure forward proxying for API proxies](https://cloud.google.com/apigee/docs/hybrid/v1.14/forward-proxy) ## Feature **Guardrails checks to ensure backups before upgrade** Starting in version 1.14 new guardrails checks have been added to ensure a backup is enabled and has been made before proceeding with an upgrade. See: * [Upgrading Apigee hybrid to version v1.14](https://cloud.google.com/apigee/docs/hybrid/v1.14/scale-and-autoscale#metrics-scaling) * [Diagnosing issues with guardrails](https://cloud.google.com/apigee/docs/hybrid/v1.14/config-prop-ref#customautoscaling-enabled) ## Fix | Bug ID | Description | | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **382323427** | **Added a guardrails check that requires backup to be enabled for Apigee Hybrid upgrades. Backups are required prior to upgrading to support restoring to the previous version, if necessary.** | | **380346557** | **Added a guardrails check that requires the backup within the last 24 hours to be present if the CSI backup is enabled. This will minimize potential data loss if a restore to the previous version is needed.** | | **377573589** | **Fix a bug where manually created rollbacks would interfere with existing rotations instead of cancelling them.** | | **362305438** | **Users can now add additional env variables to the runtime component.** See [runtime.envVars](https://cloud.google.com/apigee/docs/hybrid/v1.14/config-prop-ref#runtime-envvars) | | **319152386** | **Fix AccessTokenGenerationFailure in runtime when using a forward proxy.** | | **335357961** | **Fixed an issue where Apigee hybrid could claim uploads of backups with the Cloud provider when no bucket had been configured** | | **290183372** | **The need to whitelist oauth2 and iamcredentials.googleapis.com directly from MP in fwd proxy setup is removed.** | | **237656263** | **Resolved issue with ServiceCallout policy not working in async mode as expected.** | | **373722434** | **Fixed support for backups to Google Cloud Storage buckets with retention policies.** (Fixed in [v1.13.2](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1132)) | | **368646378** | **Fixed an issue affecting control Plane connectivity testing in Guardrails.** (Fixed in [v1.12.3](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1123)) | | **364282883** | **Remove check for dc-expansion flag and add timeout to multi-region seed host connection test.** (Fixed in [v1.13.1](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1131)) | | **362979563** | **Fix for Ingress Health Check failure /healthz/ingress - route\_not\_found.** (Fixed in [1.13.0-hotfix.1](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5F1130-hotfix1)) | | **362690729** | **Fix for aggressive scaling of runtime pods & cpu spike.** (Fixed in [1.13.0-hotfix.1](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5F1130-hotfix1)) | | **362305438** | **You can now add additional env variables to the runtime component.** (Fixed in [v1.13.1](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1131)) | | **361044374** | **Fixes assign message not correctly highlighting the set payload action in the debug trace.** (Fixed in [v1.13.2](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1132)) | | **355122464** | **This release contains a few error-handling fixes for CSI backup and restore.** (Fixed in [v1.13.2](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1132)) | | **353527851** | **WebSocket connection drops when using VerifyJwt or OAuthV2 VerifyJWTAccessToken operations.** (Fixed in [v1.13.1](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1131)) | | **351440306** | **An issue was fixed where trace could not be viewed in the UI for orgs with DRZ enabled.** (Fixed in [v1.13.1](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1131)) | | **347798999** | **You can now configure forward proxy for opentelemetry pods in Apigee hybrid.** (Fixed in [v1.12.2](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1122)) | | **338638343** | **An ID is now added at the end of apigee-env and virtualhost guardrails pods to make the pod names unique.** (Fixed in [v1.13.1](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1131)) | | **237656263** | **Fix added to make use of asynchronous ServiceCallout execution when the [ServiceCallout policy <Response> element](https://cloud.google.com/apigee/docs/api-platform/reference/policies/service-callout-policy#response) is not present** (Fixed in [v1.13.2](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1132)) | | **181569113** | **Fixed an issue in new debug session creation.** (Fixed in [v1.12.3](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1123)) | ## Security | Bug ID | Description | | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **N/A** | **Security fixes for apigee-redis.** This addresses the following vulnerabilities: [CVE-2022-24834](https://nvd.nist.gov/vuln/detail/CVE-2022-24834) [CVE-2022-24735](https://nvd.nist.gov/vuln/detail/CVE-2022-24735) | | **N/A** | **Security fixes for livenessprobe.** This addresses the following vulnerability: [CVE-2023-45288](https://nvd.nist.gov/vuln/detail/CVE-2023-45288) | | **376104926** | **Security fixes for apigee-kube-rbac-proxy.** (Fixed in [v1.12.3](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1123)) This addresses the following vulnerabilities: [CVE-2024-41110](https://nvd.nist.gov/vuln/detail/CVE-2024-41110) [CVE-2024-28180](https://nvd.nist.gov/vuln/detail/CVE-2024-28180) [CVE-2024-24790](https://nvd.nist.gov/vuln/detail/CVE-2024-24790) [CVE-2024-24789](https://nvd.nist.gov/vuln/detail/CVE-2024-24789) [CVE-2023-4039](https://nvd.nist.gov/vuln/detail/CVE-2023-4039) [CVE-2022-27943](https://nvd.nist.gov/vuln/detail/CVE-2022-27943) [CVE-2019-1010025](https://nvd.nist.gov/vuln/detail/CVE-2019-1010025) [CVE-2019-1010024](https://nvd.nist.gov/vuln/detail/CVE-2019-1010024) [CVE-2019-1010023](https://nvd.nist.gov/vuln/detail/CVE-2019-1010023) [CVE-2019-1010022](https://nvd.nist.gov/vuln/detail/CVE-2019-1010022) [CVE-2019-9192](https://nvd.nist.gov/vuln/detail/CVE-2019-9192) [CVE-2018-20796](https://nvd.nist.gov/vuln/detail/CVE-2018-20796) [CVE-2012-2663](https://nvd.nist.gov/vuln/detail/CVE-2012-2663) [CVE-2010-4756](https://nvd.nist.gov/vuln/detail/CVE-2010-4756) | | **N/A** | **Security fixes for apigee-redis.** (Fixed in [v1.13.2](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1132)) This addresses the following vulnerabilities: [CVE-2022-24834](https://nvd.nist.gov/vuln/detail/CVE-2022-24834) [CVE-2022-24735](https://nvd.nist.gov/vuln/detail/CVE-2022-24735) | | **N/A** | **Security fixes for apigee-open-telemetry-collector.** (Fixed in [v1.13.1](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1131)) This addresses the following vulnerability: [CVE-2024-36129](https://nvd.nist.gov/vuln/detail/CVE-2024-36129) | | **N/A** | **Security fixes for apigee-open-telemetry-collector.** (Fixed in [v1.12.3](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1123)) This addresses the following vulnerability: [CVE-2024-36129](https://nvd.nist.gov/vuln/detail/CVE-2024-36129) | | **N/A** | **Security fixes for apigee-cassandra-backup-utility and apigee-hybrid-cassandra.** (Fixed in [v1.12.2](https://cloud.google.com/apigee/docs/hybrid/release-notes#hybrid%5Fv1122)) This addresses the following vulnerability: [CVE-2023-37920](https://nvd.nist.gov/vuln/detail/CVE-2023-37920) |