GKE on Bare Metal 1.29.0-gke.1449 is now available for download
Share
Services
## Announcement
### Release 1.29.0-gke.1449
GKE on Bare Metal 1.29.0-gke.1449 is now available for [download](https://cloud.google.com/anthos/clusters/docs/bare-metal/latest/downloads). To upgrade, see [Upgrade clusters](https://cloud.google.com/anthos/clusters/docs/bare-metal/latest/how-to/upgrade). GKE on Bare Metal 1.29.0-gke.1449 runs on Kubernetes 1.29.
If you use a third-party storage vendor, check the [GDCV Ready storage partners](https://cloud.google.com/anthos/docs/resources/partner-storage) document to make sure the storage vendor has already passed the qualification for this release of GKE on Bare Metal.
## Announcement
**Version 1.15 end of life**: In accordance with the [Version Support Policy](https://cloud.google.com//anthos/docs/support/getting-support#version%5Fsupport%5Fpolicy), version 1.15 (all patch releases) of GKE on Bare Metal has reached its end of life and is no longer supported.
## Feature
* **GA**: Support [GKE Identity Service v2 capability](https://cloud.google.com/kubernetes-engine/enterprise/identity/setup/user-access#alternativeuserloginaccess) for an improved security flow when you authenticate with third-party identity solutions.
The GA offering of GKE Identity Service v2 has the following requirements and restrictions:
* GKE Identity Service v2 now requires ports `11001` and `11002` on the control plane load balancer nodes, instead of `8443` and `8444`. Ensure these ports are open and available before you upgrade a cluster to version 1.29.0-gke.1449 and higher. If the ports aren't open, upgrade preflight checks fail.
* GKE Identity Service v2 requires version 1.5.1 or higher of the Anthos Auth gcloud CLI component. If necessary, update the Anthos Auth component (`gcloud components update anthos-auth`). If you use the Google Cloud SDK, updating the SDK (`gcloud components update`) to version 474.0.0 or later also updates the Anthos Auth component to the required version.
* GKE Identity Service v2 doesn't work with GKE on Bare Metal clusters with the following configurations:
* Clusters with a single control plane node only.
* Clusters that use control plane nodes for load balancing. That is, clusters that aren't configured with either a separate load balancing node pool or manual load balancing.
* **GA**: Added support for skews of up to two minor versions for [selective node pool upgrades](https://cloud.google.com/anthos/clusters/docs/bare-metal/latest/how-to/upgrade-lifecycle#np%5Fversion%5Frules).
* **GA**: Added capability to [pause and resume cluster upgrades](https://cloud.google.com/anthos/clusters/docs/bare-metal/latest/how-to/upgrade#pause%5Fresume).
* **GA**: Maintenance mode now uses eviction-based draining for nodes, instead of taint-based draining. Eviction-based draining uses the Eviction API, which honors Pod Disruption Budgets (PDBs). Draining nodes this way provides better protection against workload disruptions.
* **Preview**: Added support for node-level private registry configuration for workload images.
* **Preview**: Added support for rolling back [select node pool upgrades](https://cloud.google.com/anthos/clusters/docs/bare-metal/latest/how-to/upgrade#select%5Fnp%5Fup).
* **Preview**: Added support for admin and hybrid clusters to manage multiple versions user clusters concurrently.
* **Preview**: Added support for using an intermediate Certificate Authority (CA) as the cluster root CA.
* **Preview**: Added support to route workload logs to a third-party custom Kafka destination. This capability isn't enabled by default. You enable this capability in the cluster `stackdriver` resource spec by adding the `unmanagedKafkaOutputConfig` section. This section lets you specify the IP addresses of Kafka message brokers (`brokers`), topic names (`topics`), and keys to map the topics to partitions (`topicKeys`).
* Improved command-line interface errors and [error documentation](https://cloud.google.com/distributed-cloud/docs/reference/gke-error-ref).
## Change
**Functionality changes:**
* GKE Identity Service v2 now sends extra parameters (`extraParams`) to your OIDC provider.
* Extra node viewing permissions are added for accounts specified with the `spec.clusterSecurity.authorization.clusterViewer.gcpAccounts` field in the Cluster resource.
* Added `Status.Available` field to `BareMetalMachine` resources to indicate whether the machine is available.
* Updated preflight checks add a check for networking kernel modules (`ip_tables` or `np_tables`) and remove the `iptables` package check.
* The Google plugin for the GKE Identity Service now caches the public keys based on `max-age` in `cache-control` header.
## Fix
**Fixes:**
* Fixed an issue where the kubelet doesn't honor shortened, 1-second grace period for pod deletion during eviction-based draining.
* Fixed a cluster upgrade issue where the `lifecycle-controller-deployer` Pod was unable to migrate existing GKE on Bare Metal resources to the latest API version. This issue blocked upgrades to earlier version 1.28 releases.
* Fixed an issue with configuring a proxy for your cluster that required you to manually set `HTTPS_PROXY` and `NO_PROXY` environment variables on the admin workstation.
* Fixed an issue where upgrades are blocked because `cluster-operator` can't delete stale, failing preflight check resources.
* Fixed an issue where the network check ConfigMap wasn't updated when nodes were added or removed.
## Fix
The following container image security vulnerabilities have been fixed in version 1.29.0-gke.1449:
* Critical container vulnerabilities:
* [CVE-2021-38297](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-38297)
* [CVE-2022-23806](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23806)
* [CVE-2023-24538](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-24538)
* [CVE-2023-24540](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-24540)
* [CVE-2023-25775](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-25775)
* [CVE-2023-29402](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-29402)
* [CVE-2023-29404](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-29404)
* [CVE-2023-29405](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-29405)
* High-severity container vulnerabilities:
* [CVE-2020-29652](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-29652)
* [CVE-2021-29923](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-29923)
* [CVE-2021-33195](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-33195)
* [CVE-2021-33196](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-33196)
* [CVE-2021-33198](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-33198)
* [CVE-2021-39293](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-39293)
* [CVE-2021-41771](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41771)
* [CVE-2021-41772](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41772)
* [CVE-2021-44716](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44716)
* [CVE-2022-2879](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-2879)
* [CVE-2022-2880](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-2880)
* [CVE-2022-21698](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-21698)
* [CVE-2022-23772](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23772)
* [CVE-2022-23773](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23773)
* [CVE-2022-24675](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-24675)
* [CVE-2022-24921](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-24921)
* [CVE-2022-28131](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-28131)
* [CVE-2022-28327](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-28327)
* [CVE-2022-28948](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-28948)
* [CVE-2022-30580](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30580)
* [CVE-2022-30630](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30630)
* [CVE-2022-30631](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30631)
* [CVE-2022-30632](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30632)
* [CVE-2022-30633](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30633)
* [CVE-2022-30635](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30635)
* [CVE-2022-32189](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-32189)
* [CVE-2022-41715](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-41715)
* [CVE-2022-41724](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-41724)
* [CVE-2022-41725](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-41725)
* [CVE-2023-5717](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-5717)
* [CVE-2023-6040](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6040)
* [CVE-2023-6356](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6356)
* [CVE-2023-6536](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6536)
* [CVE-2023-6606](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6606)
* [CVE-2023-6931](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6931)
* [CVE-2023-6932](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6932)
* [CVE-2023-24534](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-24534)
* [CVE-2023-24536](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-24536)
* [CVE-2023-24537](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-24537)
* [CVE-2023-24539](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-24539)
* [CVE-2023-29400](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-29400)
* [CVE-2023-29403](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-29403)
* [CVE-2023-29499](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-29499)
* [CVE-2023-35827](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-35827)
* [CVE-2023-46838](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-46838)
* [CVE-2023-51780](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-51780)
* [CVE-2023-51781](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-51781)
* [CVE-2023-51782](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-51782)
* [CVE-2023-52436](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52436)
* [CVE-2023-52439](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52439)
* [CVE-2023-52444](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52444)
* [CVE-2023-52445](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52445)
* [CVE-2023-52451](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52451)
* [CVE-2023-52464](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52464)
* [CVE-2023-52469](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52469)
* [CVE-2024-1086](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-1086)
* [CVE-2024-26586](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-26586)
* [CVE-2024-26597](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-26597)
* [CVE-2024-26598](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-26598)
* Medium-severity container vulnerabilities:
* [CVE-2020-29509](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-29509)
* [CVE-2020-29511](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-29511)
* [CVE-2021-33197](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-33197)
* [CVE-2021-34558](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-34558)
* [CVE-2021-36221](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-36221)
* [CVE-2021-44879](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44879)
* [CVE-2022-1705](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1705)
* [CVE-2022-1962](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1962)
* [CVE-2022-32148](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-32148)
* [CVE-2022-41717](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-41717)
* [CVE-2023-3446](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-3446)
* [CVE-2023-3817](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-3817)
* [CVE-2023-6004](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6004)
* [CVE-2023-6121](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6121)
* [CVE-2023-6915](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6915)
* [CVE-2023-6918](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6918)
* [CVE-2023-24532](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-24532)
* [CVE-2023-29406](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-29406)
* [CVE-2023-29409](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-29409)
* [CVE-2023-32611](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-32611)
* [CVE-2023-32665](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-32665)
* [CVE-2023-34324](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-34324)
* [CVE-2023-39198](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-39198)
* [CVE-2023-39804](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-39804)
* [CVE-2023-45863](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-45863)
* [CVE-2023-46218](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-46218)
* [CVE-2023-46343](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-46343)
* [CVE-2023-49290](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-49290)
* [CVE-2023-52443](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52443)
* [CVE-2023-52449](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52449)
* [CVE-2023-52470](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52470)
* [CVE-2024-21664](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21664)
* [CVE-2024-28085](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-28085)
* [GHSA-2c7c-3mj9-8fqh](https://github.com/advisories/GHSA-2c7c-3mj9-8fqh)
* Low-severity container vulnerabilities:
* [CVE-2021-25743](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-25743)
* [CVE-2022-30629](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30629)
* [CVE-2023-26604](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-26604)
* [CVE-2023-2975](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-2975)
* [CVE-2023-5178](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-5178)
* [CVE-2023-5197](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-5197)
* [CVE-2023-6531](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6531)
* [CVE-2023-6817](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-6817)
* [CVE-2023-46813](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-46813)
* [CVE-2023-46862](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-46862)
* [CVE-2023-52438](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52438)
* [CVE-2023-52448](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52448)
* [CVE-2023-52454](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52454)
* [CVE-2023-52456](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52456)
* [CVE-2023-52457](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52457)
* [CVE-2023-52462](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52462)
* [CVE-2023-52463](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52463)
* [CVE-2023-52467](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52467)
* [CVE-2023-52503](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52503)
* [CVE-2023-52513](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52513)
* [CVE-2023-52524](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52524)
* [CVE-2023-52564](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52564)
* [CVE-2023-52573](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52573)
* [CVE-2023-52575](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-52575)
* [CVE-2024-0193](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-0193)
* [CVE-2024-0641](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-0641)
* [CVE-2024-0646](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-0646)
* [CVE-2024-24860](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-24860)
## Issue
**Known issues:**
* Clusters that use bundled load balancing with BGP might have [performance degradation as the total number of Services of type LoadBalancer approaches 2,000](https://cloud.google.com/anthos/clusters/docs/bare-metal/latest/troubleshooting/known-issues#perf-degrad-bgp-lb).
For information about the latest known issues, see [GKE on Bare Metal known issues](https://cloud.google.com/anthos/clusters/docs/bare-metal/latest/troubleshooting/known-issues) in the Troubleshooting section.