Configure streaming for LLM responses and other traffic You can now create API Gateway gateways that stream requests and responses instead of buffering them
Share
Services
## Feature
Feature
**Configure streaming for LLM responses and other traffic**
You can now create API Gateway gateways that stream requests and responses instead of buffering them. This Public Preview feature supports incremental response delivery over HTTP/2 or HTTP/1.1 chunked transfer encoding, Server-Sent Events (SSE), WebSockets, and gRPC bidirectional streaming. A common use is streaming token-by-token responses from a large language model (LLM).
To enable streaming, create a gateway with the `--enable-streaming` flag. The streaming mode is fixed when you create the gateway and can't be changed later.
For more information, see [Configure streaming for LLM responses and other traffic](https://cloud.google.com/api-gateway/docs/streaming-configure).
What else is happening at Google Cloud Platform?
Read update
Services
Share
You can configure Direct VPC ingress for Cloud Run instances
about 1 hour ago
Services
Share
Read update
Services
Share
Read update
Services
Share
Patch updates to supported PHP runtimes are available for Cloud Run resources deployed from source code
about 21 hours ago
Services
Share