Calling Google Cloud APIs From a Streaming Gateway in Practice

Streaming gateways are becoming common in applications that need to connect clients to backend services while processing responses incrementally. A typical application may have a browser or mobile client, a streaming gateway, an ASP.NET Core service, and one or more Google Cloud APIs behind it.

This architecture can improve control over authentication, request routing, observability, and response handling. But it also introduces another network hop that must correctly handle headers, authentication, timeouts, cancellation, and streaming data.

This article walks through a practical approach to calling Google Cloud APIs from an application that sits behind a streaming gateway.

Understanding the Request Flow

A simple architecture can look like this:

Client
   |
   v
Streaming Gateway
   |
   v
ASP.NET Core API
   |
   v
Google Cloud API

The gateway does not necessarily call Google Cloud directly. In many applications, the gateway forwards the request to an ASP.NET Core backend, which then authenticates and calls the Google Cloud service.

For example:

Client
   |
   | HTTP request
   v
Streaming Gateway
   |
   | Forwarded request
   v
ASP.NET Core
   |
   | Authenticated Google API request
   v
Google Cloud
   |
   | Response
   v
ASP.NET Core
   |
   | Streaming response
   v
Streaming Gateway
   |
   v
Client

The important point is that streaming at the outer HTTP layer does not automatically mean that every internal call is streaming.

Your application should decide where streaming is actually required.

Why Put a Gateway in Front of the API?

A gateway can provide a common entry point for multiple clients and backend services.

Common responsibilities include:

Authentication and authorization
Request routing
Rate limiting
Request validation
Logging
Correlation IDs
Traffic control
API versioning
Response streaming

For example, a client might call:

POST /api/generate

The gateway forwards the request to an ASP.NET Core service:

POST /internal/generate

The ASP.NET Core service then calls the required Google Cloud API.

This keeps Google Cloud credentials and service-specific implementation details on the server instead of exposing them to the client.

Keep Google Cloud Authentication on the Server

A common mistake is to treat a Google Cloud API key or credential as something that can be safely forwarded through the client.

For server-side workloads, authentication should normally be handled by the backend using an appropriate Google Cloud identity mechanism.

For example, an ASP.NET Core service can use application default credentials:

using Google.Apis.Auth.OAuth2;

var credential =
    await GoogleCredential.GetApplicationDefaultAsync();

var accessToken =
    await credential.UnderlyingCredential
        .GetAccessTokenForRequestAsync();

The exact authentication method depends on the Google Cloud service and deployment environment.

The important architectural rule is:

Client
   |
   | No Google credentials
   v
Gateway
   |
   v
Backend
   |
   | Google authentication
   v
Google Cloud API

Do not forward service-account credentials through the gateway.

Calling a Google Cloud API From ASP.NET Core

The backend can use HttpClient when the Google Cloud API exposes a REST endpoint.

A simple service can look like this:

public sealed class GoogleCloudService
{
    private readonly HttpClient _httpClient;

    public GoogleCloudService(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<HttpResponseMessage> SendAsync(
        string endpoint,
        HttpContent content,
        CancellationToken cancellationToken)
    {
        using var request = new HttpRequestMessage(
            HttpMethod.Post,
            endpoint);

        request.Content = content;

        return await _httpClient.SendAsync(
            request,
            HttpCompletionOption.ResponseHeadersRead,
            cancellationToken);
    }
}

The important option here is:

HttpCompletionOption.ResponseHeadersRead

This tells HttpClient not to wait for the entire response body before returning.

That becomes important when the downstream Google service produces a response that needs to be processed incrementally.

Forwarding the Request Through the Gateway

Suppose the client sends:

POST /api/generate
Content-Type: application/json

The gateway forwards the request to the ASP.NET Core service.

The backend can then construct the Google Cloud request.

For example:

[HttpPost("generate")]
public async Task<IActionResult> Generate(
    [FromBody] GenerateRequest request,
    CancellationToken cancellationToken)
{
    var result = await _googleCloudService.GenerateAsync(
        request,
        cancellationToken);

    return Ok(result);
}

For a normal request-response API, this is enough.

For streaming, the response handling needs to be different.

Reading a Streaming Response

When the downstream service returns data progressively, avoid reading the complete response into memory.

Instead:

using var response = await _httpClient.SendAsync(
    request,
    HttpCompletionOption.ResponseHeadersRead,
    cancellationToken);

response.EnsureSuccessStatusCode();

await using var stream =
    await response.Content.ReadAsStreamAsync(cancellationToken);

using var reader = new StreamReader(stream);

while (!reader.EndOfStream)
{
    var line = await reader.ReadLineAsync(
        cancellationToken);

    if (line is null)
        break;

    await ProcessChunkAsync(line, cancellationToken);
}

This approach is useful when the downstream response is delivered as a sequence of chunks, lines, or events.

The actual parsing logic depends on the response format used by the Google Cloud service.

Streaming the Response Back to the Client

After receiving data from Google Cloud, the ASP.NET Core application can write the data to the outgoing HTTP response.

For example:

Response.ContentType = "text/event-stream";

await foreach (var chunk in GetResponseChunksAsync(
    cancellationToken))
{
    await Response.WriteAsync(
        $"data: {chunk}\n\n",
        cancellationToken);

    await Response.Body.FlushAsync(
        cancellationToken);
}

FlushAsync is important when the application needs the data to leave the application buffer rather than waiting for a larger response buffer to fill.

The complete flow becomes:

Google Cloud
     |
     | response chunk
     v
ASP.NET Core
     |
     | write + flush
     v
Streaming Gateway
     |
     | forward
     v
Client

However, the gateway itself must also support streaming correctly.

Do Not Assume the Gateway Streams Automatically

A frequent production problem is that the backend sends chunks correctly, but the client receives them only after the complete response is available.

This usually means buffering exists somewhere in the path.

Possible buffering layers include:

Client
   |
Gateway
   |
Load Balancer
   |
ASP.NET Core
   |
HttpClient
   |
Google Cloud

Every layer should be checked.

If the backend calls:

await Response.Body.FlushAsync(cancellationToken);

but the gateway buffers the response, the client may still experience delayed output.

Streaming therefore needs to be treated as an end-to-end behavior rather than an ASP.NET Core setting.

Handling Authentication Tokens

Access tokens should be generated and attached by the backend when required.

For example:

var credential =
    await GoogleCredential.GetApplicationDefaultAsync();

var token =
    await credential.UnderlyingCredential
        .GetAccessTokenForRequestAsync();

request.Headers.Authorization =
    new AuthenticationHeaderValue(
        "Bearer",
        token);

Do not log the resulting token.

Also avoid copying authorization headers blindly from the incoming client request.

Instead, explicitly control which headers are forwarded.

For example:

var allowedHeaders = new[]
{
    "Accept",
    "Content-Type",
    "X-Correlation-Id"
};

This is safer than forwarding every incoming header.

Propagating Cancellation

Streaming requests can remain open for a long time.

If the user closes the browser or cancels the request, the backend should stop unnecessary downstream work.

ASP.NET Core provides a cancellation token for this purpose:

public async Task Stream(
    CancellationToken cancellationToken)
{
    await _googleCloudService.StreamAsync(
        cancellationToken);
}

Pass the same cancellation token to the downstream request:

await _httpClient.SendAsync(
    request,
    HttpCompletionOption.ResponseHeadersRead,
    cancellationToken);

And pass it while reading:

await response.Content.ReadAsStreamAsync(
    cancellationToken);

This prevents the backend from continuing an expensive Google Cloud operation after the client has already disconnected, where the downstream service supports cancellation.

Managing Timeouts and Deadlines

Streaming requests require different timeout thinking from ordinary APIs.

A request that takes 30 seconds may be normal for one workload but unacceptable for another.

Avoid relying on a single global timeout for every request.

For example:

services.AddHttpClient<GoogleCloudService>(client =>
{
    client.Timeout = TimeSpan.FromMinutes(5);
});

The appropriate timeout should be based on the actual workload.

Also remember that the total request path may contain multiple timeout controls:

Client timeout
      |
Gateway timeout
      |
Load balancer timeout
      |
ASP.NET Core timeout
      |
HttpClient timeout
      |
Google Cloud deadline

The shortest timeout can determine the effective behavior.

If the gateway allows five minutes but the backend HttpClient times out after one minute, increasing the gateway timeout will not solve the problem.

Avoiding Retry Problems With Streaming Requests

Retries require extra care when the request has side effects or when a response has already started streaming.

A simple retry policy may work well for a short idempotent request:

Request
  |
Failure
  |
Retry

But streaming changes the situation.

For example:

Request starts
     |
Response headers received
     |
20 chunks delivered
     |
Connection fails
     |
Should the request be retried?

Automatically retrying may execute the operation twice or produce duplicated output.

For streaming operations, define retry behavior based on the operation semantics.

Good candidates for retries are generally transient failures that occur before meaningful response processing begins.

Once a stream has started producing application data, recovery may need to happen at the application level.

Logging Without Logging Sensitive Data

Streaming requests can generate a large amount of diagnostic information.

Avoid logging every response chunk in production.

Instead, log metadata such as:

Request ID
Correlation ID
Google Cloud service
Operation
Start time
Time to first byte
Total duration
HTTP status
Bytes processed
Failure reason

For example:

_logger.LogInformation(
    "Google Cloud request completed. Operation={Operation}, " +
    "StatusCode={StatusCode}, DurationMs={DurationMs}",
    operation,
    response.StatusCode,
    stopwatch.ElapsedMilliseconds);

Do not log access tokens, credentials, sensitive request bodies, or potentially sensitive generated content.

Tracking Time to First Byte

For streaming workloads, total response time is not the only useful metric.

Consider:

Request started
      |
      | 400 ms
      v
First response chunk
      |
      | 8 seconds
      v
Final response chunk

The user experiences the first 400 ms differently from waiting eight seconds for the complete response.

Track at least:

Time to first byte
Time to first meaningful chunk
Total duration
Number of chunks
Chunk processing time
Downstream HTTP status
Cancellation count
Timeout count

This helps determine whether a delay originates in Google Cloud, ASP.NET Core, or the gateway.

Handling Downstream Errors

Do not assume every failure will arrive as a normal JSON response.

Errors can occur at different stages.

Before the response starts

The backend may receive:

401 Unauthorized

or:

429 Too Many Requests

The application can return an appropriate HTTP response to the client.

After streaming starts

The situation is different.

Suppose the application already sent:

data: first result

data: second result

and then Google Cloud returns an error.

The HTTP status code may no longer be changeable because response headers have already been sent.

The application needs an application-level error event or another protocol-specific mechanism.

For example:

data: first result

data: second result

event: error
data: downstream operation failed

The client should understand this protocol.

Using Correlation IDs

A request passing through multiple systems should have a correlation identifier.

For example:

Client
  correlation-id: 7f91...
       |
       v
Gateway
       |
       v
ASP.NET Core
       |
       v
Google Cloud

Generate a correlation ID when necessary and propagate it through the systems that support it.

Example:

request.Headers.TryAddWithoutValidation(
    "X-Correlation-Id",
    correlationId);

This makes it easier to connect gateway logs, application logs, and downstream request logs.

Common Problems and Their Causes

Problem	Likely Cause	What to Check
Client receives everything at once	Gateway buffering	Gateway response configuration
Google API returns 401	Authentication issue	Service identity and scopes
Request ends too early	Timeout/deadline	Gateway and HttpClient timeouts
Duplicate operation	Aggressive retry	Retry policy
Stream stops after client disconnects	Cancellation not propagated	CancellationToken
First chunk is slow	Backend or downstream startup	Time to first byte
Large memory usage	Entire response buffered	`ResponseHeadersRead` and streaming reads
Intermittent 429	Rate limiting	Quotas and retry strategy
Errors after partial response are unclear	Response already started	Application-level error events

Practical Production Checklist

Before putting a streaming gateway and Google Cloud integration into production, verify:

Google Cloud authentication is handled by the backend.
Credentials are never sent to the client.
Only required headers are forwarded.
ResponseHeadersRead is used where incremental response processing is required.
Response data is not unnecessarily buffered.
Cancellation is propagated from the client to downstream calls.
Gateway and backend timeout values are compatible.
Retry policies account for streaming and side effects.
Sensitive request and response data is not logged.
Correlation IDs are available across service boundaries.
Time to first byte is measured.
The gateway is confirmed to preserve streaming behavior.
Downstream errors after streaming begins have a defined protocol.
Large responses are tested under realistic load.

Advantages

A streaming gateway architecture can provide several practical benefits:

Centralized authentication and routing
Consistent API policies
Better control over client access
Incremental response delivery
Reduced memory pressure for large responses
Centralized observability
Easier integration of multiple backend services

It also allows the backend to keep Google Cloud credentials and service-specific implementation details away from client applications.

Disadvantages

The architecture also introduces additional operational complexity:

More network hops
More timeout configurations
Gateway buffering risks
More complicated retry behavior
More difficult error handling after streaming begins
Additional monitoring requirements
Potential latency from gateway processing

For a simple API that does not require streaming, introducing a gateway may add complexity without providing enough value.

Conclusion

Calling Google Cloud APIs through a streaming gateway requires more than forwarding an HTTP request.

The complete request path needs to be designed as a system. Authentication should remain on the server, downstream responses should be consumed incrementally when appropriate, cancellation should flow through the request chain, and timeout policies should be aligned across every layer.

The most important production lesson is to test streaming end to end.

A backend may correctly stream data while the gateway buffers it. An application may correctly handle cancellation while the downstream operation continues running. A Google Cloud call may succeed while the client still experiences a timeout because another layer has a shorter deadline.

By treating the gateway, ASP.NET Core service, and Google Cloud API as one request pipeline, you can build a streaming integration that is easier to troubleshoot and more predictable under production workloads.