Many organizations store their data across multiple cloud platforms. An application might generate transaction data in Amazon S3, store customer information in Azure, and use Google Cloud for analytics and reporting.
The problem starts when teams need to analyze all this data together. Traditionally, they build data pipelines to copy files from one cloud to another before running queries. These pipelines require storage, network bandwidth, scheduling, and ongoing maintenance.
Google Cloud's Lakehouse cross-cloud data access provides another option. It allows BigQuery to query supported Apache Iceberg tables stored in other cloud environments without first migrating the underlying data files.
It also provides cross-cloud caching. When a query reads data from remote storage, Lakehouse can temporarily cache the required file blocks in Google Cloud. Subsequent queries can reuse those blocks instead of retrieving them from the remote cloud again.
In this article, we will look at how BigQuery Lakehouse caching works, how it reduces repeated cross-cloud data transfers, and what to consider when using it in an analytics environment.
The Problem With Moving Data Between Clouds
Consider an organization that stores its sales and transaction data in Amazon S3. Its analytics team uses BigQuery for dashboards, reporting, and business analysis.
Without direct cross-cloud access, the team might build a pipeline like this:
- Read data files from Amazon S3.
- Transfer the files to Google Cloud Storage.
- Load or process the data in BigQuery.
- Run analytical queries.
- Repeat the process whenever new data arrives.
This approach works, but it introduces additional responsibilities.
| Challenge | Impact |
|---|
| Data duplication | The same data exists in multiple cloud environments |
| Transfer costs | Moving large datasets creates network charges |
| Pipeline maintenance | Data ingestion and synchronization require ongoing work |
| Data freshness | Reports might use data from the previous pipeline run |
| Storage costs | Teams pay to maintain additional copies |
For frequently updated datasets, keeping multiple copies synchronized can become difficult.
BigQuery Lakehouse cross-cloud data access addresses this problem by allowing supported remote tables to be queried directly.
How BigQuery Lakehouse Cross-Cloud Caching Works
Google Cloud Lakehouse uses Apache Iceberg and remote catalog integrations to access data stored in other environments.
For example, an organization might use AWS Glue Data Catalog to manage Iceberg tables stored in Amazon S3. BigQuery can access these tables through Lakehouse without requiring the organization to move the underlying files into Google Cloud.
The process has three main parts.
Metadata discovery
Lakehouse connects to the remote Apache Iceberg REST catalog to discover namespaces, tables, schemas, and other metadata.
Supported catalog providers include:
- AWS Glue Data Catalog.
- Databricks Unity Catalog.
- Snowflake Horizon Catalog.
- Workday Data Lake.
- SAP Business Data Cloud.
Authentication can use credentials stored in Secret Manager or OpenID Connect (OIDC) token federation, depending on the catalog provider.
Reading remote data
When a BigQuery query accesses a remote table, Lakehouse uses the table metadata to identify the required data files.
The query reads the required file blocks from the remote object storage. Depending on your network configuration, this traffic can travel over the public internet or a private Cross-Cloud Interconnect.
Reusing cached file blocks
When cross-cloud caching is enabled, Lakehouse temporarily stores retrieved file blocks in the Google Cloud region where the query executes.
If another query needs the same blocks, Lakehouse can read them from the local cache instead of fetching them from Amazon S3 or Azure Blob Storage again.
This is useful when multiple reports access the same datasets or when users repeatedly run queries against the same data.
Understand the Architecture With an Example
Imagine an e-commerce company stores its order history in Amazon S3 using Apache Iceberg tables.
Its analytics team uses BigQuery to generate daily sales reports.
The architecture looks like this:
Amazon Web Services
+------------------------+
| AWS Glue Data Catalog |
| |
| Amazon S3 |
| Iceberg / Parquet Data |
+-----------+------------+
|
| Remote data access
|
v
+------------------------+
| Google Cloud Lakehouse |
| |
| Metadata Discovery |
| Cross-Cloud Connection |
| Local Block Cache |
+-----------+------------+
|
v
+------------------------+
| BigQuery |
| |
| SQL Analytics |
| Dashboards |
| Reporting |
+------------------------+
The important point is that the original Iceberg data files remain in Amazon S3.
Lakehouse reads the required blocks and can keep temporary cached copies in Google Cloud. It does not require a separate data migration pipeline just to make the remote tables available for querying.
The cache is not a replacement for the original storage system. Amazon S3 remains the source of the data.
Enable Cross-Cloud Caching
Cross-cloud caching is available for supported Lakehouse federated catalogs and BigQuery cross-cloud connections.
It is disabled by default, so you need to enable it when creating or updating the relevant catalog or connection.
The general setup process is:
Prepare the remote data
Make sure your remote data is available through a supported Apache Iceberg catalog.
For an AWS environment, this might involve:
- An AWS Glue Data Catalog.
- Apache Iceberg tables stored in Amazon S3.
- An AWS account and Google Cloud project configured for cross-cloud access.
- Appropriate permissions for catalog discovery and object access.
For Azure environments, the remote data can be stored in Azure Data Lake Storage Gen2 and accessed through a supported catalog integration.
Configure the federated catalog
In Google Cloud, configure the Lakehouse federated catalog using the supported provider setup.
The configuration includes the remote catalog details, authentication method, data location, and metadata refresh settings.
For AWS Glue, Databricks Unity Catalog, and other supported providers, follow the relevant provider-specific setup instructions.
Enable caching
Enable cross-cloud caching on the federated catalog or BigQuery cross-cloud connection.
Google Cloud manages the cache capacity, retention, and eviction automatically.
You do not need to create a separate storage bucket or maintain your own cache refresh process.
One important detail: cross-cloud caching is documented as having no additional charge. However, remote data transfers, interconnect usage, and other cloud services can still incur costs.
Query the remote table
After the catalog is configured, the remote tables become available through the Lakehouse integration.
You can use GoogleSQL to query the table.
For example, assume the remote Iceberg table is exposed in BigQuery as sales_lake.orders.
SELECT
customer_id,
COUNT(*) AS total_orders,
SUM(order_amount) AS total_sales
FROM
`sales_lake.orders`
WHERE
order_date >= DATE '2026-10-01'
GROUP BY
customer_id
ORDER BY
total_sales DESC
LIMIT 20;
The example uses a representative table reference. The actual project, dataset, and table path depend on how your federated catalog is configured.
When the query reads remote data, Lakehouse can populate the local cache with the required file blocks if caching is enabled.
A later query that needs the same blocks can reuse the cached data.
Measure Cache Usage and Data Transfer
Enabling caching does not automatically tell you how much data transfer it has saved.
You should monitor actual cache usage through BigQuery job statistics.
Google Cloud exposes the objectStorageStats field in the BigQuery JobStatistics2 query job statistics.
Two useful fields are:
| Field | Meaning |
|---|
| cacheBytesRead | Bytes read from the local Google Cloud cache |
| objectStorageBytesRead | Bytes read directly from remote object storage |
For example, suppose a query reports the following values:
| Metric | Example value |
|---|
| Cache bytes read | 8 GB |
| Remote storage bytes read | 2 GB |
| Total bytes read from these sources | 10 GB |
This indicates that 8 GB came from the local cache and 2 GB came directly from remote storage.
These numbers help you understand how much of the query's object data was served locally.
You can inspect the job details in the BigQuery console or retrieve the statistics through the BigQuery API.
For workloads with repeated access to the same data, monitor these metrics over several days rather than judging the results from a single query.
Important Considerations Before Enabling Caching
Cross-cloud caching can improve repeated query performance, but it is important to understand its limitations.
Cache hits depend on query patterns
Caching helps most when queries repeatedly access the same file blocks.
For example, multiple dashboards might read the same monthly sales data. Those queries can benefit from previously cached blocks.
However, a query that scans completely different files every time might see fewer cache hits.
Source data changes
When the underlying files change, the query still needs to use the current metadata and permissions from the remote source.
Lakehouse checks metadata and access permissions before serving cached data. This helps prevent stale data from being returned and prevents cached blocks from bypassing revoked permissions.
Data residency requirements
Cached file blocks are temporarily stored in the Google Cloud region where the query executes.
This matters when the original data is stored in another country or jurisdiction.
For example, an organization might store customer data in an EU region but execute queries in a US region. Before enabling caching, review whether temporary cached copies in the query region comply with the organization's data residency requirements.
Encryption requirements
Cross-cloud cached data is encrypted at rest using Google-owned and Google-managed encryption keys.
Customer-managed encryption keys (CMEK) are not supported for this cross-cloud caching feature.
If your organization enforces the constraints/gcp.restrictNonCmekServices organization policy, Lakehouse automatically disables caching for queries that access restricted tables. The queries can still run, but they read directly from remote storage.
Network configuration
Caching does not remove the need for secure access to the remote catalog and storage.
You can use the public internet with TLS or configure private connectivity through Dedicated Cross-Cloud Interconnect or Partner Interconnect.
Private connectivity can provide more predictable network performance and may reduce transfer costs compared with public internet access.
Common Mistakes to Avoid
Confusing caching with data migration
Cross-cloud caching does not create a permanent copy of the complete dataset in BigQuery. It temporarily caches the file blocks that queries retrieve.
Expecting every query to use cached data
A query that accesses different files might need to retrieve those files from the remote storage again. Cache benefits depend on access patterns and cache retention.
Ignoring data residency
Temporary cached copies are stored in the Google Cloud query execution region. Review regional and compliance requirements before enabling the feature.
Assuming caching eliminates all transfer costs
The first read of a file block can still require a remote transfer. Cache misses and changing data can also result in additional remote reads.
Confusing different BigQuery caching features
BigQuery has several caching mechanisms, and they serve different purposes.
| Feature | What it caches |
|---|
| BigQuery query result cache | Results of previously executed queries |
| BigLake metadata cache | External file metadata, such as filenames and partition information |
| Lakehouse cross-cloud cache | Retrieved data blocks from remote cloud storage |
These features should not be treated as interchangeable.
Summary
BigQuery Lakehouse cross-cloud caching helps organizations analyze data stored in Amazon S3 and Azure without first copying the complete dataset into Google Cloud. It uses remote Apache Iceberg catalogs to discover tables and reads the required data blocks when queries execute. When caching is enabled, repeated queries can reuse locally cached blocks, reducing remote data transfers and improving performance. Before enabling it, review query patterns, network connectivity, data residency, encryption requirements, and actual cache usage. This approach is useful for organizations that want to keep their data in its original cloud while using BigQuery for analytics and reporting.
Join the conversation! Your thoughts help the community grow.