Connecting BigQuery to Databricks and Snowflake Data Without Moving the Data

as relevant permissions for querying federated Lakehouse data.

The source system also needs to authorize the connection.

A successful BigQuery permission configuration does not automatically mean that the user can access the source data.

5. Discover the federated tables

Once the connection and catalog configuration are available, the external catalog can be represented through the BigQuery Lakehouse environment.

The goal is to make external tables discoverable without creating a second physical dataset.

6. Query the data

The developer can then build analytical queries against the federated tables.

For example:

SELECT
    customer_id,
    SUM(order_amount) AS total_orders
FROM federated_sales.orders
WHERE order_date >= DATE '2026-01-01'
GROUP BY customer_id
ORDER BY total_orders DESC;

The exact table path and SQL syntax depend on the configured BigQuery Lakehouse resource.

The important architectural point is that the query references the federated table rather than an ETL-created copy.

Joining BigQuery Data With Federated Data

One of the most useful patterns is combining native BigQuery data with external data.

For example:

BigQuery
Customer dimension
        |
        | customer_id
        |
        +------ Join ------+
                           |
                     Databricks
                     Sales data

A query can conceptually look like:

SELECT
    c.customer_id,
    c.customer_name,
    SUM(o.amount) AS revenue
FROM `analytics.customers` AS c
JOIN `federated_sales.orders` AS o
    ON c.customer_id = o.customer_id
WHERE o.order_date >= DATE '2026-01-01'
GROUP BY
    c.customer_id,
    c.customer_name;

This pattern becomes useful when the authoritative customer data is already in BigQuery but detailed transactional data remains in another platform.

The organization can combine the datasets without first creating another full copy.

Query Performance Still Matters

Federation does not automatically make every cross-platform query fast.

There are additional layers involved:

BigQuery query planner
        |
Catalog metadata
        |
Network access
        |
External storage
        |
File scanning

A query that reads a small amount of data can work very well.

A query that scans terabytes of remote data repeatedly may require a different architecture.

This is why query design remains important.

Use filters early:

WHERE order_date >= DATE '2026-01-01'

Select only required columns:

SELECT customer_id, amount

instead of:

SELECT *

Also take advantage of partitioning, clustering, and table statistics supported by the underlying data architecture.

Security Considerations

A zero-copy architecture does not remove security requirements.

You still need to control:

• Who can discover the catalog
• Who can query the tables
• Which service account accesses the data
• Which network paths are permitted
• Which source-level policies apply
• How credentials are issued
• How query activity is audited

A useful production model is:

Developer
    |
    v
BigQuery IAM
    |
    v
Federated Connection
    |
    v
Source Catalog Authorization
    |
    v
Source Storage Permissions

Every layer should be reviewed independently.

Do not give developers broad storage permissions simply because a federated query needs read access.

Data Freshness

One of the biggest benefits of querying source data directly is avoiding synchronization delays.

However, freshness depends on how the source catalog and metadata are synchronized.

For example, Google Cloud's Databricks integration uses Iceberg metadata, and changes can become visible after the configured metadata refresh process. Google Cloud describes refresh intervals of as little as five minutes for the relevant Lakehouse integration.

Therefore, do not describe federated data as automatically equivalent to real-time data.

Instead, define a freshness expectation:

Source update
      |
      v
Metadata refresh
      |
      v
Federated table visibility
      |
      v
BigQuery query

This distinction matters for dashboards, operational reporting, and AI applications.

Common Mistakes

Treating federation like a replacement for every ETL pipeline

Federation is useful when data should remain in place.

It is not always the best choice for high-volume workloads that repeatedly scan the same remote data.

Ignoring network location

Cross-cloud queries can introduce additional network and latency considerations.

Keep related workloads geographically close when possible and understand where the underlying data resides.

Using SELECT * on large tables

This can dramatically increase the amount of data that must be read.

Prefer:

SELECT
    customer_id,
    order_date,
    amount

instead of:

SELECT *

Ignoring schema evolution

A source team may add, rename, or change columns.

Before putting a federated table behind production dashboards or applications, define how schema changes will be detected and handled.

Assuming SQL behavior is identical everywhere

BigQuery, Databricks, and Snowflake each have their own SQL dialects and execution behavior.

A query that works in Snowflake may require changes before it works efficiently in BigQuery.

Keep business logic simple at the federation boundary and test important queries against production-like data.

Federation vs Data Replication

The choice should be based on workload rather than ideology.

When Federation Is a Good Fit

Federated access is a strong option when:

• The source data should remain authoritative in its existing platform.
• Teams need analytical access without creating another copy.
• Data freshness is important.
• The workload reads selected portions of the source data.
• Multiple analytics engines need access to the same open-format data.
• Governance can be established across the source and query platform.

When Replication Is Still Better

A replicated BigQuery dataset can make more sense when:

• The same data is queried continuously.
• Queries scan large volumes repeatedly.
• Predictable query performance is more important than avoiding duplication.
• The source platform cannot provide the required federation path.
• The application requires BigQuery-native features that are not available for federated data.
• A separate analytical snapshot is required for compliance or disaster recovery.

In practice, many organizations will use both approaches.

For example:

                 Source data
                     |
          +----------+----------+
          |                     |
      Federation            Replication
          |                     |
      Ad hoc / fresh        High-volume BI
          |                     |
       BigQuery             BigQuery

Production Checklist

Before moving a federated architecture into production, verify:

[ ] Source catalog is supported
[ ] Table format is compatible
[ ] Cross-cloud connection is configured
[ ] IAM permissions are reviewed
[ ] Source permissions are reviewed
[ ] Data location and region are documented
[ ] Metadata refresh behavior is understood
[ ] Query performance has been tested
[ ] Large-table queries use appropriate filters
[ ] Schema evolution is monitored
[ ] Audit logging is enabled
[ ] Cost ownership is clearly defined
[ ] Failure behavior has been tested
[ ] Replication fallback is understood

Advantages and Disadvantages

Advantages

• Avoids unnecessary permanent copies of data.
• Reduces dependency on synchronization pipelines.
• Improves access to current source data.
• Allows teams to keep existing data platforms.
• Supports multi-cloud analytics.
• Makes open table formats more useful across engines.
• Can simplify data architecture for selected workloads.

Disadvantages

• Cross-cloud architecture is more complex.
• Query performance depends on remote data access.
• Network and cloud boundaries still matter.
• Source catalog availability becomes part of the query path.
• Security requires coordination between multiple platforms.
• Not every table or workload is suitable for federation.
• SQL and feature differences between platforms can affect portability.

Final Thoughts

Connecting BigQuery with Databricks and Snowflake does not have to mean building another pipeline that copies the same data into another warehouse.

With modern lakehouse federation and open table formats such as Apache Iceberg, organizations can expose data through supported external catalogs while keeping the underlying data in place. BigQuery can then act as another analytical engine over that data rather than becoming another mandatory storage destination.

The most important design decision is not whether federation is better than replication.

It is deciding which data should remain in place, which workloads need native BigQuery performance, and where a physical copy actually provides value.

For ad hoc analysis, current-data access, and cross-platform analytics, federation can remove unnecessary data pipelines. For workloads that repeatedly scan large datasets, native BigQuery storage may still be the better choice.

A practical architecture often combines both approaches: federate data when it makes sense, replicate selectively when performance, isolation, or operational requirements justify the copy.