Modern applications often need to work with two very different types of data.
Operational data belongs in systems designed for fast application transactions. Customer records, orders, payments, inventory, and account information are common examples.
Analytical data has different requirements. Large event histories, logs, clickstream records, historical transactions, and machine-generated data may need to be stored for long periods and queried across large datasets.
Apache Iceberg is designed for this analytical side of the architecture. It provides a table format for large analytical datasets stored in object storage and accessed by engines such as Spark, Trino, Flink, and other compatible systems.
PostgreSQL, on the other hand, is commonly used by applications for transactional workloads. Developers naturally want to query operational and analytical data together, but copying everything from an Iceberg lakehouse into PostgreSQL is usually not a good solution.
A better architecture is to keep each workload in the system designed for it and provide a controlled way for PostgreSQL workloads to access Iceberg data when required.
This article explains the architecture behind querying Iceberg data from PostgreSQL-oriented workloads, how the data path works, where PostgreSQL fits, what the performance considerations are, and when this approach makes sense.
What Is Apache Iceberg?
Apache Iceberg is an open table format designed for large analytical datasets.
It is not a database server.
An Iceberg table normally consists of data files stored in object storage together with metadata describing the table.
A simplified architecture looks like this:
Application
|
v
PostgreSQL
|
| Operational queries
v
Transactional tables
Analytical workloads
|
v
Iceberg table
|
v
Object Storage
|
v
Data filesThe important difference is that Iceberg provides table-level metadata and consistency on top of data stored in files.
Instead of treating a directory of Parquet files as an unmanaged collection of files, an Iceberg table keeps track of the table's structure, snapshots, schemas, partitions, and changes.
This makes large analytical datasets easier to manage over time.
Why PostgreSQL Workloads May Need Iceberg Data
Consider an e-commerce application.
PostgreSQL may contain current operational information:
customers
orders
products
payments
inventoryThe company may also have billions of historical events:
page views
search events
product impressions
checkout events
application logs
device events
historical order eventsKeeping all of this historical data inside PostgreSQL may increase storage requirements and make large analytical queries compete with application traffic.
Instead, historical event data can be written into an Iceberg table.
The architecture can then look like this:
+------------------+
| Application |
+--------+---------+
|
v
+------------------+
| PostgreSQL |
| Operational Data|
+------------------+
|
| Analytical access
v
+------------------+
| Iceberg Tables |
+--------+---------+
|
v
+------------------+
| Object Storage |
+------------------+PostgreSQL remains responsible for transactional workloads while the lakehouse handles large analytical datasets.
PostgreSQL and Iceberg Solve Different Problems
It is important not to think of Iceberg as a replacement for PostgreSQL.
The systems are designed around different workload patterns.
Requirement | PostgreSQL | Apache Iceberg |
|---|---|---|
OLTP transactions | Strong fit | Not the primary purpose |
Small point lookups | Strong fit | Usually not the best choice |
Frequent updates | Strong fit | Supported through table operations but optimized differently |
Large analytical scans | Possible but can become expensive | Strong fit |
Object storage | External integration | Core storage model |
Historical analytical data | Possible | Strong fit |
Complex lakehouse workloads | Limited compared with analytical engines | Strong fit |
Application request processing | Strong fit | Usually not the primary workload |
The goal is not to make one system perform every type of workload.
The goal is to connect them when business requirements need data from both.
How Iceberg Tables Store Data
A typical Iceberg table may use Parquet files for the actual data.
A simplified table might look like:
sales/
metadata/
...
data/
part-00001.parquet
part-00002.parquet
part-00003.parquetThe metadata provides information about the table and its snapshots.
The application should not normally treat those files as ordinary files and manually query them.
Instead, an Iceberg-aware engine uses the table metadata to understand which files belong to a particular table snapshot.
This distinction is important.
If a PostgreSQL workload needs Iceberg data, it should use a compatible query layer or connector rather than bypassing Iceberg's metadata management.
PostgreSQL Does Not Directly Become an Iceberg Engine
One of the most important architectural points is that PostgreSQL itself is not automatically an Iceberg query engine.
A PostgreSQL-based application may access Iceberg through another layer.
For example:
PostgreSQL Workload
|
v
Federation / Query Layer
|
v
Iceberg Catalog
|
v
Object Storage
|
v
Parquet DataThe exact implementation depends on the PostgreSQL distribution, extensions, federation technology, and analytical engine used by the environment.
This layer is responsible for translating the query or data access request into operations that the Iceberg storage and query engine can execute.
Therefore, when designing this architecture, do not assume that installing a PostgreSQL extension automatically turns PostgreSQL into a full Iceberg processing engine.
A Practical Query Flow
Suppose the application needs to answer this question:
"How many orders did each customer make during the last quarter?"
The current customer and order information may be stored in PostgreSQL.
Historical event information may be stored in Iceberg.
A federated query architecture could conceptually perform the following:
1. PostgreSQL receives the query
|
v
2. Query planner identifies external data
|
v
3. Federation layer sends analytical operation
|
v
4. Iceberg query engine reads required files
|
v
5. Iceberg engine performs filtering and aggregation
|
v
6. Result is returned to PostgreSQL workloadThe important part is that PostgreSQL does not need to copy the entire Iceberg table into local tables just to answer the question.
Keep Filtering Close to the Iceberg Data
Large analytical datasets can contain billions of rows.
Sending unnecessary data across the network can quickly become expensive.
Suppose the Iceberg table contains five years of order history, but the application only needs one month.
A poor query strategy may scan and transfer a large amount of data before filtering.
A better strategy is to push filtering into the analytical layer:
SELECT
customer_id,
COUNT(*) AS order_count
FROM external_orders
WHERE order_date >= DATE '2026-10-01'
AND order_date < DATE '2026-11-01'
GROUP BY customer_id;The exact SQL support depends on the federation layer and query engine.
The architectural principle is more important than the syntax.
Filter early.
Project only the columns you need.
Aggregate data before transferring large result sets back to PostgreSQL.
This reduces network traffic and improves query efficiency.
Avoid SELECT * for Large Iceberg Tables
A query like this:
SELECT *
FROM external_orders;may be harmless against a small table.
It can be a serious problem against an analytical dataset containing billions of rows.
If the application only needs three columns, request three columns:
SELECT
customer_id,
order_date,
amount
FROM external_orders
WHERE order_date >= DATE '2026-10-01';Columnar formats such as Parquet are particularly useful when queries read only a subset of columns.
The query engine can often avoid reading unrelated columns from the underlying files.
This is one of the basic principles that makes analytical lakehouse storage efficient.
Partitioning and File Layout Matter
Iceberg supports partitioning strategies that help query engines identify relevant data.
For example, a table may be logically partitioned by a transformed date value.
The application may frequently query:
order_date
region
customer_id
event_typeThe table design should reflect actual query patterns rather than simply creating a partition for every available column.
Too many small partitions can create metadata and file-management overhead.
Poor partition choices can also cause large scans.
The important principle is to design the Iceberg table around real analytical access patterns.
Do not treat partitioning as a cosmetic database setting.
Avoid the Small File Problem
Analytical pipelines can accidentally create thousands or millions of tiny files.
For example:
data/
file001.parquet
file002.parquet
file003.parquet
...
file900000.parquetEven if the total data volume is reasonable, the query engine may spend significant time discovering, opening, and processing the files.
This is commonly called the small file problem.
A production pipeline should therefore include appropriate compaction or file management.
Join the conversation! Your thoughts help the community grow.