By Daniel Nadia · Apache SeaTunnel Contributor
Meet SeaTunnel: efficient integration across data sources, built for reliable ongoing synchronization
Apache SeaTunnel is an open-source distributed data integration tool for batch data movement and continuous synchronization. It connects databases, messaging systems, and analytical storage so that operational data can reach reporting systems, data warehouses, and other downstream applications without a separate custom synchronization program for every pair of systems.
Engineering teams usually ask three questions when choosing a synchronization tool: Can it move data efficiently? Can it connect to our existing systems? Can a long-running job recover from failures? These are useful ways to understand SeaTunnel's value:
- Efficient synchronization without repeatedly copying the entire table. With MySQL CDC, for example, SeaTunnel can split a snapshot into chunks for parallel reads. In the incremental phase, it captures changes from the binlog instead of repeatedly scanning the full table. This puts historical initialization and ongoing updates in the same pipeline. Actual throughput still depends on the source database, destination, data distribution, and parallelism settings. MySQL CDC 3.0.0
- A broad connector ecosystem that reduces repetitive integration work. SeaTunnel 3.0.0's project overview lists more than 160 connectors and covers batch, real-time, and CDC scenarios. Teams can organize reads, field processing, and writes around a common Source, Transform, and Sink model instead of maintaining unrelated scripts for different systems. Evaluate connector coverage and engine support for the combination you intend to use. Apache SeaTunnel website
- Recoverable state for reliable operation. SeaTunnel's built-in Zeta engine uses checkpoints to save job state for failure recovery; savepoints support planned stops and subsequent recovery. A continuously running pipeline can therefore recover from saved processing progress rather than defaulting to a complete reload. End-to-end consistency also depends on the source, sink connector, and commit protocol. Enabling checkpoints alone does not establish exactly-once semantics for every destination. Zeta 3.0.0 state storage and recovery
Together, these capabilities serve a practical goal: moving data efficiently across systems while maintaining the state the business expects through ongoing changes and recovery. Delete propagation is one concrete way to examine that goal.
This article addresses a scenario that can affect reporting correctness: when SeaTunnel synchronizes orders into PostgreSQL, why might an order remain in the destination after it has been deleted from the source? We will follow the released JDBC sink implementation in SeaTunnel 3.0.0 to explain how deletion events become database operations and which settings to check.
1. Define what the destination table represents
Suppose the reporting database in PostgreSQL contains a current-state table:
CREATE TABLE public.orders_current (
order_id BIGINT PRIMARY KEY,
status VARCHAR(32) NOT NULL
);
It stores the latest state of each order. For a stable, unchanging key, order_id = 42, the synchronization contract should be explicit:
| Source operation | Expected destination state after the operation is applied |
|---|---|
Insert order 42 with status created | Order 42 exists with status created |
Update order 42 to paid | Order 42 exists with status paid |
| Delete order 42 | Order 42 no longer exists |
For an audit table, retaining an event that says “order 42 was deleted” may be exactly the right behavior. Before troubleshooting, distinguish a current-state replica from a complete change history. This article uses the former contract.
2. Capture, transport, execution, and visibility are separate stages
Change data capture, or CDC, identifies changes in the source database. After the source captures a DELETE, several steps remain before reporting queries see the change:
The source database deletes an order
→ The Source emits a delete event and the order identifier
→ The Transform preserves the operation semantics and the correct key
→ The JDBC Sink selects the delete execution path
→ The destination database successfully applies and commits the operation
→ Queries read the updated destination table
A running job and a captured source deletion are therefore insufficient evidence that the destination is correct. Follow a specific order key through the pipeline instead of relying only on job status or aggregate throughput.
Start with the key. If a transformation drops order_id, or renames it without updating the destination configuration, the sink cannot locate the intended row. If the actual unique identifier is (tenant_id, order_id), configuring only order_id does not express the same delete predicate.
For this example, the intended destination operation is:
DELETE FROM public.orders_current WHERE order_id = ?;
The parameter must contain the key of the deleted order.
3. How SeaTunnel routes change events to the right SQL executor
SeaTunnel's JDBC sink supports more than one write path. Understanding which path is selected is more useful than simply checking that the job “uses JDBC.”
In version 3.0.0, JdbcOutputFormatBuilder considers COPY, custom SQL, and primary-key configuration when selecting an executor. In the ordinary keyed path used here, with COPY disabled and no custom SQL, the builder creates separate upsert and delete executors. The database dialect and key fields determine the generated DELETE statement. Executor construction
Join the conversation! Your thoughts help the community grow.