Storage engines play an important role in database performance, durability, and recovery. Although application developers usually interact with a database through queries and APIs, the storage engine determines how data is written to disk, how transactions are maintained, and how the database recovers after a failure.
SurrealDB has traditionally used RocksDB as an on-disk storage engine. However, SurrealDB has developed its own storage engine, SurrealKV, to better align storage behavior with the database's architecture.
SurrealKV is written in Rust and uses a Log-Structured Merge-tree (LSM-tree) architecture. It is designed to provide transactional storage, concurrent access, and tighter integration with SurrealDB.
With SurrealDB 4.0, RocksDB and IndexedDB have been removed as storage engines. SurrealKV becomes the on-disk storage engine, and existing RocksDB data directories can be converted automatically when opened by the newer version.
This change affects more than the database startup command. Developers and database administrators need to understand migration behavior, storage configuration, versioned data, backup requirements, and operational differences.
This article explains what changes when moving from RocksDB to SurrealKV and how to prepare an existing SurrealDB application for the transition.
Why SurrealDB Is Replacing RocksDB
RocksDB is a mature embedded key-value storage engine originally developed by Facebook. It is written in C++ and uses an LSM-tree architecture to manage data efficiently.
SurrealDB used RocksDB to provide persistent storage for single-node deployments.
However, relying on an external storage engine introduces additional integration and maintenance considerations.
For example:
- Storage behavior must be adapted to SurrealDB's transaction model.
- Database-specific storage improvements depend partly on an external project.
- The database must integrate with another engine's configuration and operational behavior.
- Native dependencies can increase build and deployment complexity.
SurrealKV addresses these concerns by providing a storage engine developed specifically for SurrealDB.
It is written in Rust and designed around SurrealDB's storage requirements.
The goal is to make storage behavior easier to evolve alongside the database rather than depending on an external engine for every storage-related improvement.
Understanding the Architecture of SurrealKV
SurrealKV uses an LSM-tree architecture.
An LSM-tree organizes writes through memory structures and immutable files on disk. Instead of modifying existing disk structures for every update, the engine collects changes and periodically merges data into more permanent storage files.
A simplified write process looks like this:
- A transaction submits changes.
- The engine records the changes in its write-ahead log.
- Updated values are maintained in memory structures.
- Committed data is written to persistent storage files.
- Background compaction reorganizes storage files over time.
This approach is useful for workloads that perform frequent writes.
SurrealKV also uses multi-version concurrency control (MVCC), which allows transactions to work with consistent snapshots of database data.
Important SurrealKV Components
| Component | Responsibility |
|---|
| Write-Ahead Log (WAL) | Records changes needed for durability and recovery |
| MemTable | Holds recently written data in memory |
| SSTables | Store sorted, immutable data on disk |
| Manifest | Tracks storage file organization and metadata |
| Compaction | Merges storage files and manages obsolete versions |
| Value Log | Stores larger values separately to reduce write amplification |
| MVCC | Maintains transaction snapshots and version visibility |
These components work together to provide persistent key-value storage.
Developers using SurrealDB's query interface do not normally interact with these components directly. However, understanding them helps explain why storage migration, compaction, recovery, and disk usage matter.
What Changes in SurrealDB 4.0?
The most important change is that RocksDB is no longer available as an on-disk storage engine in SurrealDB 4.0.
SurrealKV becomes the standard on-disk engine for server and embedded deployments.
IndexedDB is also removed as a separate browser storage engine. Browser deployments use SurrealKV through the Origin Private File System (OPFS).
This means existing applications may need to update their storage connection configuration.
For example, an older deployment may use:
surreal start \
--user root \
--pass secret \
rocksdb://./database
The SurrealDB 4.0 configuration uses:
surreal start \
--user root \
--pass secret \
surrealkv://./database
The exact command-line options depend on the installed SurrealDB release, but the important change is the storage scheme.
A rocksdb:// path is still accepted as a deprecated spelling in SurrealDB 4.0. It opens the directory using SurrealKV and logs a warning.
For new configurations, use surrealkv:// rather than retaining the deprecated scheme.
How Automatic RocksDB Migration Works
One of the important changes in SurrealDB 4.0 is its ability to convert an existing RocksDB data directory when SurrealKV opens it.
This means a typical migration does not require manually exporting every database record and importing it into a new database.
The conversion process reads the RocksDB files and builds the corresponding SurrealKV storage structures.
At a high level, the process includes:
- SurrealKV obtains the database directory lock.
- It reads the existing RocksDB storage files.
- It creates the new SurrealKV data structures.
- It verifies the converted data.
- It switches the directory to the new storage format.
- It moves the original RocksDB files into a backup folder inside the data directory.
The conversion is designed to handle interrupted operations through migration markers and recovery steps.
However, developers should not treat this as a substitute for a separate backup.
The conversion is one-way. RocksDB cannot open the converted directory as a normal RocksDB database.
Prepare the Database Before Migration
Before upgrading, stop the existing SurrealDB process and create a separate copy of the complete data directory.
For example:
cp -a ./database ./database-backup
This example assumes the database is stored in a local directory named database.
Use a backup method appropriate for the deployment environment. For large databases, ensure sufficient disk capacity for the original files, converted data, and backup copy.
Do not start the old and new database processes against the same directory simultaneously.
Start SurrealDB With SurrealKV
After creating the backup, start the newer version using the SurrealKV scheme:
surreal start \
--user root \
--pass secret \
surrealkv://./database
SurrealDB detects the existing RocksDB data and performs the conversion.
The first startup may take longer than a normal database startup because the conversion must process the existing data.
During this operation, the server does not become ready to serve requests until migration finishes.
Update deployment health checks and startup timeouts to account for this initial conversion.
What Happens to Existing Data?
The conversion preserves the current values stored in the RocksDB database.
Applications should still validate their data after migration rather than assuming that a successful startup proves everything is correct.
Important validation areas include:
- Record counts.
- Database and namespace availability.
- Index definitions.
- Table schemas.
- User and permission configuration.
- Application queries.
- Transaction behavior.
- Backup and restore procedures.
For example, if an application maintains a customer table, compare the number of customer records before and after migration.
A simple SurrealQL query might look like this:
SELECT count() AS total
FROM customer
GROUP ALL;
Run the same query against the old and new environments where possible.
Also execute important application operations, including inserts, updates, deletes, and relationship queries.
For production systems, test the migration against a copy of the actual database before scheduling the upgrade.
What Changes for Versioned Databases?
SurrealDB supports versioned data through SurrealKV.
This allows applications to query historical versions of data using the VERSION clause when versioning is enabled.
For example:
SELECT *
FROM customer
VERSION d'2026-10-01T00:00:00Z';
This query illustrates retrieving records as they existed at a particular timestamp.
The important migration consideration is that an existing RocksDB database may already have versioning enabled.
If historical versions need to be preserved, the SurrealKV connection must also enable versioning.
For example:
surreal start \
--user root \
--pass secret \
"surrealkv://./database?versioned=true"
A database that previously used versioning retains its versions and timestamps when opened with the appropriate SurrealKV configuration.
If versioning is not enabled for the new connection, the application should not assume that historical version queries will continue to work.
Before upgrading, identify whether the application depends on historical queries, audit-style data access, or version-based recovery.
Review Existing RocksDB Configuration
RocksDB-specific configuration is another area that changes.
Older SurrealDB deployments may use environment variables to configure RocksDB behavior.
For example, an existing deployment might contain:
SURREAL_ROCKSDB_PARALLELISM=4
SURREAL_ROCKSDB_MAX_OPEN_FILES=1000
These settings are examples of legacy RocksDB configuration. They should not be carried forward as if they controlled SurrealKV.
SurrealDB 4.0 no longer reads RocksDB settings. This includes the SURREAL_ROCKSDB_* and SURREAL_DS_ROCKSDB_* environment variables and RocksDB-specific parameters in connection paths.
Review:
- Docker environment variables.
- Kubernetes ConfigMaps.
- Kubernetes Secrets.
- Deployment scripts.
- CI/CD variables.
- Container startup commands.
- Infrastructure configuration files.
Remove obsolete settings and replace them with supported SurrealKV configuration where required.
SurrealKV has a smaller configuration surface, so not every previous RocksDB tuning option has a direct equivalent.
Changes for Rust SDK Applications
Applications embedding SurrealDB through the Rust SDK may also need code and dependency changes.
Older Rust projects can enable RocksDB through a feature such as:
[dependencies]
surrealdb = {
version = "3",
features = ["kv-rocksdb"]
}
For SurrealDB 4.0, use the SurrealKV feature:
[dependencies]
surrealdb = {
version = "4",
features = ["kv-surrealkv"]
}
The exact dependency version should match the release used by the application.
The engine configuration also changes.
For example, an older embedded application may use a RocksDB engine:
use surrealdb::engine::local::RocksDb;
use surrealdb::Surreal;
#[tokio::main]
async fn main() -> surrealdb::Result<()> {
let db = Surreal::new::<RocksDb>("./database").await?;
Ok(())
}
The SurrealKV equivalent uses the SurrealKV engine:
use surrealdb::engine::local::SurrealKv;
use surrealdb::Surreal;
#[tokio::main]
async fn main() -> surrealdb::Result<()> {
let db = Surreal::new::<SurrealKv>("./database").await?;
Ok(())
}
Check the SDK documentation for the exact API and feature requirements of the release being used.
Projects that directly depend on RocksDB-specific Rust types or features must update those references as part of the migration.
Performance and Resource Considerations
SurrealKV and RocksDB both use LSM-tree storage, but their internal implementations differ.
SurrealKV is designed around SurrealDB's transaction and storage requirements. It uses components such as radix-tree memory structures, a commit ring, MVCC, and value-log separation.
These design choices affect how the engine handles writes, reads, concurrency, and storage compaction.
However, developers should avoid assuming that every application will become faster after switching engines.
Actual performance depends on:
- Database size.
- Read-to-write ratio.
- Transaction size.
- Query patterns.
- Storage hardware.
- Available memory.
- Concurrent operations.
- Compaction activity.
- Durability requirements.
For example, an application that performs frequent point lookups may behave differently from one that scans large ranges of records.
Before upgrading production, run representative benchmarks against a copy of the application database.
Measure query latency, write throughput, memory usage, disk usage, and startup recovery time.
Common Migration Mistakes
| Mistake | Why it causes problems | Better approach |
|---|
| Opening production data without a backup | Conversion is one-way | Create a separate backup first |
| Starting the old and new processes together | Both may attempt to access the same database directory | Stop the old process before migration |
| Ignoring initial conversion time | Health checks may report the database as unavailable | Allow sufficient startup time |
| Keeping RocksDB environment variables | SurrealDB 4.0 ignores legacy settings | Review and remove obsolete configuration |
Forgetting versioned=true | Historical version queries may stop working | Preserve versioning configuration |
| Assuming migration guarantees application correctness | Data conversion does not validate every application workflow | Run functional and data checks |
| Reusing the converted directory for rollback | The original RocksDB format is no longer available there | Keep a separate untouched backup |
| Changing SDK features without testing | Embedded applications may fail to compile or start | Update dependencies and test locally |
Advantages and Disadvantages of SurrealKVAdvantages
- Developed specifically for SurrealDB.
- Written in Rust.
- Uses an LSM-tree storage architecture.
- Supports transactional operations and MVCC.
- Supports versioned data when configured.
- Integrates with SurrealDB's storage and query requirements.
- Can automatically convert supported RocksDB data directories in SurrealDB 4.0.
- Supports embedded and browser-based storage scenarios.
Disadvantages
- The storage format differs from RocksDB.
- Migration is one-way and requires a separate backup for rollback.
- Initial conversion can increase upgrade downtime.
- Legacy RocksDB configuration options are no longer supported.
- Existing Rust SDK integrations may require code changes.
- Performance should be validated against the application's actual workload.
Summary
Replacing RocksDB with SurrealKV is an important storage change in SurrealDB 4.0.
SurrealKV provides a storage engine developed specifically for SurrealDB, using an LSM-tree architecture, transactional storage, and MVCC.
For existing deployments, SurrealDB 4.0 can automatically convert supported RocksDB data directories when they are opened with SurrealKV. However, the conversion is one-way, so a separate backup is essential.
Before upgrading, review storage paths, environment variables, Rust SDK dependencies, versioned data settings, and deployment health checks.
After migration, validate important records, application queries, transaction behavior, and recovery procedures.
The main benefit is tighter integration between SurrealDB and its storage layer. The main responsibility for developers is to treat the transition as a database migration, not simply a change to the storage URL.
Join the conversation! Your thoughts help the community grow.