Another Tool For Analyzing RDB Files Of Redis

Background

Sometimes, in the process of using Redis, we can encounter problems, such as too much memory or full bandwidth. When we encounter these problems, we need to analyze the memory of Redis.

There are two kinds of memory analysis for Redis: online analysis and offline analysis. Online analysis directly connects to the Redis server and analyzes its memory, which will increase load on the Redis server. This may affect the stability of the Redis server. Offline analysis is based on RDB backup files, which will not affect the stability of the Redis server. This is recommended when the used memory is greater than 2GB.

How can we analyze Redis' memory offline? There are two well-known tools in the open source world that can help us: redis-rdb-tools and rdr.

redis-rdb-tools is implemented by Python, and rdr is implemented by Golang. If developers are not familiar with these two languages and want to expand some functions based on them, the process may be more difficult. Next, I will introduce another offline analysis tool implemented by c# that will be easier for c# developers to expand.

rdb-tools

rdb-tools is a tool to parse/analyze redis rdb files that is implemented by c#.

It consists of two parts:

A parser library, that you can customize by yourself.
An anslysis cli tool, that you can analyze some basic usages for command line.

The parser library is based on net6.0 so that we can easily expand it.

The cli tool is also based on net6.0, which is built using the command-line-api and can be packaged into binary files without runtime dependencies.

Usage of CLI tool

There are two ways to install rdb-cli.

If you don't install .NET 6.0 SDK, download the tool from the latest stable release based on your OS version.
If you have installed .NET 6.0 SDK, you can use dotnet tool install --global rdb-cli to install.

Here we will show the first way.

After downloading and extracting, you will get an executable file named rdb-cli.

Input ./rdb-cli -h to get the help information.

As you can see, there are two commands that we can use. The most important command is memory.

From the help information, you can see that you need to specify the RDB file and some specific options.

A simple and common example is as follows:

./rdb-cli memory /tmp/test/demo.rdb -ot html

This command will analyze the rdb file, and the analysis results will be presented in the form of HTML.

The screenshot below shows the output of the execution command. It is evident that it takes only 32 seconds to analyze a 2.1GB RDB file, which is a relatively fast speed.

Detailed HTML report output in /tmp/test/res.html

Let's take a look at it.

The HTML result contains three parts:

The first part is basic information, including RDB version information, redis version information, total memory and total number of keys.

The second part is a few histograms, mainly the memory and quantity distribution of different data types, as well as the memory and quantity distribution of expiration time.

The third part is a few tables, including the top key prefix list, the top big key list, the top stream list, and the function list (redis 7.0).

Additionally, if you don't want to analyze all the information and want to filter some conditions, you can specify different parameter options.

Two examples:

1. When you only want to analyze DB 9 and DB 10:

./rdb-cli memory /tmp/test/demo.rdb -ot html --db 9 --db 10

2. When you only want to analyze hash and string types:

./rdb-cli memory /tmp/test/demo.rdb -ot html --type string --type hash

The analysis of 2~8GB RDB files is supplemented for reference.

The CLI tools described above may not be completely suitable for your needs. At this time, we can customize our customization requirements based on the parsing class library.

Usage of parser library

Install RDBParse package from nuget at first.

dotnet add package RDBParse

Next, you need to add an implementation class to implement the IReaderCallback interface.

Then, create a new instance of BinaryReaderRDBParser class.

At last, call Parse method of BinaryReaderRDBParser's instance.

Following this below code for example.

public class MyReaderCallBack : IReaderCallback
{
}

var path = "/yourpath/your.rdb"
var cb = new MyReaderCallBack();
var parser = new RDBParser.BinaryReaderRDBParser(cb);
parser.Parse(path);

Summary

This article introduces an RDB analysis tool based on c# and understands its basic usage and how to customize it.

I hope this will help you!

Reference