Eugula ValueSight#

Engula Overview#

Engula is a self-developed caching kernel product by Montplex, specifically designed for users of open-source Redis. Its core advantage lies in minimizing hardware costs for existing Redis clusters with extremely low risk. Key features include:

• Minimal Core Code Modifications: Ensures system stability and compatibility. Current version is 100% protocol compatible with Redis 7.2.4.
• Memory Savings: Significantly reduces metadata overhead, equipped with self-developed Compressed HybridLog storage engine to further reduce memory footprint.
• Performance Optimization: Optimizes end-to-end performance through improved HashTable implementation, optimized network thread communication mechanisms, enhanced CPU prefetch hit rate, and introduction of asynchronous thread framework.

Based on early user test results, Engula saves an average of 50% or more memory with performance overhead of less than 10%.

Engula ValueSight#

Engula ValueSight is a tool that helps users quickly and accurately evaluate the value of Engula. Users can compare memory usage between Engula and Redis based on their actual business data with one click, and quantify Engula's advantages in real business scenarios.

Core Features#

• Memory Usage Statistics: Compare RSS memory usage of the same dataset in Engula and Redis
• Memory Comparison Charts: Generate memory usage comparison charts with clear visualization
• Memory Usage Analysis Summary: Output memory usage analysis summary for easy understanding of optimization effects

Usage Instructions#

Runtime Environment#

Engula ValueSight is provided as a Docker image and can run on Linux or macOS systems with Docker installed. Environment requirements:

• CentOS: Version ≥ 7.9
• Ubuntu: Version ≥ 18.04
• Docker: Recommended to use official Docker version (installation guide, VPN may be required in some networks)
• CPU: Recommended host CPU cores ≥ 2

Running Steps#

1. Prepare RDB File: Generate a Redis dump.rdb file on the host machine (use BGSAVE command).

2. Download Engula ValueSight Image:

   1docker pull registry.cn-guangzhou.aliyuncs.com/montplex/engula-valuesight

3. Create Log Directory:

   1# You can change the local directory name, but make sure the docker command mapping is correct
   2mkdir -m 777 analysis_output

4. Execute Analysis:

   1docker run -it --rm \
   2  -v "<RDB_FILE_PATH>:/tmp/dump.rdb" \
   3  -v "$(pwd)/analysis_output:/engula/analysis_output" \
   4  registry.cn-guangzhou.aliyuncs.com/montplex/engula-valuesight

   Note: Replace <RDB_FILE_PATH> with the actual path to your dump.rdb file.

Running Example#

Assuming Docker is installed on the host machine and there is a dump.rdb file in the current directory, execute the following commands to generate a compression evaluation report for the dump.rdb file:

Note: Engula ValueSight is updated from time to time. It is recommended to pull the latest image if you haven't used it recently.

1# Create log directory
2mkdir analysis_output
3
4# Update Engula ValueSight image
5docker pull registry.cn-guangzhou.aliyuncs.com/montplex/engula-valuesight
6
7# Execute analysis
8docker run -it --rm \
9  -v "$(pwd)/dump.rdb:/tmp/dump.rdb" \
10  -v "$(pwd)/analysis_output:/engula/analysis_output" \
11  registry.cn-guangzhou.aliyuncs.com/montplex/engula-valuesight

Running Process:

Running Results:

After analysis completes, press Enter to view details, or press Q/Ctrl+C to exit.

Batch Compression Rate Testing#

Running Steps#

1. Create Necessary Directories:

   1# Create RDB file directory
   2mkdir rdb_dir
   3
   4# Create log directory
   5mkdir analysis_output

2. Place RDB Files:

   • Move RDB files to be analyzed into the rdb_dir directory
   • Supports subdirectory structure
   • Program automatically scans all .rdb files

3. Pull Analysis Tool Image:

   1docker pull registry.cn-guangzhou.aliyuncs.com/montplex/engula-valuesight

4. Execute Batch Compression Evaluation Command:

   1docker run -it --rm \
   2  -v "$(pwd)/rdb_dir:/tmp/rdb_dir" \
   3  -v "$(pwd)/analysis_output:/engula/analysis_output" \
   4  registry.cn-guangzhou.aliyuncs.com/montplex/engula-valuesight --batch

   Note: Make sure rdb_dir and analysis_output paths match the actual created directories. Please check the mapped path configuration carefully to avoid errors.

Shard Calculation Function#

If the RDB filename ends with an underscore followed by a number, that number will be treated as the shard count and included in the final calculation. This provides a more accurate assessment of memory savings scale.

Example: r_profile_100w_1000.rdb will be treated as having 1000 shards.

Running Process:

Running Results:

analysis_output Directory Contents#

For dump_1.rdb, there will be four files:

• _dump_1.rdb_cmd_output_for_engula.log - Log for dump_1.rdb during Engula evaluation
• _dump_1.rdb_cmd_output_for_redis.log - Log for dump_1.rdb during Redis evaluation
• _dump_1.rdb_engula-stats-rdb.json - Statistics for dump_1.rdb in Engula
• _dump_1.rdb_redis-stats-rdb.json - Statistics for dump_1.rdb in Redis

Each RDB file will have corresponding separate files generated. If you need details or encounter errors, you can refer to these files for relevant information.

Additionally, info.log will be generated to record general log information.

Background Running#

Since batch evaluation of multiple RDB files may take a long time, it is recommended to run the task in the background to improve work efficiency and avoid long waits.

1nohup docker run --rm \
2  -v "$(pwd)/rdb_dir:/tmp/rdb_dir" \
3  -v "$(pwd)/analysis_output:/engula/analysis_output" \
4  registry.cn-guangzhou.aliyuncs.com/montplex/engula-valuesight --batch \
5  > analysis_output/docker.log 2> analysis_output/docker.error.log &

You can view task progress using:

1# View output log
2tail -f analysis_output/docker.log
3
4# View error log
5tail -f analysis_output/docker.error.log

FAQ#

Docker Memory Setting Too Small#

Error Symptom: Runtime error: Error2: Child process quit abnormally, ExitCode=9

Cause and Solution: This error is usually caused by insufficient Docker memory due to large RDB files. Recommendations:

• Increase memory allocation in Docker Desktop settings (recommended ≥ 4GB)
• Split large RDB files and process in batches

Debug Method#

After analysis completes, the analysis_output directory will contain program execution logs and statistics that can be used for debugging. If you encounter issues, please contact the Montplex technical team for consultation.

Red Hat System Docker Version Installation#

In Red Hat Linux 9 systems, the Docker version installed through the following command does not meet the testing requirements:

1yum install docker -y
2docker --version
3# podman version 4.9.4-rhel

It is recommended to install according to the Docker official documentation. Specific steps:

1. Uninstall old version:

   1sudo yum remove docker \
   2              docker-client \
   3              docker-client-latest \
   4              docker-common \
   5              docker-latest \
   6              docker-latest-logrotate \
   7              docker-logrotate \
   8              docker-engine \
   9              podman \
   10              runc

2. Install yum-utils and add Docker official repository:

   1sudo yum install -y yum-utils
   2sudo yum-config-manager --add-repo https://download.docker.com/linux/rhel/docker-ce.repo

3. Install latest version of Docker:

   1sudo yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

4. Start Docker service:

   1sudo systemctl start docker

Batch Mode Error: "rdb file not found"#

The image default program does not run under the root user. This error may be caused by this issue.

Solution: Add --user root to the batch command:

1docker run --user root -it --rm \
2  -v "$(pwd)/rdb_dir:/tmp/rdb_dir" \
3  -v "$(pwd)/analysis_output:/engula/analysis_output" \
4  registry.cn-guangzhou.aliyuncs.com/montplex/engula-valuesight --batch

Contact Us#

If you encounter any issues while using Engula or Engula ValueSight, or would like to learn more about the product, please feel free to contact us:

Email: support@montplex.com

The Montplex technical team will respond to your inquiries promptly during business hours 9:00 — 18:00 (UTC+8). Thank you for your support! 🚀