This is a place for stduying RocksDB (Facebook) by Dankook University
- Writer : Hojin Shin
- Laboratory : Embedded System Lab. in Dankook University
- 2021 DKU RocksDB Festival Lecture Information link
git clone https://github.com/DKU-StarLab/RocksDB_Explorer.git
cd RocksDB_Explorer
make db_bench (It will take some time)
You need to edit your DEV_PATH before starting experiments.
If you want to evaluate latency, please check variable RocksDB_Histogram_Percentile (0,1) in monitoring/histogram.cc line 20
2.1 What if we turn off the WAL (Write-Ahead-Log)
-
Throughput
./policy_wal.sh [YOUR_DEV_PATH] Ex) ./policy_wal.sh ../mnt -
Latency
./wal_latency.sh [YOUR_DEV_PATH] Ex) ./wal_latency.sh ../mnt cd ../python_parser python3 latency_parser.py [RESULT_TXT] [RESULT_CSV] Ex) python3 latency_parser.py ../result_txt/wal_on_latency_result.txt ../parsing_csv/wal_on_latecny.csv
2.2 What if we turn off the Compaction
-
Throughput
./policy_compaction.sh [YOUR_DEV_PATH] Ex) ./policy_compaction.sh ../mnt -
Latency
./compaction_latency.sh [YOUR_DEV_PATH] Ex) ./compaction_latency.sh ../mnt python3 latency_parser.py [RESULT_TXT] [RESULT_CSV] Ex) python3 latency_parser.py ../result_txt/compaction_on_latency_result.txt ../parsing_csv/compaction_on_latecny.csv
2.2 Explore files that are used and created during compaction
Not yet, it will be update soon
2.3 Details explanation about options
-
WAL (Write-Ahead-Log)
- disalbe_wal = true/false (Control WAL operation) -
Compaction
- disable_auto_compactions = true/false (Control automatic compactions) - level0_file_num_compaction_trigger = integer (Number of files to trigger level-0 compaction. A value < 0 means that level-0 compaction will not be triggered by number of files at all)
3.1 What if we change the number of background thread
./configuration_thread.sh [YOUR_DEV_PATH] [NUM_THREAD]
Ex) ./configuration_thread.sh ../mnt 16
3.2 What if we change the memtable setting
./configuration_memtable.sh [YOUR_DEV_PATH] [MEM_NUM] [MEM_SIZE]
Ex) ./configuration_memtable.sh ../mnt 2 64
MEM_SIZE is in MB, 64 means 64MB
3.3 Details explanation about options
- max_background_jobs = integer (Maximum number of concurrent background jobs, compactions and flushes)
- max_write_buffer_number = integer (Maximum number of memtable)
- write_buffer_size = integer (byte, Memtable size)
4.1 + 2 What if the key-value size and the key pattern are changed
./workload_key_value_size.sh [YOUR_DEV_PATH] [KEY_SIZE] [VALUE_SIZE] [BENCHMARK]
Ex) ./workload_key_value_size.sh ../mnt 16 100 fillrandom
Value distribution is fixed in this configure.
4.3 What if the value size distribution is changed
./workload_value_size_distribution.sh [YOUR_DEV_PATH] [BENCHMARK] [DISTRIBUTION_TYPE] [VALUE_MAX_SIZE] [VALUE_MIN_SIZE]
Ex) ./workload_value_size_distribution.sh ../mnt fillrandom uniform 1024 100
4.4 Details explanation about options
- key_size = integer (Control key size)
- value_size = integer (Control value size)
- value_size_distribution = fixed/uniform/normal
- value_size_min, value_size_manx = integer (Control value max, min size)
5.1 Source code level exploration
Not yet, it will be update soon
Operatring System : Ubuntu 18.04
CPU : 8 * Intel(R) Core(TM) i7-6700 CPU @ 3.40Hz
RAM : 8GB
SSD : Samsung EVO 860 Pro 1TB
RocksDB ver. 6.14 -> 6.21
If you have any problem to run our explorer, please leave your comment in the Issue section
RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.