Elektra
0.9.7
|
The following table shows a summary of the results of a benchmark_storage
run: NOTE: factor
is always dump / quickdump
(bigger = quickdump
is faster), this test was only run with version 1, see below for a comparison of with version 2
operation | dump (µs) | quickdump (µs) | factor |
---|---|---|---|
write keyset | 81661 | 4741 | 17.224 |
read keyset | 94782 | 44673 | 2.122 |
iterate keyset | 125 | 135 | 0.926 |
delete keyset | 1823 | 3518 | 0.518 |
re-read keyset | 92939 | 43749 | 2.124 |
strcmp key name | 609 | 1826 | 0.334 |
strcmp key value | 684 | 761 | 0.899 |
A script was used to generate KeySets with 2
, 200
, 2,000
, 200,000
and 2,000,000
Keys. For keynames we used increasing numbers (prefixed with _
to keep correct order), each key had a randomized value of 15 characters. Additionally we added 4 metakeys to each key. The names and values of these were created similarly.
The parent key used was dir:/tests/bench
.
We then used hyperfine
to compare the performance of dump
and quickdump
.
This means we used the same key sets for each of the runs, but the key values shouldn't really matter. It also means that no shared metadata (keyCopyMeta
) was involved.
We also used a modified version of benchmark_plugingetset
that returned after the get
call, to test the get
performance by itself. Because set
doesn't scale linearly because of the checks for keyCopyMeta
, the largest KeySet was only used for the get
only version. (The test was aborted after the initial run of dump
took longer than all the runs for quickdump
together.)
The complete results can be found in this repository and the files used for these benchmarks are quite big and can be found in another separate repository.
For reference in the file size tests below, here is a list of the raw data sizes of the keysets. These sizes are calculated as the sum of: 1) length of all keynames 2) length of all key values 3) length of all metakeys 4) length of all meta values All lengths include a null terminator, sizes are calculated once with (WP) and once without the parent key (WOP).
no. of keys | raw size WP (B) | raw size WOP (B) |
---|---|---|
2 | 252 | 220 |
200 | 25784 | 22584 |
2000 | 262786 | 229786 |
200000 | 26977790 | 23777790 |
20000000 | 273777792 | 241777792 |
factor
is dump / quickdump
like above (bigger = quickdump
has smaller files)
no. of keys | dump (B) | quickdump (B) | factor |
---|---|---|---|
2 | 430 | 412 | 1.04 |
200 | 41206 | 40988 | 1.01 |
2000 | 415807 | 413788 | 1.00 |
200000 | 42377809 | 42177788 | 1.00 |
2000000 | 427777810 | 425777788 | 1.00 |
The values are mean ± standard deviation. factor
is dump / quickdump
like above
no. of keys | dump (s) | quickdump (s) | factor |
---|---|---|---|
2 | 0.0016 ± 0.0003 | 0.0006 ± 0.0001 | 2.67 |
200 | 0.0091 ± 0.0009 | 0.0018 ± 0.0000 | 5.06 |
2000 | 0.0770 ± 0.0030 | 0.0127 ± 0.0008 | 6.06 |
200000 | 10.7876 ± 4.2564 | 1.2640 ± 0.0061 | 8.53 |
The values are mean ± standard deviation. factor
is dump / quickdump
like above
no. of keys | dump (s) | quickdump (s) | factor |
---|---|---|---|
2 | 0.0014 ± 0.0000 | 0.0005 ± 0.0000 | 2.8 |
200 | 0.0032 ± 0.0005 | 0.0013 ± 0.0002 | 2.46 |
2000 | 0.0179 ± 0.0002 | 0.0086 ± 0.0008 | 2.08 |
200000 | 1.6830 ± 0.0222 | 0.8515 ± 0.0048 | 1.98 |
2000000 | 17.1814 ± 0.2201 | 8.8808 ± 0.0344 | 1.93 |
Running the same benchmark_plugingetset
tests with version 2 of the plugin yields:
factor
is v1 / v2
like above (bigger is better)
no. of keys | quickdump v1 (B) | quickdump v2 (B) | factor |
---|---|---|---|
2 | 412 | 380 | 1.08 |
200 | 40988 | 23788 | 1.72 |
2000 | 413788 | 381788 | 1.08 |
200000 | 42177788 | 38977788 | 1.08 |
2000000 | 425777788 | 393777788 | 1.08 |
The values are mean ± standard deviation. factor
is v1 / v2
, i.e. bigger is better
no. of keys | quickdump v1 (s) | quickdump v2 (s) | factor |
---|---|---|---|
2 | 0.0006 ± 0.0001 | 0.0013 ± 0.0035 | 0.47 |
200 | 0.0018 ± 0.0000 | 0.0018 ± 0.0002 | 1.02 |
2000 | 0.0127 ± 0.0008 | 0.0112 ± 0.0021 | 1.13 |
200000 | 1.2640 ± 0.0061 | 1.2602 ± 0.0372 | 1.00 |
2000000 | - | 13.2104 ± 0.1185 | - |
The values are mean ± standard deviation. factor
is v1 / v2
like above
no. of keys | quickdump v1 (s) | quickdump v2 (s) | factor |
---|---|---|---|
2 | 0.0005 ± 0.0000 | 0.0006 ± 0.0001 | 0.88 |
200 | 0.0014 ± 0.0002 | 0.0010 ± 0.0001 | 1.33 |
2000 | 0.0086 ± 0.0008 | 0.0072 ± 0.0005 | 1.20 |
200000 | 0.8516 ± 0.0048 | 0.6413 ± 0.0082 | 1.33 |
2000000 | 8.8809 ± 0.0345 | 6.3756 ± 0.0443 | 1.39 |
Running the same benchmark_plugingetset
tests with version 3 of the plugin yields:
factor
is v1 / v2
like above (bigger is better)
no. of keys | quickdump v2 (B) | quickdump v3 (B) | factor |
---|---|---|---|
2 | 380 | 240 | 1.58 |
200 | 23788 | 23788 | 1.00 |
2000 | 381788 | 241788 | 1.58 |
200000 | 38977788 | 24977788 | 1.56 |
2000000 | 393777788 | 253777788 | 1.55 |
The values are mean ± standard deviation. factor
is v2 / v3
, i.e. bigger is better
no. of keys | quickdump v2 (s) | quickdump v3 (s) | factor |
---|---|---|---|
2 | 0.0013 ± 0.0035 | 0.0012 ± 0.0029 | 1.08 |
200 | 0.0018 ± 0.0002 | 0.0017 ± 0.0002 | 1.05 |
2000 | 0.0112 ± 0.0021 | 0.0106 ± 0.0011 | 1.06 |
200000 | 1.2602 ± 0.0372 | 1.1473 ± 0.0287 | 1.10 |
2000000 | 13.2104 ± 0.1185 | 11.5082 ± 0.2624 | 1.15 |
The values are mean ± standard deviation. factor
is v2 / v3
like above
no. of keys | quickdump v2 (s) | quickdump v3 (s) | factor |
---|---|---|---|
2 | 0.0006 ± 0.0001 | 0.0006 ± 0.0001 | 0.99 |
200 | 0.0010 ± 0.0001 | 0.0012 ± 0.0001 | 0.84 |
2000 | 0.0072 ± 0.0005 | 0.0070 ± 0.0004 | 1.03 |
200000 | 0.6413 ± 0.0082 | 0.6288 ± 0.0218 | 1.02 |
2000000 | 6.3756 ± 0.0443 | 6.2309 ± 0.0462 | 1.02 |