C
Cubrim
EN RU
Back to the dashboard

CUBR-0034 · real measurements

World benchmark

An open measurement on the recognised world corpora (Silesia, enwik8, Canterbury) against the standard archivers. Lower ratio = better.

Honest scope

This is an open measurement on recognised world corpora against standard archivers. Cubrim now leads the overall aggregate, but the per-type table still shows where it loses: ppmd leads text, and 7z remains strong on exe and binary. Every gap is the target of the next hypothesis in the research loop. Dead ends and losses are shown as they are.

overall aggregate: cubrim 0.2227 · leader cubrim 0.2227

Overall archiver ranking

Standings by aggregate ratio across all corpora (lower is better). Each name shows the compression level it ran at.

#1
cubrim competitive
0.2227
#2
ppmd 7z -m0=PPMd
0.2286
#3
xz -9e
0.2344
#4
7z -m0=LZMA2 -mx9
0.2355
#5
brotli -q 11
0.2408
#6
zstd --ultra -22
0.2490
#7
rar a -m5
0.2573
#8
bzip2 -9
0.2671
#9
gzip -9
0.3330
#10
lz4 -12
0.3822
cubrim

Aggregate by corpus

corpus cubrim rank cubrim gzip bzip2 xz zstd brotli lz4 ppmd 7z rar
canterbury #1/10 0.1605 0.2600 0.1931 0.1754 0.1854 0.1746 0.3328 0.1837 0.1761 0.1818
enwik8 #2/10 0.2371 0.3645 0.2901 0.2483 0.2533 0.2574 0.4199 0.2240 0.2486 0.2735
silesia #1/10 0.2168 0.3191 0.2572 0.2286 0.2478 0.2339 0.3651 0.2313 0.2301 0.2507

Per-file, grouped by data type

The best archiver in each row is highlighted green, the worst red; #rank marks where Cubrim places among all archivers (1 = best).

Text

type aggregate: ppmd 0.2014 · cubrim 0.2127 · xz 0.2316

Type rank is by total compressed size (Σ compressed bytes ÷ Σ original bytes across the type's files) — the same size-weighted method as Silesia/lzbench, not an average of per-file ranks. An archiver can win the type without leading any single file if its total output is smallest.

file cubrim rank cubrim gzip bzip2 xz zstd brotli lz4 ppmd 7z rar
alice29.txt #2/10 0.2625 0.3563 0.2841 0.3191 0.3236 0.3057 0.4140 0.2563 0.3195 0.3365
asyoulik.txt #2/10 0.2913 0.3901 0.3161 0.3562 0.3606 0.3412 0.4660 0.2903 0.3568 0.3749
cp.html #1/10 0.2692 0.3244 0.3099 0.3110 0.3136 0.2803 0.4189 0.2720 0.3140 0.3282
lcet10.txt #2/10 0.2268 0.3384 0.2524 0.2800 0.2844 0.2658 0.3846 0.2262 0.2802 0.2963
plrabn12.txt #2/10 0.2819 0.4032 0.3021 0.3434 0.3475 0.3388 0.4663 0.2750 0.3438 0.3644
xargs.1 #2/10 0.3598 0.4154 0.4168 0.4287 0.4088 0.3463 0.5725 0.3809 0.4443 0.4303
enwik8 #2/10 0.2371 0.3645 0.2901 0.2483 0.2533 0.2574 0.4199 0.2240 0.2486 0.2735
dickens #2/10 0.2392 0.3779 0.2747 0.2778 0.2796 0.2774 0.4302 0.2253 0.2778 0.2975
reymont #1/10 0.1668 0.2748 0.1880 0.1985 0.2033 0.2010 0.3118 0.1722 0.1989 0.2180
webster #2/10 0.1712 0.2909 0.2085 0.2019 0.2040 0.2033 0.3341 0.1578 0.2023 0.2179
xml #1/10 0.0726 0.1239 0.0825 0.0814 0.0848 0.0806 0.1424 0.0929 0.0851 0.0907

Source code

type aggregate: cubrim 0.1615 · xz 0.1732 · 7z 0.1741

Type rank is by total compressed size (Σ compressed bytes ÷ Σ original bytes across the type's files) — the same size-weighted method as Silesia/lzbench, not an average of per-file ranks. An archiver can win the type without leading any single file if its total output is smallest.

file cubrim rank cubrim gzip bzip2 xz zstd brotli lz4 ppmd 7z rar
fields.c #1/10 0.2304 0.2813 0.2726 0.2719 0.2708 0.2437 0.3786 0.2476 0.2767 0.2835
grammar.lsp #1/10 0.2846 0.3349 0.3448 0.3472 0.3263 0.3023 0.4668 0.3163 0.3668 0.3542
samba #1/10 0.1615 0.2503 0.2106 0.1731 0.1795 0.1743 0.2827 0.1943 0.1740 0.1901

Database

type aggregate: cubrim 0.0958 · xz 0.0984 · brotli 0.0994

Type rank is by total compressed size (Σ compressed bytes ÷ Σ original bytes across the type's files) — the same size-weighted method as Silesia/lzbench, not an average of per-file ranks. An archiver can win the type without leading any single file if its total output is smallest.

file cubrim rank cubrim gzip bzip2 xz zstd brotli lz4 ppmd 7z rar
nci #3/10 0.0478 0.0890 0.0540 0.0432 0.0481 0.0453 0.1081 0.0686 0.0519 0.0554
osdb #2/10 0.2553 0.3685 0.2779 0.2820 0.3072 0.2792 0.3923 0.2366 0.2828 0.3209

Executables

type aggregate: 7z 0.2749 · xz 0.2755 · brotli 0.2850 · cubrim 0.2936

Type rank is by total compressed size (Σ compressed bytes ÷ Σ original bytes across the type's files) — the same size-weighted method as Silesia/lzbench, not an average of per-file ranks. An archiver can win the type without leading any single file if its total output is smallest.

file cubrim rank cubrim gzip bzip2 xz zstd brotli lz4 ppmd 7z rar
mozilla #4/10 0.2818 0.3708 0.3498 0.2612 0.2922 0.2708 0.4302 0.3165 0.2605 0.2959
ooffice #2/10 0.3912 0.5023 0.4653 0.3945 0.4224 0.4029 0.5750 0.4134 0.3943 0.3742

Binary / numeric

type aggregate: 7z 0.5378 · xz 0.5393 · cubrim 0.5542

Type rank is by total compressed size (Σ compressed bytes ÷ Σ original bytes across the type's files) — the same size-weighted method as Silesia/lzbench, not an average of per-file ranks. An archiver can win the type without leading any single file if its total output is smallest.

file cubrim rank cubrim gzip bzip2 xz zstd brotli lz4 ppmd 7z rar
kennedy.xls #6/10 0.0695 0.2037 0.1265 0.0504 0.0677 0.0597 0.3148 0.1347 0.0491 0.0345
sum #4/10 0.2757 0.3340 0.3376 0.2484 0.2937 0.2652 0.4265 0.3065 0.2513 0.3034
sao #3/10 0.6244 0.7346 0.6813 0.6103 0.6895 0.6324 0.7820 0.6561 0.6087 0.7643

Images

type aggregate: cubrim 0.3122 · ppmd 0.3271 · bzip2 0.3451

Type rank is by total compressed size (Σ compressed bytes ÷ Σ original bytes across the type's files) — the same size-weighted method as Silesia/lzbench, not an average of per-file ranks. An archiver can win the type without leading any single file if its total output is smallest.

file cubrim rank cubrim gzip bzip2 xz zstd brotli lz4 ppmd 7z rar
ptt5 #5/10 0.0942 0.1021 0.0970 0.0777 0.0846 0.0798 0.1289 0.0958 0.0822 0.0960
mr #1/10 0.2104 0.3685 0.2448 0.2760 0.3115 0.2831 0.4204 0.2308 0.2756 0.2784
x-ray #1/10 0.4451 0.7125 0.4781 0.5300 0.6084 0.5526 0.8473 0.4545 0.5286 0.4912

ratio = compressed / original (lower is better); round-trip byte-exact verified for cubrim. · 7z -m0=LZMA2 -mx9 · xz -9e · lz4 -12 · rar a -m5 · gzip -9 · ppmd 7z -m0=PPMd · zstd --ultra -22 · bzip2 -9 · brotli -q 11 · cubrim competitive-min corrected: CUBR-0043 CM19 rows + prior verified rows + fresh exact-317a323 replacements for nci/ptt5/kennedy.xls

How these numbers are measured

What's next

Each gap → a candidate hypothesis

The weakest classes are exactly where the next research rounds begin — as H-29 columnar field-split was born from a telemetry-CSV gap. Ranked by return on investment.

IW-02

mr image flip — landed at rank #1

Images DONE

MED16 at stride width 512 flipped mr from rank 3 to rank #1 (0.2104 vs ppmd 0.2308, ~9% ahead), residual round-trip byte-exact. Already live in the benchmark above — with x-ray #1 the image class is two-thirds closed.

MED16 width-512 RESID-RT-OK
IW-04

Context-mixing text backend (PPMd-class)

Text HIGH

text remains the biggest type prize even after Cubrim reached overall #1: 11 files, ppmd leads with gaps up to +33% (enwik8 0.2622 vs 0.2240; webster 0.2105 vs 0.1578). A logistic context-mixing model (order-0..6 + match model) is already proven on dickens slices to beat 7z-PPMd by ~9%; integrating it as the text backend is the direct path to winning the text type, where enwik8 (100 MB) dominates the size-weighted metric.

CM paq-lite match model logistic mixing
IW-01

MED16 adaptive-width ladder

Images HIGH

Generalise the mr flip: an autocorrelation width detector + a 256..2048 candidate ladder under competitive-min, so any 16-bit image gets its optimal stride automatically while x-ray holds #1 (0.4451 vs ppmd 0.4545). Turns a hand-tuned win into a systemic one.

MED16 autocorrelation competitive-min
NEW-21

ML-lite content dispatcher

Source code MEDIUM

Cheap feature extraction (histogram, entropies, stride autocorrelation, format magics) predicts the top-K ensemble branches before trial compression. This removes the mode-errors that pin xargs.1, grammar.lsp and fields.c at rank 9, cuts benchmark time 2–4×, and makes heavy branches (CM, bilevel) affordable.

dispatcher feature-extractor top-K
IW-03

Small-file context + container diet

Text MEDIUM

The four rank-9 tiny files sit 20–31% behind brotli only because the BWT block is too small for them (xargs.1 0.4530 vs brotli 0.3463; grammar.lsp 0.3873 vs 0.3023; fields.c 0.3056 vs 0.2437). Dispatching <64 KB inputs to a small-block context coder with a static dictionary — plus a lean ≤12-byte container (NEW-20) — moves them toward pack parity.

tiny-dispatcher static dict container diet