Cubrim — Global Addresser

The honest limit, stated first

A fixed short code can distinguish only a finite number of states, while the space of possible files grows exponentially with length — so a short address alone can never uniquely denote every long sequence. An address is meaningful only together with a catalog where the object already exists. And any honest result must charge the full cost: the addresses, the metadata, the residual data — and the shared matrix itself, which is only worth its size when amortized across many files and devices. Cubrim-2 therefore does not promise to “compress any file into 16 bytes”. It maps where global addressing genuinely beats local compression — and records, just as openly, where it structurally cannot.

How much space would ALL possible matrices take

We count in bits. An N-dimensional cube is a visual arrangement of a bit sequence: a 4×4×4 cube is 64 bits laid out in three dimensions. Dimensionality (1D/2D/3D/4D) is a way to arrange the same bits, not a different amount of them. Hence the number of all possible matrices of length B bits is 2^B and depends only on B: a 64-bit 1D string and a 4×4×4 cube share the same 2^64 ≈ 1.8×10^19 states.

B, bits	cube example	all possible = 2^B	store them all (bytes)	fits in 20,000×1 TB?
8	2×2×2	2.56×10²	2.56×10² B	yes
27	3×3×3	1.34×10⁸	4.53×10⁸ B	yes
51	—(порог/threshold)	2.25×10¹⁵	1.44×10¹⁶ B	yes
64	4×4×4	1.84×10¹⁹	1.48×10²⁰ B	no
125	5×5×5	4.25×10³⁷	6.65×10³⁸ B	no
256	4×4×4×4 (4D)	1.16×10⁷⁷	3.71×10⁷⁸ B	no
1000	10×10×10	1.07×10³⁰¹	1.34×10³⁰³ B	no
4096	8×8×8×8 (4D)	1.04×10¹²³³	5.35×10¹²³⁵ B	no

Capacity yardstick: 20,000 disks of 1 TB = 2×10^16 bytes = 1.6×10^17 bits. The full-enumeration threshold is B = 51 bits: the last length at which all 2^B matrices (with their contents, 2^B×B bits = 1.44×10^16 bytes) still fit; at B = 52 they no longer do (2.93×10^16 bytes). A 4×4×4 cube (64 bits): just the 64-bit addresses of all 2^64 matrices would take 1.48×10^20 bytes ≈ 147.6 EB — four orders of magnitude beyond the yardstick. At B = 256 there are 1.16×10^77 matrices — still slightly fewer than the atoms in the observable universe (~10^80); the “more than atoms” line is crossed at B ≈ 266. Beyond that, super-exponential growth with no physical storage prospect whatsoever.

The honest conclusion: “all possible matrices” cannot be enumerated — 2^B diverges super-exponentially at lengths below a single machine word. So the Addresser stores not all possible blocks but only the ones ACTUALLY ENCOUNTERED (CAS/deduplication): their number is bounded by the volume of real data and obeys the track’s measured laws — 89.9% of blocks in an uncurated matrix occur exactly once (AH-19), and an honest 16–64 bytes per object is achievable only on an exact match against the catalog (AH-05). This is the same fundamental limit as the block above: an address is meaningful only together with a catalog where the object already exists.

On dimensionality: 2^B does not depend on it, but dimensionality changes which bits land in one cube when REAL data is tiled — and therefore how many unique cubes occur in practice. That is measured by a scan (part B of the experiment), not by a formula.

Formulas: number of matrices = 2^B; storing the contents = 2^B × B bits; the ordinal address of one matrix = B bits (2^B states). All numbers are computed with exact integer arithmetic (python), not estimates.

How much space the ACTUALLY encountered matrices need (scan experiment)

A real CROSS-DEVICE corpus of 13.48 GB (a union of three hosts: arcana-devs 9.06 + arcana-www 3.38 + arcana-prod 1.04 GB; the hosts exchanged only cube hashes) was tiled into cubes of B = 4096 bits (512 bytes) in four layouts of the same length: a 1D string of 4096, 2D 64×64, 3D 16×16×16, 4D 8×8×8×8. A layout groups DIFFERENT bits of the file into one cube (strided tiling with a 4096-byte row), so the number of unique cubes on real data differs — even though the number of possible ones (2^4096) is identical.

layout	cubes scanned	unique	% unique	cross-host overlap (cubes)	store the unique
1D 4096	25 890 520	14 965 607	57.80%	1 847 617	7.14 GiB
2D 64×64	19 368 448	12 077 059	62.35%	1 260 924	5.76 GiB
3D 16×16×16	13 959 168	8 020 728	57.46%	943 354	3.82 GiB
4D 8×8×8×8	10 518 528	5 578 363	53.03%	768 085	2.66 GiB

There is NO saturation: the number of unique cubes grows almost linearly to the very end of the scan (devs-slice curve: 1D 10.9M unique at 7.3 GB → 12.3M at 8.3 GB) — real data at this cube length hardly “runs out”. On the devs slice, 72–82% of unique cubes were seen exactly once — the same measured law as AH-19 (89.9% on CDC chunks): an uncurated matrix is mostly dead weight. Cross-host overlap is real (0.77–1.85M cubes are shared between hosts), and it is what lowers the union corpus unique share to 53–62%.

Disks for the scan itself: 2.7–7.1 GiB of unique matrices per 13.48 GB of cross-device data — a fraction of one 1 TB disk. Extrapolating to the world’s stored data (on the order of 10 ZB = 10^22 bytes; order-of-magnitude per IDC Global DataSphere reports): with the measured 53–62% unique share on the cross-device corpus and NO saturation, the matrices would take ~5.3–6.2 ZB — billions of 1 TB disks, i.e. the same order as the data itself. THE EXTRAPOLATION IS LINEAR AND MARKED AS AN ASSUMPTION: the world corpus differs in composition, and saturation at larger volumes is not excluded — it was simply not observed in the scan.

Conclusions: (1) even storing only the ENCOUNTERED matrices at world scale is the same order as the data itself — the Addresser’s win lives not in a “warehouse of all cubes” but in deduplicating the repeated share (38–47% on the union corpus) and in r≥2 curation (AH-19); (2) the optimal layout by the “fewer unique” criterion is 4D (53.0% unique vs 62.4% for 2D on the union corpus) — multidimensional grouping does gather repeating bits more often, with a moderate margin (~6–9 pp); (3) the comparison is honest with a caveat: layouts need different alignment (2D — 256 KB blocks, 4D — 2 MB), so corpus coverage differs — figures are per each layout’s covered share.

Scripts: probe_matrix_scan.py + matrix_scan_dump.py (strides in the header; the cross-device union exchanges hashes only, MTX-scan-crossdevice-v1); cube hash blake2b-96; saturation curve checkpointed every 500 MB; all figures measured, the extrapolation labeled.

Wave 1 — research complete

Wave-1 deep research is complete: each of the 24 hypotheses carries a real measured verdict (full-cost charged accounting, falsification test executed, script+SHA on the card). GO means the mechanism works and is measured; NO-GO means it is closed by measurement or strict arithmetic. Predicted levers remain predictions and are labeled; the measured numbers live in every card below.

NO-GO · 9 GO · 15

shared context (dictionary / fragments) · 4 identity dedup — reference ≪ payload on exact match · 3 structurally cannot win (boundary) · 4 infrastructure cost accounting · 10 near-match + delta · 3

Generated: 2026-08-28T21:13:56Z · db:addressor_hypotheses

Hypotheses · 24

Page 1 / 12

AH-01 NO-GO W1 · 2026-07-13

Matrix as a shared dictionary

If a versioned, immutable global dictionary is fixed as a matrix section and transmission becomes dict_id + a stream compressed against it, then net bytes on small structured files (<128 KiB) drop versus plain local zstd, because a small file never has time to learn its own statistics.

Data class (Z)

small structured files (JSON/HTML/logs/configs)

Address target

n/a

Predicted lever

>=25% on smalls (prediction), tending to 0 above 4 MiB

Ceiling category

shared context (dictionary / fragments)

Mechanism

ready-made context from the very first byte; one-time dictionary delivery amortized

Falsification test

corpus of real small files: gain <25% or the dictionary never amortizes -> NO-GO

Full cost (total_cost)

address: 8 B dict_id · metadata: zstd frame header · residual: the ENTIRE compressed stream (not a reference scheme) · amortized: dict_size/N_files per device + one-time dictionary delivery.

Probe verdict

NO-GO by the pre-registered bar: a real zstd dictionary (110 KiB, hash-parity train/test split) yielded 2.94 pp absolute / 9.95% relative versus plain zstd-3 — far from the predicted >=25%. The lever exists but its magnitude is eaten by the strength of the local codec itself (Gotcha #11 family); the dictionary remains the baseline competitor for fragment schemes.

Measured result (raw research log; descriptors partly in the RU original)

date: 2026-07-14
probe: W1-fragments-v1b-fair

split

test_bytes: 305.87 MiB (320 725 794 B)
test_files: 59 936
train_files: 61 704

class_z

small structured files (<128 KiB, text-ish), real KB

verdict: NO-GO
dict_bytes: 110.00 KiB (112 640 B)
gain_abs_pp: 2.94 pp
gain_rel_pct: 9.95%

verdict_note

NO-GO by the pre-registered bar: a real zstd dictionary (110 KiB, hash-parity train/test split) yielded 2.94 pp absolute / 9.95% relative versus plain zstd-3 — far from the predicted >=25%. The lever exists but its magnitude is eaten by the strength of the local codec itself (Gotcha #11 family); the dictionary remains the baseline competitor for fragment schemes.

L1_dict_ratio: 0.2664
L0_plain_ratio: 0.2959

AH-02 GO W1 · 2026-07-13

Matrix as a CAS chunk store (CDC)

If the matrix is a global content-addressable store of immutable CDC chunks, then net transmitted bytes on recurring content collapse to a sequence of 32 B hash references, because the receiver already holds the chunks and CDC survives shifts and inserts.

Data class (Z)

OS images, packages, re-shared files, backups

Address target

32 B

Predicted lever

hit_rate x (1 - 0.4%) by mass; enterprise backup literature reports 10-30x (external, not our measurement)

Ceiling category

identity dedup — reference ≪ payload on exact match

Mechanism

exact chunk dedup against the catalog; the receiver stores the chunks

Falsification test

chunk hit-rate by mass on a realistic multi-device mix; <30% -> only as a router mode (AH-18)

Full cost (total_cost)

address: 32 B per chunk on hit · metadata: file manifest (chunk list, lengths, order) · residual: new chunks in full plus their upload · amortized: CAS storage x time + catalog (AH-09/10) + integrity (AH-21) + the first transfer of every block (AH-19).

Probe verdict

GO: chunk dup mass 43.01% on the 3-host union (the <30% falsification bar did not fire) plus a MEASURED competitive win of the charged Addresser over local zstd-3 (+7.81 pp with the 11.6 pp chunk-context penalty). The verdict is conditioned on realistic corpus composition; multi-device emulated by hash aggregation (AH-23), payload never left the hosts. MULTI-CLASS (2026-07-15): GO holds across all 7 real classes — chunk dup from 7% (media) to 68.4% (archives), cross-device slice 5.8-27.6%; no class refutes the lever. CDC-SENSITIVITY (2026-07-15): net-charged dup plateaus at 33.86/33.18/32.25% for 4/8/16 KiB average chunks (one walk, three chunkers): smaller chunks gain gross dedup (+1.1 pp) but pay more in refs+catalog (+0.47 pp); the 8 KiB operating point is robust — the parameter is non-critical (±0.8 pp across a 4x range).

Measured result (raw research log; descriptors partly in the RU original)

cdc

max: 65 536
min: 2 048
avg_chunk: 8 192

date: 2026-07-14
probe: W1-dupmass-v0

corpus

host: arcana-devs
files: 204 275
roots: ~/arcanada~/cubrim-sources~/cubr-cm-work/cubrim-code

caveat

single-host snapshot; cross-device realism via AH-23; worktree copies inflate dup-mass

total_bytes: 7.93 GiB (8 517 809 533 B)

verdict: GO

multiclass

date: 2026-07-15
probe: DR-multiclass-v1
cross_pools: prod∪www device-external

chunk_dup_pct_by_class

	cross	intra
web	27.23	27.16
code	19.73	56.68
docs	11.53	24.26
media	6.89	7.02
other	5.82	40.98
config	16.2	52.85
archives	27.59	68.42

competitive

date: 2026-07-14
codec: zstd-3 both sides (level cancels out)
probe: W1-competitive-v1

corpus

host: arcana-devs
files: 197 910

caveat

single-host; worktree copies inflate dup-mass; cross-device pending AH-23

total_bytes: 8.11 GiB (8 709 420 841 B)

local_ratio: 0.5562

charge_terms

refs_bytes: 40.98 MiB (42 971 680 B)
catalog_bytes: 20.69 MiB (21 690 132 B)
residual_comp_bytes: 3.82 GiB (4 099 360 171 B)

verdict_note

REAL competitive win of the charged Addresser over local compression on this corpus (+7.81 pp) despite the 11.6 pp chunk-context penalty; the operator's criterion (net bytes < local compression with reference << payload) is MET on the single-host slice; GO after cross-device confirmation (AH-23, delivered).

win_delta_pct: 7.81%
addresser_charged_ratio: 0.4781
chunk_context_penalty_pct: 11.6%

crossdevice

date: 2026-07-14
probe: W1-crossdevice-v1

union

hosts: arcana-devsarcana-prodarcana-www
total_bytes: 12.05 GiB (12 941 464 032 B)

chunk_dup_intra_pct: 26.8%
chunk_dup_total_pct: 43.01%
chunk_dup_cross_device_pct: 16.21%

charge_terms

catalog_bytes: 20.89 MiB (21 907 639 B)
addr_manifest_bytes: 40.99 MiB (42 980 416 B)

chunks_total: 934 588

verdict_note

GO: chunk dup mass 43.01% on the 3-host union (the <30% falsification bar did not fire) plus a MEASURED competitive win of the charged Addresser over local zstd-3 (+7.81 pp with the 11.6 pp chunk-context penalty). The verdict is conditioned on realistic corpus composition; multi-device emulated by hash aggregation (AH-23), payload never left the hosts. MULTI-CLASS (2026-07-15): GO holds across all 7 real classes — chunk dup from 7% (media) to 68.4% (archives), cross-device slice 5.8-27.6%; no class refutes the lever. CDC-SENSITIVITY (2026-07-15): net-charged dup plateaus at 33.86/33.18/32.25% for 4/8/16 KiB average chunks (one walk, three chunkers): smaller chunks gain gross dedup (+1.1 pp) but pay more in refs+catalog (+0.47 pp); the 8 KiB operating point is robust — the parameter is non-critical (±0.8 pp across a 4x range).

chunks_unique: 638 474

cdc_sensitivity

date: 2026-07-15
probe: DR-cdc-sens-v1 sha:8cc2983c1f8f7d4e
corpus_bytes: 8.60 GiB (9 233 769 960 B)

gross_vs_overhead

4KiB: 34.911.052
8KiB: 33.770.584
16KiB: 32.610.353

net_charged_dup_pct

4KiB: 33.86%
8KiB: 33.18%
16KiB: 32.25%

chunk_dup_mass_pct: 28.09%
naive_stored_ratio: 0.7191
charged_stored_ratio: 0.7267

Page 1 / 12

Where the Addresser cannot beat local compression

Six boundaries are fixed by wave 1 as explicit anti-hypotheses and no-win zones: unique high-entropy data (personal media, encrypted streams — nothing repeats globally); tiny unique files below the inversion point (fixed catalog costs exceed any saving); mathematically generated matrices (an address into an exhaustive or random pool costs at least as much as the content itself); a byte histogram or hash used as the data carrier (order is lost and buying it back costs the file’s entropy); long-tail content fetched roughly once (the first transfer is never repaid); and fragment schemes whose gain collapses into what a shared dictionary already provides. A tiny fixed-size reference per object (tens of bytes) is honestly achievable only on an exact match against a catalog that already stores the object; everywhere else the win criterion is simply that the reference plus all charged costs stay well below the payload it replaces — the reference size is a metric to minimize, not a hard gate.