Cubrim — Global Addresser

The honest limit, stated first

A fixed short code can distinguish only a finite number of states, while the space of possible files grows exponentially with length — so a short address alone can never uniquely denote every long sequence. An address is meaningful only together with a catalog where the object already exists. And any honest result must charge the full cost: the addresses, the metadata, the residual data — and the shared matrix itself, which is only worth its size when amortized across many files and devices. Cubrim-2 therefore does not promise to “compress any file into 16 bytes”. It maps where global addressing genuinely beats local compression — and records, just as openly, where it structurally cannot.

How much space would ALL possible matrices take

We count in bits. An N-dimensional cube is a visual arrangement of a bit sequence: a 4×4×4 cube is 64 bits laid out in three dimensions. Dimensionality (1D/2D/3D/4D) is a way to arrange the same bits, not a different amount of them. Hence the number of all possible matrices of length B bits is 2^B and depends only on B: a 64-bit 1D string and a 4×4×4 cube share the same 2^64 ≈ 1.8×10^19 states.

B, bits	cube example	all possible = 2^B	store them all (bytes)	fits in 20,000×1 TB?
8	2×2×2	2.56×10²	2.56×10² B	yes
27	3×3×3	1.34×10⁸	4.53×10⁸ B	yes
51	—(порог/threshold)	2.25×10¹⁵	1.44×10¹⁶ B	yes
64	4×4×4	1.84×10¹⁹	1.48×10²⁰ B	no
125	5×5×5	4.25×10³⁷	6.65×10³⁸ B	no
256	4×4×4×4 (4D)	1.16×10⁷⁷	3.71×10⁷⁸ B	no
1000	10×10×10	1.07×10³⁰¹	1.34×10³⁰³ B	no
4096	8×8×8×8 (4D)	1.04×10¹²³³	5.35×10¹²³⁵ B	no

Capacity yardstick: 20,000 disks of 1 TB = 2×10^16 bytes = 1.6×10^17 bits. The full-enumeration threshold is B = 51 bits: the last length at which all 2^B matrices (with their contents, 2^B×B bits = 1.44×10^16 bytes) still fit; at B = 52 they no longer do (2.93×10^16 bytes). A 4×4×4 cube (64 bits): just the 64-bit addresses of all 2^64 matrices would take 1.48×10^20 bytes ≈ 147.6 EB — four orders of magnitude beyond the yardstick. At B = 256 there are 1.16×10^77 matrices — still slightly fewer than the atoms in the observable universe (~10^80); the “more than atoms” line is crossed at B ≈ 266. Beyond that, super-exponential growth with no physical storage prospect whatsoever.

The honest conclusion: “all possible matrices” cannot be enumerated — 2^B diverges super-exponentially at lengths below a single machine word. So the Addresser stores not all possible blocks but only the ones ACTUALLY ENCOUNTERED (CAS/deduplication): their number is bounded by the volume of real data and obeys the track’s measured laws — 89.9% of blocks in an uncurated matrix occur exactly once (AH-19), and an honest 16–64 bytes per object is achievable only on an exact match against the catalog (AH-05). This is the same fundamental limit as the block above: an address is meaningful only together with a catalog where the object already exists.

On dimensionality: 2^B does not depend on it, but dimensionality changes which bits land in one cube when REAL data is tiled — and therefore how many unique cubes occur in practice. That is measured by a scan (part B of the experiment), not by a formula.

Formulas: number of matrices = 2^B; storing the contents = 2^B × B bits; the ordinal address of one matrix = B bits (2^B states). All numbers are computed with exact integer arithmetic (python), not estimates.

How much space the ACTUALLY encountered matrices need (scan experiment)

A real CROSS-DEVICE corpus of 13.48 GB (a union of three hosts: arcana-devs 9.06 + arcana-www 3.38 + arcana-prod 1.04 GB; the hosts exchanged only cube hashes) was tiled into cubes of B = 4096 bits (512 bytes) in four layouts of the same length: a 1D string of 4096, 2D 64×64, 3D 16×16×16, 4D 8×8×8×8. A layout groups DIFFERENT bits of the file into one cube (strided tiling with a 4096-byte row), so the number of unique cubes on real data differs — even though the number of possible ones (2^4096) is identical.

layout	cubes scanned	unique	% unique	cross-host overlap (cubes)	store the unique
1D 4096	25 890 520	14 965 607	57.80%	1 847 617	7.14 GiB
2D 64×64	19 368 448	12 077 059	62.35%	1 260 924	5.76 GiB
3D 16×16×16	13 959 168	8 020 728	57.46%	943 354	3.82 GiB
4D 8×8×8×8	10 518 528	5 578 363	53.03%	768 085	2.66 GiB

There is NO saturation: the number of unique cubes grows almost linearly to the very end of the scan (devs-slice curve: 1D 10.9M unique at 7.3 GB → 12.3M at 8.3 GB) — real data at this cube length hardly “runs out”. On the devs slice, 72–82% of unique cubes were seen exactly once — the same measured law as AH-19 (89.9% on CDC chunks): an uncurated matrix is mostly dead weight. Cross-host overlap is real (0.77–1.85M cubes are shared between hosts), and it is what lowers the union corpus unique share to 53–62%.

Disks for the scan itself: 2.7–7.1 GiB of unique matrices per 13.48 GB of cross-device data — a fraction of one 1 TB disk. Extrapolating to the world’s stored data (on the order of 10 ZB = 10^22 bytes; order-of-magnitude per IDC Global DataSphere reports): with the measured 53–62% unique share on the cross-device corpus and NO saturation, the matrices would take ~5.3–6.2 ZB — billions of 1 TB disks, i.e. the same order as the data itself. THE EXTRAPOLATION IS LINEAR AND MARKED AS AN ASSUMPTION: the world corpus differs in composition, and saturation at larger volumes is not excluded — it was simply not observed in the scan.

Conclusions: (1) even storing only the ENCOUNTERED matrices at world scale is the same order as the data itself — the Addresser’s win lives not in a “warehouse of all cubes” but in deduplicating the repeated share (38–47% on the union corpus) and in r≥2 curation (AH-19); (2) the optimal layout by the “fewer unique” criterion is 4D (53.0% unique vs 62.4% for 2D on the union corpus) — multidimensional grouping does gather repeating bits more often, with a moderate margin (~6–9 pp); (3) the comparison is honest with a caveat: layouts need different alignment (2D — 256 KB blocks, 4D — 2 MB), so corpus coverage differs — figures are per each layout’s covered share.

Scripts: probe_matrix_scan.py + matrix_scan_dump.py (strides in the header; the cross-device union exchanges hashes only, MTX-scan-crossdevice-v1); cube hash blake2b-96; saturation curve checkpointed every 500 MB; all figures measured, the extrapolation labeled.

Wave 1 — research complete

Wave-1 deep research is complete: each of the 24 hypotheses carries a real measured verdict (full-cost charged accounting, falsification test executed, script+SHA on the card). GO means the mechanism works and is measured; NO-GO means it is closed by measurement or strict arithmetic. Predicted levers remain predictions and are labeled; the measured numbers live in every card below.

NO-GO · 9 GO · 15

shared context (dictionary / fragments) · 4 identity dedup — reference ≪ payload on exact match · 3 structurally cannot win (boundary) · 4 infrastructure cost accounting · 10 near-match + delta · 3

Generated: 2026-08-29T00:23:18Z · db:addressor_hypotheses

Hypotheses · 24

Page 8 / 12

AH-15 GO W1 · 2026-07-13

Near-match address + delta (rsync kin)

If the address points to the nearest matrix fragment and only a delta is transmitted, then net bytes on versioned data fall to address + delta << full, because adjacent versions are close in content.

Data class (Z)

code, documents, backup chains

Address target

32 B

Predicted lever

large (rsync practice — external reference, not a measurement)

Ceiling category

near-match + delta

Mechanism

small deltas between versions; rolling-hash nearest search

Falsification test

charged: address + delta + catalog share vs local zstd-19 with dictionary on multiversion; loses -> NO-GO

Full cost (total_cost)

address: 32 B of the nearest fragment · metadata: delta application instructions · residual: the delta itself · amortized: a rolling-hash index over the ENTIRE matrix — a heavy catalog line item.

Probe verdict

GO — the verdict holds under the HONEST strong bar zstd --ultra -22 (+dictionary): charged delta 0.0338-0.0358 vs the 0.1409-0.3048 bar — 4.2-8.5x better, winning in every similarity decile (real zstd patch-from, 339 real git pairs). Scope unchanged: version-chain (base identity is free). Binary-media/config classes unmeasured (too few pairs in the KB git history); follow-up — backup chains. TAILS (2026-07-16): the config class measured at a small n=34 (honestly flagged) — delta 0.030 vs plain-22 0.298, the same order of win as docs/code; binary-media version pairs DO NOT EXIST in the fleet history (assets are add-only) — that class is covered by AH-05 identity dedup by construction, delta is not needed there. DEPTH+NEAR-MATCH (2026-07-16): (1) decay by ancestor depth is graceful — 0.021/0.054/0.106/0.201 at k=1/2/4/8 (n=28, flagged): even against a base 8 versions back, delta is 5x better than plain-22, so the backup chain can be sparse; (2) incremental near-match BEYOND version chains is alive: 5.72% of corpus bytes have a partner with >=20% chunk overlap (candidates are a free by-product of the CAS chunk->file catalog, NOT signatures), delta on them is 0.198 and wins on 100% of files — the pre-registered <5%-of-bytes bar did not fire. THRESHOLD/TOP-K/CROSS-DEVICE (2026-07-16): delta wins in EVERY overlap band down to 10-20% (0.666<1; bands 20+ give 0.12-0.44) — the near-match enable threshold can drop to ~10%, adding +2.3 pp of mass over the 20% threshold; a top-3 base concatenation yields only 0.971 vs best-1 — the complexity does not pay, one best base suffices; cross-device: 23.49% of devs bytes are covered >=10% by the other hosts’ pool, 19.72% fully (that is already pure CDC dedup), the partial band is ~3.8 pp — cross-device near-match exists on top of dedup, with the base assembled from shared chunks.

Measured result (raw research log; descriptors partly in the RU original)

date: 2026-07-15
probe: DR-delta-v1

tails

date: 2026-07-16

binary_media

finding

пар версий НЕ СУЩЕСТВУЕТ во всей истории флота (40000 коммитов): бинарные ассеты add-only — не мутируют

implication: класс покрывается identity-дедупом (AH-05), дельте там нечего делать

config_smalln

pairs: 34
probe: DR-delta-config-smalln-v1

caveat

малое n=34, порог ослаблен до 1КиБ

L22_plain: 0.2976
D22_delta_charged: 0.03

backend: zstd patch-from (RAWCONTENT dict + LDM) — реальный rsync-потомок
charges: 32Б адрес базы + 64Б манифест + 2Б каталог/объект (измеренная точка AH-09)

classes

	L3	pairs	L19_dict_bar	D19_delta_charged
code	0.3406	139	0.1411	0.0338
docs	0.4066	200	0.305	0.0358

verdict: GO

scenario

version-chain: база известна по идентичности пути (истории версий/бэкап-цепочки); generic-поиск ближайшего НЕ заряжается — он закрыт AH-14

v2_ultra22

date: 2026-07-15
probe: DR-delta-v2-ultra22

classes

	L22_dict_bar	D22_delta_charged
code	0.1409	0.0338
docs	0.3048	0.0358

baseline: zstd --ultra -22 + словарь (директива оператора: базлайн мирового бенчмарка)

sensitivity

выигрыш во ВСЕХ децилях похожести (D19/bar = 0.10-0.21); уровень 3 vs 19 — дельта доминирует на обоих

verdict_note

GO — the verdict holds under the HONEST strong bar zstd --ultra -22 (+dictionary): charged delta 0.0338-0.0358 vs the 0.1409-0.3048 bar — 4.2-8.5x better, winning in every similarity decile (real zstd patch-from, 339 real git pairs). Scope unchanged: version-chain (base identity is free). Binary-media/config classes unmeasured (too few pairs in the KB git history); follow-up — backup chains. TAILS (2026-07-16): the config class measured at a small n=34 (honestly flagged) — delta 0.030 vs plain-22 0.298, the same order of win as docs/code; binary-media version pairs DO NOT EXIST in the fleet history (assets are add-only) — that class is covered by AH-05 identity dedup by construction, delta is not needed there. DEPTH+NEAR-MATCH (2026-07-16): (1) decay by ancestor depth is graceful — 0.021/0.054/0.106/0.201 at k=1/2/4/8 (n=28, flagged): even against a base 8 versions back, delta is 5x better than plain-22, so the backup chain can be sparse; (2) incremental near-match BEYOND version chains is alive: 5.72% of corpus bytes have a partner with >=20% chunk overlap (candidates are a free by-product of the CAS chunk->file catalog, NOT signatures), delta on them is 0.198 and wins on 100% of files — the pre-registered <5%-of-bytes bar did not fire. THRESHOLD/TOP-K/CROSS-DEVICE (2026-07-16): delta wins in EVERY overlap band down to 10-20% (0.666<1; bands 20+ give 0.12-0.44) — the near-match enable threshold can drop to ~10%, adding +2.3 pp of mass over the 20% threshold; a top-3 base concatenation yields only 0.971 vs best-1 — the complexity does not pay, one best base suffices; cross-device: 23.49% of devs bytes are covered >=10% by the other hosts’ pool, 19.72% fully (that is already pure CDC dedup), the partial band is ~3.8 pp — cross-device near-match exists on top of dedup, with the base assembled from shared chunks.

depth_sensitivity

date: 2026-07-16
probe: DR-delta-depth-v1

caveat

n=28 длинных цепочек (>=9 версий)

chains: 28

delta_ratio_vs_plain22_by_depth

k1: 0.021
k2: 0.054
k4: 0.106
k8: 0.201

nearmatch_overlap

date: 2026-07-16
probe: DR-nearmatch-overlap-v1 sha:f6a760bdf642

sample

bytes: 765.05 MiB (802 210 346 B)
files: 5 372

delta_wins_on_pct_files: 100
delta_ratio_on_candidates: 0.198
files_with_partner_ge20pct_bytes_pct: 5.72%

overlap_sensitivity

date: 2026-07-16

topk

n: 61
verdict: не окупается
top3_concat_vs_best1: 0.971

xdev

probe: DR-xdev-overlap-v1
partial_band_pp: 3.77 pp
fully_covered_ge99.9pct: 19.72%
devs_bytes_covered_by_prodwww_ge10pct: 23.49%

probe: DR-overlap-sens-v1 sha:4bdf309ac992

delta_ratio_by_band

>=40: 0.239
10-20: 0.666
20-30: 0.12
30-40: 0.437

bytes_pct_with_partner_ge

10: 7.32
20: 5
30: 4.58
40: 3.97

falsification_executed: бар «не бьёт zstd-19 со словарём» НЕ сработал: дельта лучше бара в 8.5x (docs) и 4.2x (code)

AH-16 GO W1 · 2026-07-13

Impossibility class: unique already-compressed/encrypted — anti-hypothesis

If any addressing is applied to unique already-compressed or encrypted streams, then total_cost is strictly >= the original, because such streams are statistically indistinguishable from random, never recur in the global pool (hit-rate ~0), and addresses plus manifest are pure overhead.

Data class (Z)

personal .jpg/.mp4/.zst, TLS dumps

Address target

n/a

Predicted lever

boundary confirmation: 0 whole-chunk hits (predicted)

Ceiling category

structurally cannot win (boundary)

Mechanism

statistical randomness + absence of global repeats

Falsification test

hit-rate on a unique-media corpus; a systematic non-zero hit would refute (not expected)

Full cost (total_cost)

address+manifest: pure overhead · residual: the whole file · amortized: repaid by nothing — hit-rate ~0 by class construction.

Probe verdict

GO — the boundary is CONFIRMED on a measurement-verified unique corpus (the follow-up executed): 2,871 real media files (2.8 GiB) whose uniqueness is proven by the absence of copies across the whole fleet show only 2.75% repeated chunk mass (the residue being shared container/EXIF structures). For the truly unique class, addressing is pure overhead as predicted; class membership is decided by MEASURED dup-ness (the wave-1 key finding stands).

Measured result (raw research log; descriptors partly in the RU original)

date: 2026-07-14
probe: W1-unique-media-v1

by_ext

gz: 92.59
jpg: 93.54
mp3: 100
pdf: 100
jpeg: 93.51
png_2.99GiB: 8.82

corpus: 3347 media files 3.1 GiB (exact file-dups excluded), union 3-host occurrence pool
verdict: GO

verdict_note

GO — the boundary is CONFIRMED on a measurement-verified unique corpus (the follow-up executed): 2,871 real media files (2.8 GiB) whose uniqueness is proven by the absence of copies across the whole fleet show only 2.75% repeated chunk mass (the residue being shared container/EXIF structures). For the truly unique class, addressing is pure overhead as predicted; class membership is decided by MEASURED dup-ness (the wave-1 key finding stands).

verified_unique

date: 2026-07-15
bytes: 2.61 GiB (2 805 221 650 B)
class: медиа с fleet-wide whole-file occurrence == 1 (уникальность ИЗМЕРЕНА, не по типу)
files: 2 871
probe: DR-verified-unique-v1
repeated_chunk_mass_pct: 2.752%

repeated_chunk_mass_pct: 12.04%

Page 8 / 12

Where the Addresser cannot beat local compression

Six boundaries are fixed by wave 1 as explicit anti-hypotheses and no-win zones: unique high-entropy data (personal media, encrypted streams — nothing repeats globally); tiny unique files below the inversion point (fixed catalog costs exceed any saving); mathematically generated matrices (an address into an exhaustive or random pool costs at least as much as the content itself); a byte histogram or hash used as the data carrier (order is lost and buying it back costs the file’s entropy); long-tail content fetched roughly once (the first transfer is never repaid); and fragment schemes whose gain collapses into what a shared dictionary already provides. A tiny fixed-size reference per object (tens of bytes) is honestly achievable only on an exact match against a catalog that already stores the object; everywhere else the win criterion is simply that the reference plus all charged costs stay well below the payload it replaces — the reference size is a metric to minimize, not a hard gate.