GitHub Pages live demo Repo: accelerando-developers/inkling-2-mini-llamacpp Full benchmark report: docs/BENCHMARKS.md Model links: Inkling 2 Mini · Qwen3.8-27B · Maxwell-V1.0-Flash (open-llm-radar.com)

Data-Flow Atlas — where do the bytes come from, and when do they flow where?

One decode tick (1 token) through three architectures on the same machine (2× RTX 5060 Ti).  Packets = real reads: weights from VRAM active experts loop layers pass 1 loop layers reused (passes 2/3) KV cache. Packet size ≈ bytes read.  Measurement basis: the RESULTS.md files of the three projects.
0 (empty)

Where does the time go? (decode tick, current setting)

How big is a step — decoding 1 token (ms/token → t/s)

Measured ladder at this depth

The three data-flow stories

Qwen3.8-27B (dense): There is exactly ONE source: the ENTIRE weight set (~16.3 GiB) has to pass through the compute unit once per token. 64 layer applications, each reads its full weight slice (red packets). No router, no sleeping weights, no reuse — a pure bandwidth marathon. MTP n6: one pass serves ~2.2 tokens (the packet is effectively shared).
Maxwell-V1-Flash (MoE 35B/3B-A): Per application the router wakes only 12 of 256 experts + 1 shared — ~90 % of the weights SLEEP (dark tiles, no packets!). The green packets are tiny, hence 113.8 t/s. BUT: the yellow KV packets grow with depth (49 KB/token × depth, dense attention) — from ~16k they overtake the weight packets and throttle the token.
Inkling-2-Mini (loop-MoE 26B/5.4B-A): Only 24 physical layers, but the middle 12 run THREE TIMES with the same weights (loop zone, ×3). Pass 1 reads fresh (purple packets), passes 2/3 read the SAME blocks AGAIN (gray packets — same bytes, compute time again!). Per application: 8 of 128 experts + NVFP4 dequant + top-8 routing. The packets are small, but there are 48 applications in series: overhead eats ~80 % of the time.
Prefill (pp) analogously: dense Qwen = one large GEMM per layer (1806 t/s); Maxwell feeds small expert GEMMs (up to 3402 t/s); Inkling has to traverse the loop zone 3× per ubatch + NVFP4 → only 190–475 t/s.

Measurement basis & model assumptions of the phase bars