One decode tick (1 token) through three architectures on the same machine (2× RTX 5060 Ti).
Packets = real reads: weights from VRAM
active experts
loop layers pass 1
loop layers reused (passes 2/3)
KV cache. Packet size ≈ bytes read.
Measurement basis: the RESULTS.md files of the three projects.
The three data-flow stories
Qwen3.8-27B (dense): There is exactly ONE source: the ENTIRE weight set (~16.3 GiB) has to pass through the compute unit once per token. 64 layer applications, each reads its full weight slice (red packets). No router, no sleeping weights, no reuse — a pure bandwidth marathon. MTP n6: one pass serves ~2.2 tokens (the packet is effectively shared).
Maxwell-V1-Flash (MoE 35B/3B-A): Per application the router wakes only 12 of 256 experts + 1 shared — ~90 % of the weights SLEEP (dark tiles, no packets!). The green packets are tiny, hence 113.8 t/s. BUT: the yellow KV packets grow with depth (49 KB/token × depth, dense attention) — from ~16k they overtake the weight packets and throttle the token.
Inkling-2-Mini (loop-MoE 26B/5.4B-A): Only 24 physical layers, but the middle 12 run THREE TIMES with the same weights (loop zone, ×3). Pass 1 reads fresh (purple packets), passes 2/3 read the SAME blocks AGAIN (gray packets — same bytes, compute time again!). Per application: 8 of 128 experts + NVFP4 dequant + top-8 routing. The packets are small, but there are 48 applications in series: overhead eats ~80 % of the time.
Prefill (pp) analogously: dense Qwen = one large GEMM per layer (1806 t/s); Maxwell feeds small expert GEMMs (up to 3402 t/s); Inkling has to traverse the loop zone 3× per ubatch + NVFP4 → only 190–475 t/s.