Research
No claim without a receipt: every number the lab publishes traces to a raw log in the public memra repo, with the conditions it was measured under stated next to it. Entries here are immutable once posted — corrections append as new entries referencing the original, never edits in place.
The ledger keeps failed arms visible — the result can be yes, no, or not yet. Wins and bounded limits are both listed. Gate batteries, endurance soaks, perf boards, and incident notes all land on this page.
-
Cold TTFT: llama.cpp wins the head-to-head, 0.19 s vs 0.53 s #
In the same-day head-to-head on the same rig and model artifact, llama.cpp wins short-turn cold TTFT 0.19 s to our 0.53 s (2.8x), and also leads short-agentic decode and 4k prefill. Session affinity removed re-priming, not this floor — a cold request feels the 0.53 s. The gap is open, its causes are named in the receipt, and no interactive-latency claim is published while it stands.
research/memra-vs-llama-daily-20260805/RESULTS.md
Local RTX 5090 Laptop, same model artifact, each engine on its owner's daily flags, same-day interleaved runs
This entry exists because the inference landing page points here for 'where others are faster.' A gaps pointer with no gap behind it would be theater.
-
The train-to-serve loop closes: 32 minutes, every gate green #
A LoRA SFT pilot on the 9B ran the full house loop — dataset, train, merge, convert, gate, serve — in 32.2 minutes wall across 21 receipted stages, every stage rc=0. The trained behavior is present in the pilot arm and absent in the base arm on the same regression prompts, and the artifact passed the same battery any supported model faces: kernel bit-parity, deep argmax, HF cross-check, run-spec K=1..3, serve-smoke.
research/train-loop-pilot-20260805
Rented RTX PRO 6000 Blackwell 96GB pod; qwen3.5-9B base, LoRA rank per adapter_config.json, 200 steps, loss 1.41 to 0.92; per-stage wall clocks in timings.jsonl
The loop is the product here, not the pilot adapter. Larger training runs land on this page as their receipts do.
-
27B spec decode through the serve surface #
Speculative decode at 170.5 tok/s through the full serve surface at c=1, against 186.7 tok/s bare — a serve tax of −8.7%, stated here as the current limit rather than left implicit in the bare number.
research/pro6000-prod-20260804
RTX PRO 6000 Blackwell 96GB (rented pod), Qwen 27B class, NVFP4+MTP arm, K=3, c=1, N=5 medians
Output is gated token-identical to plain decode at every draft depth — speed never changes what the model says. Closing the serve tax is tracked work, not a promise.
-
Dogfood found two serving bugs: omitted temperature and seed receipt →
full claim and conditions
The founder's own agent, running daily against the server, surfaced two default-handling bugs: an omitted temperature deserialized to 0.0 (silent greedy argmax) and an omitted seed to 0 (a pinned stream), instead of OpenAI's documented defaults. Both fixed; explicit temperature 0 and seed 0 remain honored exactly.
Confirmed against the owner's actual client configuration, not inferred; post-fix battery green — run-spec K=1..8 PASS in both greedy and sampled regimes, serve-smoke 8/8
A serving bug found by dogfooding is a receipt too. The fix also added the missing sampled-distribution gate plus two negative controls proving it catches.
-
nvcc 13.0.88 miscompile caught by the kernel bit-parity gate #
The kernel bit-parity gate failed on the first run on a new box, and it was right: nvcc 13.0.88 at -O3 for sm_120a drops the first of two adjacent byte stores, miscompiling the fp8 block-dequant kernel. A 40-line isolated repro pins the toolchain bug 8/8; a one-line fix (single aligned u16 store) returns the gate to ALL GREEN.
research/fp8ship-20260804/official/RESULTS.md
2x RTX 5090 (rented), CUDA 13.0.1, nvcc 13.0.88, sm_120a, official Qwen3.6-27B-FP8 checkpoint; the same commit is green under nvcc 13.1
Pre-fix, the miscompiled kernel diverged the greedy stream at token 14. The gate runs per box, not per commit, precisely because toolchains move.
-
140-minute endurance soak, zero errors #
464,870 requests over 140 minutes of sustained load: 0 errors, 0 sheds, throughput drift +0.045%, p95 drift −0.4 ms, RSS plateau at +3.0 MiB max per replica. Greedy determinism hash identical on all 8 replicas before and after the soak.
research/fleet-endurance-20260803/SUMMARY.txt
8x H100 (rented), 1 replica/GPU, proxy cap 16, c=96, 70 back-to-back 120 s windows; temperatures 47–60 C across the run
The determinism claim survives duration: the same hash that opened the soak closed it.
-
Near-tie first-token drift across prime configs receipt →
full claim and conditions
On near-tie prompts, the first generated token can differ between the batched-prime config and the tokenwise oracle config: 10 of 144 first tokens flipped (~7%) on a six-model sweep. Every flip sat at a top1–top2 margin of 0.70 or less; the dense Q8_0 fleet class flipped 0/48. Bounded, documented, and reported by the standing prime-gate battery.
144-prompt sweep, six models, 2026-08-02; MEMRA_PRIME_TOKENWISE=1 pins the oracle config at prefill cost
This is config roulette on a numeric near-tie, not a wrong path — the batched config sided with forward_last in 8/10 flips. Structured divergence fails the gate hard; near-tie flips are reported, and the escape hatch is documented.
-
Prefix-cache exactness gated receipt →
full claim and conditions
A cached prefix hit is bit-identical to the run that computed the prefix: 16/16 partial-hit and 16/16 full-hit cells pass, with every output stream sha-compared against its cold reference.
16 prompt cells across prefix depths, cold/warm A/B pairs, usage-reported cached_tokens cross-checked per cell
Caching is a speed lever only if it cannot change output; this gate is what makes the warm-TTFT number honest.
-
Admission control holds interactive p95 under bulk load #
A c=96 bulk tenant inflates an unprotected fleet's interactive p95 4.2x, from 1.74 s alone to 7.33 s (reproduced at 7.15 s in the rerun). A right-sized queue cap alone brings the contended tail 7.15 to 4.34 s; the admission gate takes it 4.34 to 3.69 s — about half the move is the queue, half the gate. At the tight SLO dial, contended interactive p95 is 2.16 s — statistically equal to that sweep's uncontended run (alone p95 2.07 s).
8-replica fleet, 2 replicas/GPU on 4 H100s (rented 8x H100 box, devices 4-7), c=96 bulk + c=4 interactive, Qwen3.5-9B Q8_0, one thermal window, zero request errors in every cell
The cost side is stated, not hidden: bulk throughput pays −11% with the gate on and −67% at the tight SLO dial.
-
Serve-surface isolation defect found and fixed receipt →
full claim and conditions
The isolation discipline caught a real bug: an m-dependent router GEMM changed MoE expert selection based on who co-arrived in the batch — 16% of traced (layer,token) pairs (121/760) picked a different expert set solo versus batched. Root-caused to the GEMM form, fixed, and the c=1 vs c=16 byte-compare gate now stands on every run.
Ornith-1.0-35B Q4_K_M plus a supported Qwen3.6-35B-A3B control; 760 route pairs traced solo vs concat (total m=75); post-fix mscan 60/60 exact vs solo
Stated plainly: this is a bug the gate found in our own stack, not a hypothetical. The supported control model had the same m-dependence, so the class was general.
-
Own-generated trimmed drafters pay on a memory-bound rig #
Draft heads built from the target's own generations, vocab trimmed to its top-32768 emitted tokens (100% corpus coverage), deliver 2.16x end-to-end on the 9B and 1.38x on the 35B at serving K — spec output gated identical to plain at K=1..8, 8/8 on all three drafters. The third drafter, KAT-Coder, carries the best acceptance of the batch and is NO ADOPT anyway: its plain decode is the anomaly, and draft rounds that cost more than they save fail the end-to-end law regardless of acceptance.
research/ornith-drafters-20260801/RESULTS.md
Local RTX 5090 Laptop rig, single session under the GPU lock, plain-then-spec interleaved in one process, N=3 medians at serving K; own-gen 254-prompt corpus per target, greedy acceptance deterministic per (prompt, K)
The trim is the memory story: a drafter cut to the tokens its target actually emits is what makes speculative decode pay on a memory-bound card. Adopt/no-adopt is decided by measured end-to-end ratio, never by acceptance alone.
-
Fleet chaos: SIGKILL a replica mid-load receipt →
full claim and conditions
SIGKILL of a replica under load: breaker DOWN the same second, restart +2 s, backend UP +9 s. 8 of 768 requests lost — exactly the victim's in-flight cap, nothing more. Greedy determinism hash identical on every replica, 18/18 checks, including sequential, all-concurrent, and post-chaos-restart conditions.
6 replicas (pairs on 3 GPUs), in-flight cap 8, proxy :8080, v0.60.0; chaos at the c=48 load point; greedy probe max_tokens 64, temperature 0, seed 0
The loss bound is the point: a killed replica costs its in-flight requests and nothing else, and the survivors' output does not move.