STILLPOINT
Your agent starts already knowing the project.
We don’t cache replies. We compile your knowledge into the model’s standing place and restore it from disk — so the next session does not pay to rediscover what the last one already hiked.
PIN-0001 is green. Exact restore across a full engine restart at 26k tokens: 7.0 s vs 33.9 s cold (4.9×), 98.7% hit. Suite 12/12, measured 2026-08-21.
Footprints vs. standing place
Classic transformers remember by keeping a footprint for every token. Hybrid GDN models fold the past into one fixed-size recurrent state. That changes what a cache can be.
Without vs. with
Hosted APIs already prefix-cache. That cache is best-effort and dies with idle time or a restart. Stillpoint commits the standing place to disk — these are measured numbers from PIN-0001, not a projection.
Compile the hot prefix once. Later sessions skip the hike — and the win grows with the working set. Measured 2026-08-21 (Qwen3-class 27B, vLLM + LMCache, TP=2), after a full engine restart. End-to-end wall: 2.4× at 5k · 4.0× at 13k · 4.9× at 26k. On time-to-first-token — the prefill the restore actually replaces — 3.5× / 5.0× / 5.4× (26k: 33.5 s cold → 6.2 s restored). 97–98.8% cache hit, exact answer every run.
Knows the project
A plain API has no knowledge of your repo until you paste it. A warm Stillpoint session starts with that knowledge already absorbed.
Cache that survives
Provider prefix caches evaporate. The standing place is on disk — still warm after restarts, deploys, and weekends. PIN-0001 proves it: kill the server, restore, exact answer.
Pay per job, not per re-read
Token pricing bills you for re-reading the same docs. Compile once, restore after — the re-reading line item is the one we want gone.
No cross-contamination
Verified: a never-stored context gets 0% cache hits and a clean cold prefill. Dedicated hardware, knowledge in your Git. Per-tenant salted keys are PIN-0002.
How it fits together
Three pieces. Knowledge lives in Git. It is compiled through the model once. The standing place is stored where a process restart cannot kill it.
Exact: first product is lossless prefix restore, not CacheBlend on GDN pages. Isolated: a never-stored context gets 0% hit and a clean cold prefill — verified. Recompile when the prefix changes — unchanged hot docs stay warm; a new doc at the front is a new hike.
Not a generic LLM API. Not CacheBlend. Today the pin is Qwen3.8 GDN on dedicated hardware — one architecture, one box, one working set.
An open lab, not a black box
Stillpoint is the open-source lab of 4rce.com. Every finding is a numbered pin — a committed, reproducible result on the path to persistent state for hybrid GDN models. The hard parts are public.
We traced the failure to three bugs in LMCache 0.5.3 — a layout misread that silently dropped 24 of 25 kernel pages, a shadowed variable that disguised every failure as a worker timeout, and async host-buffer corruption from mutating shared metadata. All three are on the LMCache side; vLLM needed no changes.
The 4-file patch is verified byte-for-byte against a pristine wheel, and the fix is geometry-driven, not GPU-count-driven — proven at TP=2 (25:1 page geometry) and TP=1 (17:1). That is a robustness claim no speedup number can make.
lmcache-053-pin0001-hybrid-fa.patch
·
sha256 cee37350…
·
full explainer