OMP NInfer
Durable local inference for coding agents
Your coding session should survive the process.
OMP NInfer is the qualified local inference appliance for Oh My Pi (a coding-agent client). It runs Qwen3.8 27B through the NInfer engine on one NVIDIA RTX 5090, 4090, or 3090 and preserves explicit OpenAI Responses continuation state across process restarts — a durable primitive, not a lucky prefix-cache hit.
109,589 tokens restored after a docker restart · 0.778 s to first token vs 47.920 s cold · 144.80 tok/s on the agent-shaped gate.
Use it when
- You use Oh My Pi and own a qualified RTX 5090, 4090, or 3090.
- Qwen3.8 27B is the model you want, private and on your own hardware.
- Sessions are long-lived, and restart recovery matters more than model breadth.
Use something else when
You want a broad model catalog (Ollama), a desktop GUI (LM Studio), maximum portability (llama.cpp), or multi-user serving (vLLM). The comparison is candid about this — the narrow target is what makes the recommendation credible.
How it fits together
Oh My Pi (client) → OMP NInfer (qualified appliance + releases) →
NInfer (engine) → Qwen3.8 27B (model). Loopback-only, bearer-authenticated,
fail-closed instead of cloud fallback, every byte hash-pinned.
Full detail: canonical facts.