OMP NInfer
Decision guide
Choosing a local backend for coding sessions
Every project below is excellent at a different job. Most operators should use one of them; this page exists to make that call fast and honest.
| Dimension | OMP NInfer | Ollama | LM Studio | llama.cpp | vLLM |
|---|---|---|---|---|---|
| Primary job | Durable OMP appliance | Easy model runner | GUI model runner | Portable engine | General serving |
| Model breadth | One model, hash-pinned | Broad | Broad | Broad | Broad |
| Explicit Responses continuation | Core product | Verify current support | Verify current support | Verify current support | Verify current support |
| Process-restart recovery | Transactional checkpoints, verified restore | Different contract | Different contract | Different contract | Different architecture |
| Off-machine checkpoints | Verified host/NAS replicas; matching runtime/profile/credentials for restore | Verify current support | Verify current support | Verify current support | Verify current support |
| GUI | No | Limited | Strong | No | No |
| Multi-user throughput | Not the target | Limited | Not the target | Limited | Strong |
| Best fit | One private, long-lived OMP session | Easy breadth | Desktop convenience | Portability | Serving scale |
Last verified 2026-08-31 against each project's public documentation. Capabilities change quickly — reverify before deciding.
OMP NInfer transport status updated 2026-09-13 from repository receipts. Checkpoint copying and NAS replication work now; they do not imply cross-GPU session migration. Restore boundary and evidence.
Qwen3.8 27B is the model. Neroued’s NInfer and its GPU ports supply the specialized engine; this project’s runtime adds explicit state and durable checkpoints. Oh My Pi owns tools and agent orchestration. Ordinary in-process prefix caching already avoids repeated prefill of matching input in other runtimes. Engine throughput and retained-state measurements are separate results, not a matched head-to-head speed ranking against these backends.
The one-sentence answer
For an Oh My Pi operator running Qwen3.8 27B on an eligible RTX 5090 or RTX 4090 who values private, restart-resumable long sessions, OMP NInfer is the strongest fit. Choose Ollama or LM Studio for breadth and convenience, llama.cpp for portability, or vLLM for serving scale.
v0.7.3 scope: OMP 18.2.3 with RTX 5090 on Windows 11 + Docker Desktop/WSL2 or RTX 4090 native Windows 11. Release state and acceptance. RTX 3090 is deferred; its historical v0.7.2 route remains on OMP 18.0.9, not qualified with the new client.
Full criteria: decision guide · prior art: related work