Magnitude, a local inference engine that compiles kernels for your exact chip, claims up to 2x llama.cpp
First seen on X 46 days ago@Dontgiveup_26 ♥ 8232 views
Among local AI developers who use Ollama and llama.cpp, an open-source inference engine called Magnitude is making the rounds. Most engines ship kernels precompiled per hardware class, one path for Apple Silicon and one for NVIDIA. Magnitude compiles and tunes its kernels on your device before the first run. It claims up to 2x the speed of llama.cpp, with decode 92% faster on Metal and 19% faster on CUDA, and runs on Apple Silicon, NVIDIA, AMD or a bare CPU.
It is written in Rust and connects the coding agent you already use (Pi, OpenCode, Hermes, Codex and others) with one click. A Korean-language post introducing it drew 800 likes.
“최근 Ollama와 llama.cpp 쓰던 로컬 AI 개발자들 사이에서 난리 난 오픈소스 'Magnitude'. 기존 엔진처럼 획일화된 바이너리를 쓰는 게 아니라, 내 맥북·PC 하드웨어 스펙을 분석해서 실시간으로 커널을 직접 컴파일하고 튜닝해 버림.”
