New 0.2.1: a more accurate memory check → Beta More models fit in 16 GB →

Run open AI models privately on your Mac or PC

LLMario is a free, open-source app for chatting with open models like Qwen, Gemma, Llama and gpt-oss on your own computer. It picks the right engine for your hardware, checks the model fits in memory before loading it, and never sends your messages anywhere.

Get started in 3 steps Download for macOS Download for Windows

Free · Apache-2.0 · macOS 13+ (Apple Silicon and Intel) · Windows 10/11 x64 (preview) · command line and local API included · all downloads and checksums

Beta New: LLMario Beta 0.3.0

More models fit in 16 GB of memory, with faster replies on MTP models and a speed estimate for each model. Installs next to LLMario without touching it.

Download Beta for macOS (.dmg) Download Beta for Windows (.exe)

Pre-release · macOS 13+ · Windows 10/11 x64 (not code-signed yet) · needs llama.cpp or MLX-LM · release notes and checksums

How it works

LLMario doesn't reinvent AI engines. It runs proven open-source engines for you, correctly configured for your machine.

You ask

Chat in the LLMario window, use the llmario command, or connect your own apps through the built-in local API.

LLMario plans

It detects your chip and memory, estimates what the model needs, and refuses with a clear reason if it won't fit, instead of freezing your computer.

The right engine runs it

MLX (Apple's engine, usually fastest on Apple Silicon) or llama.cpp (runs everywhere, including Windows). LLMario starts, watches and stops it for you.

Answers stay on your computer

Replies stream back as the model writes them. No accounts, no cloud, no telemetry. Your messages are never logged.

Get started in 3 steps

1
Install an engine

Engines do the actual AI work. Install one (or both):
Recommended: install both. MLX is usually fastest; llama.cpp runs the widest range of models.
```
brew install llama.cpp
```
```
pip3 install mlx-lm
```
No Homebrew? Get it at brew.sh.
Intel Macs use llama.cpp. MLX needs Apple Silicon.
```
brew install llama.cpp
```
No Homebrew? Get it at brew.sh.
Windows uses llama.cpp. Run this in PowerShell or Command Prompt, then restart LLMario so it finds the engine:
```
winget install ggml.llamacpp
```
No winget? Download a Windows zip from llama.cpp releases and add its folder to your PATH.
2
Install LLMario
Download LLMario for macOS (.dmg, Apple Silicon and Intel), open it, and drag LLMario into Applications. It is signed and notarized by Apple, so it opens without warnings.
Download for macOS

Download LLMario for Windows (10 and 11, 64-bit) and run the installer. It installs for your account only, with no admin rights needed. Windows 10 may also install Microsoft's WebView2 component; Windows 11 already has it.
Download for Windows
Preview: the installer isn't code-signed yet, so Windows may show "Windows protected your PC". Click More info → Run anyway.

Just the command-line tool? Download llmario for Windows (.zip), unzip it anywhere and run llmario.exe.
Builds and installs LLMario.app and the llmario command. Needs Rust and Xcode Command Line Tools (xcode-select --install). The script checks both first.
```
git clone https://github.com/mannysinghx/LLMario.git && cd LLMario && ./scripts/install-macos.sh
```
Just the command-line tool: add --cli-only. Check prerequisites without installing: --check.

On Windows, build the command-line tool with Rust and the Visual Studio Build Tools: cargo install --path crates/cli --locked in the cloned folder.
3
Pick a model and chat
1. Open LLMario: from Launchpad or Spotlight on a Mac, or the Start menu on Windows.
2. Click Models → Library. Every model shows what it's good at, its size, and whether it fits your computer.
3. Click Download on one marked recommended. A small model like Qwen3.5 4B or Gemma 4 E4B is a good first choice.
4. Select it in the top bar and start typing. The first message loads the model, which takes a few seconds.
Already have a model file? Drag a .gguf file (or, on a Mac, an MLX model folder) onto the window. It's used where it is, never copied.

What you get

🧭 Model library

37 current open-model families with exact Hugging Face names. Search by task (chat, reasoning, coding, agents, multilingual, long context, small) and see what fits your computer before downloading.

🧠 Memory check first

LLMario estimates weights plus conversation memory before loading and explains the limit, with a smaller setting that would fit.

⚙️ Right engine, automatically

MLX on Apple Silicon, llama.cpp on Intel Macs and Windows. LLMario also knows which model types your installed engine version can load.

🔒 Private by default

Runs offline once a model is downloaded. Local-only network access, no accounts, no telemetry, and messages are never written to logs.

✅ Verified downloads

Every file is checked against Hugging Face checksums and pinned to an exact version. Models already on your computer are reused.

🔌 Works with your apps

An OpenAI-compatible API on your own computer, so tools and scripts that speak OpenAI can use local models instead.

Models you can run

A curated library of current open models, verified against Hugging Face. The app shows the full list with sizes and licenses.

Good forTry

Everyday chat & writingQwen3.5 9B · Gemma 4 12B / 26B-A4B · Mistral Small 3.2 · Llama 3.1 8B

Reasoning & mathgpt-oss 20B · GLM-4.7 Flash · Qwen3.8 27B · Phi-4

Coding & agentsQwen3 Coder 30B-A3B · Devstral Small 2 · gpt-oss 20B

Small & fast (8 GB computers)Qwen3.5 0.8B–4B · Gemma 4 E2B/E4B · LFM2.5 · SmolLM3 3B · Llama 3.2

Long documentsGemma 4 (256K) · Qwen3.x (256K) · Nemotron 3.5 Lightning (up to 1M)

Full list with exact file names: docs/MODELS.md. Check each model's license before commercial use.

Configure

In the app: Settings

Setting	What it does
Performance profile	Latency: one chat, fastest replies (default). Balanced: up to 4 requests at once, e.g. your chat plus other apps. Throughput: many parallel requests.
Context length	How much conversation the model remembers. Longer uses more memory.
Max reply length	Upper limit on tokens per answer. Raise it for "thinking" models.
Temperature	0 = predictable, higher = more creative.
System prompt	Instructions applied to every chat.
Chat history	Stored only on this computer. Turn it off or clear it anytime.

Advanced: config file

Optional. Lives at ~/.llmario/config.toml (Windows: %USERPROFILE%\.llmario\config.toml) and applies to the app, the command line and the API.

[server]
port = 11500              # local API port

[runtime]
profile = "latency"       # latency | balanced | throughput
context = 16384           # tokens per conversation
memory_limit_gb = 24      # cap LLMario's memory use

[backends]
prefer = "mlx"            # mlx | llamacpp

See the effective settings with llmario config. Models are stored in ~/.llmario/models.

For developers

Command line

llmario doctor                    # your hardware, engines, what fits
llmario model catalog coding      # browse the library
llmario model pull qwen3.5-9b     # best version for your machine
llmario run qwen3.5-9b "Hello!"   # chat in the terminal
llmario serve                     # start the local API
llmario bench -m qwen3.5-9b       # measure speed and memory

OpenAI-compatible API

llmario serve listens on http://127.0.0.1:11500/v1, on your computer only unless you enable remote access with an API key.

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:11500/v1", api_key="local")
r = client.chat.completions.create(
    model="qwen3.5-9b",
    messages=[{"role": "user", "content": "Hello!"}])
print(r.choices[0].message.content)

Questions

Is it really private?

Yes. Models run on your own computer. After a model is downloaded, LLMario works offline. The API listens only on your computer by default, messages are never logged, and there is no telemetry.

Which computers work?

Mac: macOS 13 or later. Apple Silicon (M1 and newer) is fastest and can use both engines; Intel Macs use llama.cpp. Windows (preview): Windows 10 or 11, 64-bit, with llama.cpp. LLMario uses an NVIDIA graphics card when it finds one; otherwise models run on the processor, which is slower. Memory decides model size: 8 GB suits small models (1–4B), 16 GB mid-size (up to ~9B), and 32 GB+ larger ones. The app shows exactly what fits.

A model says "too large" or "needs a newer engine"

Too large: pick a smaller model, or lower the context length in Settings. Needs a newer engine: the model uses a new architecture your installed engine doesn't support yet. Update it with brew upgrade llama.cpp or pip3 install -U mlx-lm on a Mac, or winget upgrade ggml.llamacpp on Windows, or choose the model's other engine version.

"Engine not installed" / nothing happens when I chat

Install an engine (step 1), then reopen LLMario. Models → Engines shows what LLMario found. llmario doctor gives full details.

Windows says "Windows protected your PC"

The Windows preview isn't code-signed yet, so Microsoft SmartScreen shows this for new apps. Click More info → Run anyway. Only do this for the installer downloaded from this site or the GitHub releases page. You can check the download against its .sha256 file with Get-FileHash in PowerShell.

macOS says the app can't be opened

Official releases are notarized by Apple and open normally. If you received a build from someone else, prefer the GitHub release or install from source.

Does it use Ollama or send data to a server?

No. LLMario runs the open-source engines llama.cpp and MLX directly. It only goes online to download a model you choose from Hugging Face.

How do I remove it?

Mac: delete LLMario from Applications. Windows: Settings → Apps → LLMario → Uninstall. Models and settings are in ~/.llmario (Windows: %USERPROFILE%\.llmario); delete that folder to free the space. Remove a source-built command-line tool with cargo uninstall llmario.