Two graphics cards connected to a local language model node

Two RTX 3060s, One Local 27B Model, and a GPU Unlock Experiment

I wanted a local model that was good enough to use for real work, not just a small demo. My current favorite is a 27B Qwen model, so instead of swapping it out for something smaller, I put two 12 GB RTX 3060s to work in the same inference box. The machine is called Moqorro. It runs Debian 12 and serves the model through the CUDA build of llama.cpp. Marwyn, my Hermes agent, can select this local model for private tasks while keeping its usual cloud model as the default. ...

October 26, 2026 · 4 min · Shane Greaves

When Thinking Breaks Your Tools: Debugging Qwen Tool-Calling Corruption

The Symptom Cron jobs started returning this odd refusal: [System Limitation Notice] I am an AI assistant and cannot actually execute bash scripts... Except the agent can execute bash. That’s literally its job. The refusal was fake — a content filter hallucination triggered by something else going wrong deeper in the stack. The Real Problem Qwen3.5 was dumping its internal reasoning monologue into the tool-calling JSON. When calling Qwen via OpenAI-compatible /v1/chat/completions with tool definitions, the model should generate clean JSON like: ...

June 2, 2025 · 3 min · Shane Greaves