Why 95 Models Beats One Perfect Model for AI Agents

A developer hacked a GPU to run one model. The real lesson is about choosing many.

A Show HN post hit 395 points last month: a developer built a custom C++/CUDA inference engine that runs Llama 3.1 70B on a single RTX 3090 by streaming model weights directly from NVMe storage to GPU memory, bypassing the CPU entirely. The RTX 3090 has 24GB of VRAM. Llama 70B needs roughly 40-140GB depending on quantization. The hack works by loading layers on demand through a direct PCIe data path.

It is genuinely impressive systems engineering. Custom kernel patches, VFIO device passthrough, a userspace NVMe driver, three-tier adaptive caching. The developer claims 83x speedup over naive memory-mapped loading.

The result: 0.2-0.5 tokens per second on a 70B model.

The Hacker News comments were predictably sharp. One commenter calculated that the electricity cost of generating 3,600 tokens at 200-300W system draw exceeds the cloud API price for the same output via OpenRouter. Another pointed out that a well-quantized 8B model on the same card runs at 30+ tokens per second – 60-150x faster. NVIDIA’s own research shows 40-70% of agentic tasks can run on sub-10B models.

The developer isn’t wrong for building it. But the discussion reveals a deeper question most AI users haven’t thought through: is the right strategy one powerful model, or the right model for each task?

The one-model trap

Most AI agent platforms lock you into a single model or a single provider. ChatGPT gives you OpenAI’s models. Claude gives you Anthropic’s. OpenClaw is technically model-agnostic, but you’re managing your own API keys, and most users default to whatever they set up first.

This creates a one-model trap. You end up using Claude Opus for everything – quick summaries, email drafts, data formatting, complex reasoning – because switching models means switching providers, managing new API keys, or reconfiguring your agent. The result: you’re paying premium prices ($5/M input, $25/M output) for tasks that a $0.02/M model handles perfectly well.

The RTX 3090 hacker fell into a version of this trap. The goal was to run one specific model (Llama 70B) at any cost – custom kernel patches, hardware modifications, 5 seconds per token. Meanwhile, a $5 credit pack on a cloud platform with 95 models would have given access to Llama 70B, plus 94 other options better suited to most of the tasks that model would be used for.

The 1,050x price gap

The price gap between AI models in March 2026 is staggering. The cheapest model on LikeClaw (Mistral Nemo at $0.02/M input tokens) is 1,050 times cheaper than the most expensive (GPT-5.2 Pro at $21/M input tokens).

That is not a typo. One thousand and fifty times. The gap on output tokens is even wider: $0.04/M (Mistral Nemo) versus $168/M (GPT-5.2 Pro) – a 4,200x difference.

Both models exist for a reason. Mistral Nemo handles summarization, formatting, classification, and simple Q&A competently. GPT-5.2 Pro solves problems that require extended multi-step reasoning across massive context windows. Using GPT-5.2 Pro for email drafting is like hiring a senior architect to paint a wall. The wall gets painted. But you’ve wildly overpaid.

Here is what the budget tier looks like in March 2026:

• Mistral Nemo – $0.02/M input, $0.04/M output. Basic tasks at near-zero cost.
• DeepSeek V3.2 – $0.26/M input, $0.38/M output. Strong general-purpose, handles most daily tasks at a fraction of flagship pricing.
• MiniMax M2.5 – $0.30/M input, $1.10/M output. Scores 80.2% on SWE-bench Verified – within 0.6% of Claude Opus 4.6 – at roughly 17x less cost.
• Gemini 2.5 Flash Lite – $0.10/M input, $0.40/M output. Million-token context window at budget pricing.
• Qwen3-Coder – $0.22/M input, $1.00/M output. Purpose-built for code generation.

And here is the premium tier for when you genuinely need it:

• Claude Sonnet 4.6 – $3/M input, $15/M output. The workhorse for serious analysis.
• GPT-5.4 – $2.50/M input, $15/M output. OpenAI’s latest flagship with 1M context.
• Claude Opus 4.6 – $5/M input, $25/M output. The best reasoning model available, 1M context.
• Grok 4 – $3/M input, $15/M output. Strong on real-time knowledge with 256K context.
• Gemini 3.1 Pro – $2/M input, $12/M output. Google’s flagship with million-token context.

All 95 models, one account, one set of credits.

The 80/20 rule of model usage

The Hacker News discussion surfaced a practical insight. NVIDIA’s research on agentic AI workloads found that 40-70% of tasks can run on sub-10B models – the tiny, fast, cheap ones. Only the remaining 30-60% need larger models with stronger reasoning.

This maps directly to how people actually use AI agents. Most daily tasks – summarize this document, draft this email, format this data, classify this text, answer this quick question – don’t require a 70B parameter model running at $5-25 per million tokens. A 7-14B model at $0.02-0.30 per million tokens does the job in less time at a fraction of the cost.

The smart approach: route 80% of tasks to budget models and reserve premium models for the 20% that genuinely need them. On LikeClaw, this happens naturally because cheaper models cost fewer credits. You’re not penalized for using a budget model. You’re rewarded.

Compare this to the alternatives. ChatGPT charges $20-200/mo regardless of which internal model handles your request – you’re paying the same whether it routes to GPT-4o-mini or GPT-5.4. OpenClaw users manually configure API keys and most never change the default. Subscription-based coding agents like Cursor give you a fixed number of “premium requests” and throttle you when they run out.

The real cost of AI agents in 2026 isn’t just about which model you’re using. It’s about whether your platform even gives you the choice.

Why local inference isn’t the answer for most users

The RTX 3090 hack is a proof of concept, not a pricing strategy. Running Llama 70B locally at 0.2-0.5 tokens per second means:

• 5 seconds per token. A 500-word response takes 15-20 minutes to generate.
• 200-300W power draw. At $0.15/kWh, that’s $0.03-0.045 per hour of continuous generation.
• $1,500-2,000 hardware cost for the GPU alone, plus NVMe drives, plus a system that meets the requirements (Linux kernel 6.17+, IOMMU disabled, secondary NVMe).
• Zero fallback. If the task needs a different model – say, one with better instruction following or tool use – you’re starting over.

The same 500-word response on DeepSeek V3.2 via cloud API takes 3-5 seconds and costs a fraction of a cent. On MiniMax M2.5, similar speed, similar cost. On Claude Opus for the hardest problems, maybe 10-15 seconds and a few cents.

Local inference makes sense for specific use cases: air-gapped environments, privacy-sensitive work, research into model internals, or hobbyist experimentation. For production agent workloads where you need speed, reliability, model variety, and cost predictability – cloud APIs with model choice win.

The subscription doesn’t give you choice

The counterargument to model choice is subscriptions: pay $20/mo for ChatGPT Plus and let OpenAI figure out which model to use. Simple.

But simplicity comes at a cost. You get one provider’s models. You pay the same $20 whether you use it twice or two hundred times (until you hit the cap, then you’re stuck). And when OpenAI’s model isn’t the best fit for your specific task – when Anthropic’s Claude handles your code review better, or Google’s Gemini processes your long document more accurately, or DeepSeek’s model costs 50x less for a simple summary – you can’t switch without buying another subscription.

Research shows that most professionals are paying for multiple overlapping AI subscriptions. The subscription stack – ChatGPT Plus ($20) + Claude Pro ($20) + Google AI Pro ($20) + Cursor ($20) – costs $80/mo minimum and still doesn’t give you access to DeepSeek, MiniMax, Qwen, Mistral, Grok, or any of the other models that might be better and cheaper for specific tasks.

Credits-based pricing with model choice solves this. One account, 95 models, buy credit packs when you need them. Use the cheap model for cheap tasks. Use the expensive model when it matters. Never pay for a model you’re not using.

The right model for the right task

The developer who built NVMe-to-GPU streaming for a single RTX 3090 solved a hard engineering problem. The discussion around it solved a harder strategic one: the future of AI agents isn’t about running one model as cheaply as possible. It’s about having access to the right model for every task and paying only for what each task requires.

Ninety-five models. Twenty providers. One set of prepaid credits. The cheapest at $0.02 per million tokens, the most powerful at $21 per million tokens, and everything in between. That’s model choice. That’s cost control. That’s how AI agents should work.