Flash-Tier LLMs Are the Real Inference Efficiency Story
Here's a claim that will annoy a lot of people who spend their days arguing about model leaderboards: for most production LLM workloads, the "smartest" model...
llm
All articles tagged with “llm”
5 articles
How Talorys Runs Stateful AI Agents on Serverless
Here's my contrarian take, and I'll defend it: a personal AI agent is a better fit for serverless infrastructure than it is for the Raspberry Pi under your...
Constrained Decoding: Turn LLMs Into Fast Decision Models
Here's a cheap trick that's quietly useful: if you only care about the first token an LLM would emit, you can ask it a multiple-choice question and get the...
LLM Agents Are Just Distributed Systems
The AI community is building multi-agent systems — teams of LLMs that divide work, communicate, and collaborate to solve problems no single model can handle. A...
Running Large Language Models on Your Own Hardware
Last year I spent $4,200 building a local inference rig that could run a 70B parameter model at reasonable speed. A colleague looked at my setup and asked the...