temperature-scaling

Constrained Decoding: Turn LLMs Into Fast Decision Models

Here's a cheap trick that's quietly useful: if you only care about the first token an LLM would emit, you can ask it a multiple-choice question and get the...