Chain of Thought, letting the model think
One word before the answer changes quality drastically. Here's how CoT works in practice.
The experiment
Ask a model: "What is 3468 times 729?"
The direct answer is often wrong. Models are great at generating language but bad at mental arithmetic.
Now ask with one addition: "What is 3468 times 729? Think step by step out loud."
The answer is correct. The reason: the model now generates the intermediate steps, and each step is easier. The model literally "computes" better when allowed to think out loud.
Why this works
LLMs are token generators. Each token gives the next token more context. When you force the model to think, more relevant intermediate results sit in the context before the final answer arrives. The effect is measurable on complex tasks (math, logic, code review). The longer the reasoning chain, the bigger the gap between "answer directly" and "step by step".
When to use Chain of Thought
- Math and logic puzzles
- Plan-style tasks with multiple steps
- Code problems where the "why" matters
- Decisions where you need the rationale
When not
- Simple lookups ("What is the capital of France?")
- Pure rewordings
- Style tasks
For simple questions, CoT makes the model more verbose without making it better.
The opposite, force a short answer
Sometimes you don't want an explanation, just the number. Then say: "Answer only, no reasoning." or "Max 10 words."
That's the second half: you control how loudly the model thinks.