AI / LLM Basics Interview Questions
What is Temperature in LLM sampling?
Temperature is a setting that controls how random or predictable a model's word choices are when generating text.
- A low temperature, close to zero, makes the model consistently pick its highest-probability next token, producing more focused, repeatable output
- A higher temperature flattens the probability distribution, giving lower-probability tokens more chance of being picked, producing more varied, creative output
- Applied right before the model samples its next token, after the softmax step has produced the probability distribution
Choosing temperature is a real trade-off: lower values suit tasks needing consistency and precision, like code generation, while higher values suit tasks that benefit from variety, like brainstorming.
What does a low temperature setting produce?
What task type suits a higher temperature setting?
More Related questions...
