What are Large Language Models (LLMs) and how do they generate text?

Large Language Models (LLMs) are neural networks — almost universally transformer-based — trained on massive text corpora to learn the statistical patterns of language. At inference, they generate text autoregressively: given a sequence of input tokens, the model produces a probability distribution over the entire vocabulary for the next token, a token is sampled from that distribution, appended to the sequence, and the process repeats until a stop token or length limit is reached.

This generation process is controlled by several parameters. Temperature scales the logit distribution before softmax — temperature < 1 sharpens the distribution (more deterministic, picks the most likely token more often), temperature > 1 flattens it (more random and creative). Top-k restricts sampling to the k highest-probability tokens; top-p (nucleus sampling) restricts to the smallest set of tokens whose cumulative probability exceeds p. These prevent sampling from extremely low-probability tokens (gibberish) while preserving diversity.

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from environment

response = client.chat.completions.create(
    model='gpt-4o',
    messages=[
        {'role': 'system', 'content': 'You are a helpful assistant.'},
        {'role': 'user',   'content': 'Explain transformer attention in one paragraph.'},
    ],
    temperature=0.7,     # creativity knob: 0=deterministic, 2=very random
    top_p=0.95,          # nucleus sampling: sample from top 95% mass
    max_tokens=300,
)

print(response.choices[0].message.content)
print('Tokens used:', response.usage.total_tokens)