AI / LLM Basics Interview Questions
Define the Transformer architecture?
The Transformer is the neural network architecture underlying virtually every modern LLM, introduced as a way to process sequences of text using attention instead of reading one word at a time in strict order.
- Built from stacked blocks, each containing an attention mechanism followed by a feed-forward network
- Processes an entire input sequence in parallel rather than word by word, making training dramatically faster on modern hardware
- Comes in three broad flavors: encoder-only, decoder-only, and encoder-decoder
Its core innovation, letting every token in a sequence directly weigh the relevance of every other token, is what let language models scale to the size and capability seen today.
What replaced strictly sequential, word-by-word processing in the Transformer?
How does a Transformer process an input sequence?
More Related questions...
