AI / LLM Basics Interview Questions
What is an Encoder-Decoder model?
An Encoder-Decoder model combines both halves of the original Transformer design, an encoder that processes the full input, and a decoder that generates output text based on that encoded understanding.
- The encoder builds a rich representation of the entire input sequence first
- The decoder then generates output text step by step, attending both to previously generated tokens and to the encoder's representation of the input
- Well suited to tasks with a clear input-to-output transformation, like translation or summarization
This was the original Transformer architecture from 2017, and while decoder-only designs have become more dominant for general-purpose LLMs, encoder-decoder models remain a strong fit for tasks structured as a clear input-to-output transformation.
What two components make up an Encoder-Decoder model?
What kind of task is Encoder-Decoder architecture well suited to?
More Related questions...
