AI / LLM Basics Interview Questions
What is Self-Attention?
Self-attention is attention applied within a single sequence, each token attends to every other token in that same input, including itself, rather than attending to a separate sequence.
- Every token generates a Query, Key, and Value vector from its embedding
- The Query of one token is compared against the Keys of every token to compute relevance scores
- Those scores weight each token's Value, producing a new representation that blends in relevant context from across the sequence
This is the specific form of attention used throughout an LLM's internal layers, letting the model build up an increasingly context-aware representation of each token as it passes through the network.
Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!
Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.
Invest now!!! Get Free equity stock (US, UK only)!
Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.
The Robinhood app makes it easy to trade stocks, crypto and more.
Webull! Receive free stock by signing up using the link: Webull signup.
More Related questions...
