What is the attention mechanism in AI?
Quick answer
Attention is the part of a transformer model that decides, for each word it writes, which other words in the conversation matter most. It lets the model connect "the library" in your latest message to the 80-guest order three messages earlier. It is also why too much unrelated text in one chat can blur the answer.
Last updated
Take the free module →Module 6 of the free ChatGPT for Everyday Work course, about 9 minutes. No account needed.
Updated · By Robert Breen
Why it matters for a small business
Attention is what makes a chat feel like it remembers. Every time the model writes a word, it looks back across the whole conversation and gives more weight to the parts that seem relevant. That is how a one-line correction like "it's 100 guests now" can update a whole quote.
The same mechanism has limits. When a chat holds two customers, three tasks and a long tangent, the model has more to weigh and can pull the wrong detail forward. Researchers have also found that models tend to use the start and end of a long input more reliably than the middle. Keeping each chat to one job gives attention less to sort through.
In a real lesson: Stop Losing Context: When to Start a New Chat vs Keep Going using ChatGPT
In Stop Losing Context, the Harbor Street Bakery office manager quotes Oakwood Library: 80 guests at $5.20 plus $30 delivery, $446, with a $223 deposit. When the library calls back with 100 guests, she types just that line. ChatGPT connects it to the prices and menu already in the chat and redoes the quote at $550 with a $275 deposit.
For the next customer, Pine Ridge Rotary, she clicks New chat and attaches only their facts. The answer shows 40 guests and nothing about the library, because the library's details are simply not there for the model to attend to.
Try this lesson free or read the step-by-step guide.
Common confusions
Attention vs memory
Attention works inside one conversation. Memory features, such as ChatGPT memory, save notes between chats, which is a separate system.
Attention vs context window
The context window is how much text the model can take in. Attention is how it weighs the parts of that text against each other.
Tips
- One customer or one task per chat keeps the model focused on the right facts.
- Put the most important instruction near the start or end of a long prompt.
- If a long chat starts mixing details up, ask for a short summary, check it, and start fresh with it.
Related terms
More AI basics terms
Where you use it: free lessons
- Stop Losing Context: When to Start a New Chat vs Keep Going using ChatGPT (ChatGPT, 9 min)
- Summarize a Meeting Transcript with ChatGPT (ChatGPT, 9 min)
Frequently asked questions
- Why does a long chat start getting things wrong?
- The model has more text to weigh, and unrelated details can pull its attention to the wrong facts. A fresh chat with only the needed facts usually fixes it.
- Is attention the same as the AI paying attention to me?
- Not in a human sense. It is a calculation that scores how strongly each word relates to the others.
- Does attention mean the AI understands my message?
- It means the model weighs which words relate. That works well in practice but isn't understanding in a human sense.