LLM Next-Token Prediction
The cluster revolves around discussions claiming that large language models (LLMs) do not understand or reason but simply predict the most likely next token based on statistical patterns from training data. Debates explore whether this mechanism implies a lack of true intelligence or if it enables sophisticated capabilities.
Activity Over Time
Top Contributors
Keywords
Sample Comments
It doesnt even really comply, Id say.It just predicts whats the most likely next text token.
nope. it understands nothing except the statistical link between a sequence of words and the next word in the sequence. read up before you lash out!
Has your LLM ever known just the right next token that should come after 2 million other tokens? Mine hasn't.
I'm really confused that people don't understand this. It's just predicting the most likely next text token and its trained on most internet text, so why would we expect anything at all different?
It doesn't learn. It simple completes statistically plausible sequences of tokens.
Is that not exactly what LLMs are built to do? Iterative next-token prediction?
Isn't that exactly what an LLM does? Predicting the next token?
Considering it’s trained on predicting the next word in stuff humans estimated before AI, wouldn’t that make sense?
The problem is showing that humans aren't just doing next word prediction too.
Important addition to your partially right statement: "they’re trained to generate ‘likely’ text" is they are trained to produce most probable next word so that the current context look as "similar" to training data as possible. Where "similar" is not "equal".