LLMs as Markov Chains

Cluster centers on debates comparing large language models (LLMs) like GPT to Markov chains, with commenters frequently dismissing LLMs as glorified or fancy Markov chains and questioning if AI text generation is just probabilistic next-token prediction.

➡️ Stable 1.2x AI & Machine Learning
1,960
Comments
20
Years Active
5
Top Authors
#5284
Topic ID

Activity Over Time

2007
5
2008
20
2009
32
2010
33
2011
38
2012
58
2013
70
2014
120
2015
118
2016
123
2017
73
2018
98
2019
113
2020
65
2021
65
2022
86
2023
210
2024
274
2025
355
2026
4

Keywords

e.g AI RNN LLM ANN MOST aclweb.org W05 imgur.com reddit.com markov chain chains word predecessor models tokens initial grams model

Sample Comments

juped Jan 8, 2020 View on HN

Markov chains are pretty smart - you've probably just trained yours more

pavel_lishin Mar 5, 2015 View on HN

This comment looks like a markov chain to me.

logfromblammo Feb 18, 2020 View on HN

You mentioned markov chains which were trained earlier. Tell me more about that.

sidpatil Aug 30, 2019 View on HN

A Markov chain would have been able to produce a text like that.

guluarte Dec 7, 2024 View on HN

yep, it is just a glorified markov chain

scrollaway Apr 7, 2023 View on HN

It doesn’t sound like you understand what a Markov chain nor an LLM is, you’re just parroting something you heard and that stuck in your head. Please don’t?

jimmaswell Jun 11, 2015 View on HN

Does this do any better than a markov chain?

plants Nov 14, 2018 View on HN

Isn't that kind of just a markov chain?

frou_dh Sep 6, 2019 View on HN

This kind of statement might as well be generated by a Markov Chain.

0x0 Jan 27, 2018 View on HN

Haha nice Markov Chain going there