RNNs vs Transformers
The cluster centers on debates about whether a new neural architecture reinvents RNNs or LSTMs, with comparisons to transformers, discussions on their strengths, weaknesses, and modern alternatives like Mamba.
Activity Over Time
Top Contributors
Keywords
Sample Comments
Someone explain to me how this isn't reinventing LSTMs please.
yeesh it's an RNN and not even something like a transformer
Could be that or an RNN/LSTM as well
Are RNNs completely subsumed by transformers? IE, can I forget about learning anything about how to work with RNNs, and instead focus on transformers?
Please elaborate. What's wrong with RNNs and what should be used instead?
You're right, that does look similar... I expected this to be based on some type of RNN!
This sounds like an RNN with extra steps.
This seems like perfect fodder for an RNN.
Very cool ive read this line of paper originating from hippo, s4, hyena, mamba etc but can someone please explain how this isnt just an RNN/LSTM variant??
Unless he models are recursive neural networks :)