DSML Reading Club #8: memory for language models

Speaker: Ivan Rodkin

At this meeting, Ivan will discuss memory for language models. We will examine three major papers and their core ideas; Ivan is the first author of two of them.

At the meeting, we will discuss:

• Transformer limitations: quadratic attention and computational constraints
• The advantages and disadvantages of linear transformers: Mamba, RWKV, xLSTM, and DeltaNet
• Recurrent Memory Transformer and why it was chosen: arxiv.org/abs/2207.06881
• Associative RMT: arxiv.org/abs/2407.04841
• Reasoning in memory: arxiv.org/abs/2508.16745

Tuesday, February 3, 17:00 Kazakhstan time Google Meet: meet.google.com/rrn-enzr-chz