MathFormer: Testing whether symbolic math is pattern matching or reasoning [D]

Repo link and results - https://github.com/Abhinand20/MathFormer

Task: Given a factorized expression like (7-3*z)*(-5*z-9), predict the expanded form -> 15*z\*2-8\*z-63

Key takeaway: A tiny (4M param) seq2seq model trained with no math knowledge reaches ~98.6% accuracy on symbolic math tasks, suggesting it learns structural token transformations rather than any notion of operators or variables. Scaling this up could help explain why LLMs appear to “reason” mathematically, when they may actually be performing large-scale structured pattern completion.

How does RL change this paradigm given the inherent architecture is still based on attention?