r/LocalLLaMA · · 1 min read

A very important milestone for me in the AI field.

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

A very important milestone for me in the AI field.

https://preview.redd.it/wmuii8r68i1h1.png?width=1672&format=png&auto=webp&s=ab2a21eb9cc361fb2080ad90ec7207b0e1263419

Three days ago, I officially released my first AI research paper: STAM (Stable Training with Adaptive Momentum).

STAM introduces a new optimizer for deep learning and focuses on:

  • improving training stability,
  • reducing resource consumption during training,
  • and addressing several limitations found in optimizers like Adam, AdamW, and Muon.

The paper explains what makes STAM different, the problems it aims to solve, and includes comparisons with existing optimizers and training results.

The research paper is currently available on SSRN, and it has reached a ranking of around 646K so far.

What matters most to me is not numbers, but having AI engineers, researchers, and specialists read the paper and share honest technical feedback and criticism.

I consider STAM one of the biggest projects I’ve ever worked on, and I plan to continue improving and developing it further. I would genuinely appreciate hearing opinions from researchers and experienced people in the AI community about the paper, the optimizer design, and the reported results compared to other optimizers.

Research paper:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6699059

https://preview.redd.it/va8a5vbb8i1h1.png?width=1254&format=png&auto=webp&s=1f5edf7da0e1d7988b61dd735081e1f3d3a25c15

submitted by /u/assemsabryy
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA