r/MachineLearning · · 1 min read

Built an AI Accelerator and opensourced it. [P]

Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.

Built an AI Accelerator and opensourced it. [P]

There is a huge gap in open source AI accelerators, so I implemented mine.

Popular and well known ones are already legacy and doesn't support contemporary operations like Attention.

Here is what makes mine special:

  • Attention mechanism smelted directly into silicon
  • Prototyped end-to-end on FPGA (AWS F2)
  • Benchmarked against PyTorch-based workloads
  • Built on the RocketChip architecture (RISC-V)
  • Native BF16 support
  • Up to 225× speedup on vanilla attention mechanism
  • Up to 96× speedup on TinyBERT
  • Up to 50× speedup on ViT Up to 30× speedup on GPT-2 prefill

I would really appreciate it if you check the repo and give me feedback!

submitted by /u/Barrnie
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/MachineLearning