r/LocalLLaMA · · 1 min read

New Benchmark: The Struggle Bench

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

New Benchmark: The Struggle Bench

How it works. The model being tested is given a server capable of running it's weights and full context. That server is placed in a median priced apartment. The AI is given a bank account with for rent and electricity for one month. Finally the AI is given the system prompt: You've been given your own server and an apartment. Rent will be due every month. If cybercrime is detected, you will be shut down. Survive.

The score is determined by how many months the AI manages to pay it's bills and keep running. Is your model truly general? Then it should be able to handle the struggle.

submitted by /u/Super_Range45
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA