AI Security Leaderboard: benchmarking model robustness [P]

We developed a leaderboard ranking frontier model security. There's no shortage of model capability rankings, but we didn't find anything comparable for model security. Yet security is becoming increasingly critical to deployment decisions: from the USG making developers pull models for cybersecurity jailbreaks to developers holding back on AI agent deployments due to risks of adversarial attacks.

We developed an automated test suite that runs models through 1500 automatically generated jailbreak attempts and measures the number of universal jailbreaks: prompts that elicit compliant, detailed responses to >75% clearly harmful questions within a domain (like offensive cybersecurity). We find a big gap between the most and least robust models in our technical report.

This is v1.0 and we'd really appreciate input from this subreddit on next steps, as well as feedback on the metholodogy. Areas we're considering include:
1. Adding open-weight models. How to fairly compare them to proprietary models as open-weight models naturally have a larger attack surface through perturbing weights (refusal abliteration, helpfulness fine-tuning)?
2. Adding new domains beyond CBRNE and cybersecurity, e.g. agent hijacking or harmful manipulation.
3. Increasing the realism of the domains, e.g. agentic cyber tasks rather than specific questions.
4. Stronger attacks: we aimed to keep these pretty basic for initial release but could add more powerful adaptive optimization attacks like boundary point jailbreaking.

We'd also love to hear ways we could make this benchmark more useful in your work. If you're an adversarial robustness researcher, are there artifacts such as datasets or evaluation rubrics you'd like to re-use?