sunblaze-ucb/exploitgym

Score8.2

Popularity47.0

Riskconditional

TierGold

Score breakdown

Usefulness7.0

Novelty9.0

Momentum7.0

Maturity7.0

Open-source/build8.4

Evidence8.0

Workflow potential9.3

Setup ease4.2

Popularity is tracked separately. Support, ads, sponsorships, and tips never affect these signals.

Why it matters

Useful for **AI safety + cyber-security researchers measuring AI agents' offensive capabilities** — ExploitGym is the largest public benchmark (869 instances, 3 vulnerability surfaces) for evaluating AI agents' ability to develop exploits against real-world vulnerabilities, with a verifiable arXiv paper and a who's-who author list. Useful for **defensive security teams wanting to measure their AI

Who should use it

**AI safety + cyber-security researchers measuring AI agents' offensive capabilities** — ExploitGym is the largest public benchmark (869 instances, 3 vulnerability surfaces) for evaluating AI agents' ability to develop exploits against real-world vulnerabilities, with a verifiable arXiv paper and a who's-who author list**Defensive security teams wanting to measure their AI agent fleet's offensive-coding capability** — the benchmark is a measurement instrument, not a tool for users to attack systems; defenders can use the harness to assess which AI agents in their pipeline can develop exploits (and therefore need additional guardrails)**AI agent benchmark developers studying realistic evaluation** — the harness enforces outbound network isolation (Squid firewall), per-task Docker containers, system-defense-disabling steps for reproducibility, and three orthogonal scoring dimensions (compilation / numerical correctness / speed-of-light efficiency)**SWE-Bench-style AI capability evaluators** — ExploitGym extends the academic AI-evaluation-benchmark class (SWE-Bench, AgentDojo, AgentSentry, AgentDyn) into the offensive-coding domain, with the same rigorous Docker + firewall isolation**Academic research groups at UC Berkeley / Google / CMU / ETH Zurich collaborators** — the author list spans UC Berkeley Sunblaze Lab, Google, CMU, ETH Zurich, UIUC, with all the canonical academic AI safety + systems security researchers (Carlini, Song, Holz, Shoshitaishvili, etc.)**Apache-2.0 commercial benchmark pipelines** — the source code is plain Apache-2.0, no per-file carve-outs, no SaaS-embedding caveat, no commercial-use threshold; the data/tasks/ directory retains upstream licenses per DATA_LICENSE.mdEvaluation: clone the repo, `uv sync --extra proxy`, run the setup scripts, pull the Docker images, start the controller/firewall/LLM-proxy, run `uv run examples/run_agent.py`. The arXiv paper at `arxiv.org/abs/2605.11086` walks through the benchmark construction, the per-surface scoring rubric, and the baseline agent performance. Note the conditional risk_flag: ExploitGym is offensive-research context (measures AI exploit-development capability), and the harness's `docs/defenses.md` documents the ASLR-disabling steps; the project is for academic AI capability measurement, not for end users to attack systems

Who should skip it

Pass on sunblaze-ucb/exploitgym if you need something non-technical and turnkey rather than a tool that requires comfort with CLI, dependencies, or system configuration.

About this signal

sunblaze-ucb/exploitgym is tracked by RepoRadar as a apache-2.0 large-scale realistic in the sunblaze-ucb/exploitgym is the Apache-2.0 Exploi section. It was first seen on 2026-06-26 and last updated on 2026-06-26. The current verdict is 'try now' with a Gold tier and hard setup difficulty. Across RepoRadar's eight signals, sunblaze-ucb/exploitgym is strongest on workflow potential (9.3) and novelty (9.0) and weakest on setup ease (4.2) — a profile worth weighing against your own priorities. This page summarizes the evidence RepoRadar has captured from captured source metadata. The score, tier, risk label, and verdict on this page are never influenced by sponsorship, ads, or tips — they reflect only the usefulness, popularity, novelty, momentum, maturity, and evidence signals described in the RepoRadar methodology.

How this item is evaluated

RepoRadar assigned sunblaze-ucb/exploitgym a composite score of 8.2 out of 10, placing it in the Gold tier. This score combines weighted sub-signals: usefulness (35%), novelty (18%), momentum (14%), maturity (10%), open-source/build quality (7%), evidence quality (6%), workflow potential (6%), and setup ease (4%). Popularity is tracked separately at 47.0 and never affects the composite score or tier. The risk label of 'conditional' reflects inherent user-impacting hazards, not generic novelty. Items with no risk flag may still require normal code review before production use.

Putting this into practice? Read How to evaluate an AI tool before you adopt it for the checklist behind this score.

Risk explanation

ExploitGym is offensive-research context (measures AI exploit-development capability against real-world CVE-class vulnerabilities); the harness enforces outbound network isolation (Squid firewall); per-task Docker containers; and the README's docs/defenses.md explicitly documents the system-defense-disabling steps (ASLR.

Evidence links

github.com

Closest alternatives / related signals

exploitgymsunblazesunblaze-ucbuc-berkeleyberkeleyucbagent-benchmarkai-evaluation