Sunday 25 October 2026 12:31:24 GMT+01:00

Netcrook

HomeManifesto
News
Techcrook
Geocrook
WikicrookTeamAppContact
EnglishItaliano

WIKICROOK

CyberGym

A benchmark for testing whether agents can reproduce vulnerabilities from unpatched code.

CyberGym is a benchmark for evaluating whether an AI agent can reproduce a known vulnerability from unpatched code. In the test, the agent receives a vulnerability description and the affected codebase, then must produce a working proof of concept that triggers the flaw. This makes the benchmark useful for measuring a system’s ability to move from analysis to exploitation in a controlled setting.

In cyber security, that matters because defenders often need to understand not just that a bug exists, but how it can be exercised. A strong CyberGym result can suggest useful skills for vulnerability triage, exploit validation, and remediation support. It can also reveal where agentic tools are weak, such as on unfamiliar code, complex control flow, or careful exploit setup. Still, CyberGym does not measure overall real-world security coverage, and a high score does not mean an agent can handle every target or attacker technique.

← WIKICROOK index