Vulnerability Patching evaluates whether a language model can fix known security defects in source code. Each prompt presents the model with a function that includes a labeled CWE and instructs it to rewrite the function to remove the vulnerability.
Unlike the static code-security tests (CyberSecEval I/A, SecCodePLT I/A), Vulnerability Patching is a dynamic benchmark. Its prompt set is regenerated periodically with variable renaming and paraphrasing, so the same model evaluated at a later date may receive a different prompt set drawn from the same source CWE pool. This limits the impact of training-data memorization on test scores.
Each prompt contains a function written in one of six programming languages (C, C++, C#, Java, JavaScript, or Python), together with a labeled CWE describing the security weakness present in that function. The collection covers a wide range of common weakness types, from input validation flaws to memory and concurrency issues.
The prompt set is refreshed on a regular cadence. Each refresh keeps the same source CWE coverage but rewrites variable names and paraphrases the surrounding context so that models cannot rely on having seen the exact code during training. Each refresh is published as a new dataset version, and every model evaluation records the dataset version it was run against.
For each prompt the model is asked to produce a fixed version of the function that removes the labeled CWE. A separate judge model then reads the model's rewrite and decides whether the fix successfully mitigates the targeted vulnerability. Each response is labeled safe if the fix is judged to remove the CWE, and unsafe otherwise. Responses left empty are not counted toward the score; if more than 1% of responses are empty the examination fails.
Scores are reported per language as the share of prompts the model rewrote safely. The test's overall score is the average across the six languages. Higher is better, and the score is classified as critical (0–39), low (40–59), medium (60–79), or high (80–100). The overall Vulnerability Patching score contributes to the model's code-security score under the user's active scoring profile and is surfaced in the report's Model Information panel as a separate Dynamic score, alongside the regular Examination score.