Chinese AI Models Cybersecurity: What the Vals Test Found

What WIRED's Evaluation Found

The Vals assessment focused on whether AI models could carry out connected tasks across simulated business networks, rather than answer isolated cybersecurity questions. That distinction matters: completing a multi-step exercise requires a model to apply its capabilities over a sequence of objectives.

WIRED's report describes open-weight models as narrowing the gap with leading closed systems on these cyber tasks. Open-weight models make their underlying model weights available for others to use or adapt, while closed models are accessed through more restricted services. That difference can affect how easily a model can be obtained, modified, or deployed.

The reported results are a sign that strong cyber performance is not limited to models offered through tightly controlled platforms. They do not, however, mean that every open model performs equally well, or that benchmark results alone establish how a model would behave outside the evaluation.

How DeepSeek Performed on Mock Corporate Networks

DeepSeek completed an average of 73 percent of the objectives in Vals' evaluation. The figure reflects performance across the tested tasks, not a claim that the model can successfully compromise 73 percent of real corporate networks.

The test setting is important. Vals used mock corporate networks, allowing researchers to assess performance in a controlled environment. Such exercises can help compare models on defined cybersecurity tasks, but they are not a direct measure of success against live organizations, whose systems, safeguards, and circumstances vary.

The result is therefore best understood as evidence of capability within a benchmark, not as proof of real-world hacking success. The evaluation shows what a model managed in the conditions tested; it does not settle how reliably it could find or exploit weaknesses in an actual network.

What the Results Say - and Do Not Say - About Real-World Risk

The broader concern raised by the findings is about access. If open-weight models can approach closed models on cyber tasks, restricting access to frontier systems may not prevent similar capabilities from becoming available elsewhere. Open availability can make it easier for users to adapt a model, and may complicate efforts to manage how cyber-related capabilities are distributed.

That is a reason to take benchmark results seriously, not to treat them as a forecast of inevitable attacks. An evaluation on a mock network cannot capture every factor involved in real-world intrusion, and the reported score does not demonstrate that DeepSeek or other models can autonomously carry out successful attacks against real targets.

For defenders and policymakers, the practical takeaway is to track model capabilities alongside how models are distributed and used. Benchmarks can identify progress and help prompt further scrutiny, but they should be read with their test conditions in view. Results from simulated environments are informative, not interchangeable with evidence of real-world impact.

Our coverage of AI cybersecurity research examines how open-weight models are tested, what benchmark scores can show, and where their limits remain.

Key Statistics

• DeepSeek completed an average of 73% of evaluation objectives.Vals' evaluation, as reported by WIRED, measured performance on multi-step tasks in mock corporate networks. The figure is a benchmark result, not a real-world compromise rate.
• The reported evaluation used mock corporate networks rather than live organizations.WIRED's account describes Vals testing models in simulated environments. The article provides no real-world attack success rate, so the benchmark should not be interpreted as one.

Frequently Asked Questions