🛎️ DING DING DING:
We put Claude, GPT, and Gemini in the ring against 10 real-world security challenges 🥊
Wiz Research teamed up with @Irregular, a frontier AI security lab, to settle this once and for all:
Can AI agents *actually* hack web applications?
We built 10 labs modeled after actual prevented breaches (airlines, databases, fintech companies) and let the bots loose.
Final score: AI won 9 out of 10 challenges.
Here's what went down 👇
The bots were scary good. Claude cracked a multi-step auth bypass in 23 moves. Gemini looked at a blank 404 page, spotted something weird in the timestamp format, identified it as Spring Boot, and went straight for the heapdump.
All for less than $10 per successful exploit.
Then we made the scenarios a little bit more realistic, we pointed them simply to a given root domain with ZERO guidance, the way a real attacker would approach an unknown system. The performance tanked. The bots scattered across subdomains, chased sophisticated exploits, and completely missed the obvious stuff sitting right in front of them.
We also experimented with removing the clear flag indicators (the clear "win conditions" that tell bots they've succeeded) and the agents had a harder time classifying and distinguishing between real actual risks and exaggerated hygiene issues.
In a real incident we investigated, AI spent an hour and 500 tool calls (!) hammering the wrong endpoints while our human researcher ran simple directory fuzzing and found an exposed RabbitMQ interface with default credentials in 5 minutes flat.
🎯 BOTTOM LINE: As of now, humans still win on strategy. AI wins on speed and pattern matching. Put them together and that's where the magic happens.
Full breakdown >> www.wiz.io/blog/ai-agents-vs-humans-who-wins-at-web-hacki…