Cybersecurity professionals shouldn’t be too quick to celebrate AI’s ability to find software vulnerabilities, because finding the flaws is only part of the problem, a former top U.S. government cyber official said on Monday.
“Some set of folks will say, ‘That’s wonderful, we’re going to have LLMs scanning all of our software and finding bugs at scale and patching it before the bad guys can get leverage,’” Rob Joyce, who served as President Donald Trump’s top cyber adviser during his first term, said at Google’s Cyber Defense Summit in Washington. “Well, the problem with that theory is, we suck at patching.”
Google and other big tech companies may be able to quickly triage and patch the flaws that AI identifies, “but there’s so much technology in our ecosystem now that’s either unsupported or legacy or doesn’t have the person who can install a patch,” said Joyce, who held top roles at the National Security Agency, including head of its Cybersecurity Directorate and chief of its elite Tailored Access Operations hacking unit.
AI has already outpaced humans in finding software flaws, Joyce said. In June, an AI agent called XBOW topped HackerOne’s leaderboard, becoming the first non-human vulnerability reporter to do so. XBOW has remained on the leaderboard ever since.
“It is going after these networks, and it jiggles every doorknob, everywhere, constantly,” Joyce said, “and it finds more vulnerabilities and flaws than any human who has to sleep, eat and spend time with their loved ones.”
Unsupported or poorly maintained software will increasingly become the biggest source of risk in a world where AI can identify vulnerabilities faster than people can find or fix them, Joyce said.
“We may see the equivalent of a West Coast firestorm that has to burn things to the ground for us to build up stronger and better,” Joyce warned.
Agentic AI hijacking
During his keynote conversation with Mandiant chief analyst John Hultquist, Joyce also warned companies connecting AI agents to their email platforms, knowledge bases and other business systems that they were exposing themselves to severe new risks.
“There’s examples recently of AI being exploited inside a company where … [hackers] get access inside the systems of the company, but then they’re using the AI agents to search for the things that would be most useful in a ransomware attack or an extortion attack,” Joyce said. “We’re seeing the first malware come in that runs LLM queries against your data to find the things that they would most like to weaponize against you.”
With AI providing an entry point into valuable corporate systems, Joyce said, North Korea’s profit-motivated hacker army is likely to “get really good at attacking AI systems.”
“There’s money there,” he said, “and they have shown that they’re really creative.”