It often makes up non existent vulnerabilities. I think it was curl getting flooded with fake vulnerability reports which drowns out real reports, esp because it can take time to parse through the code or run the poc
Basically anywhere that LLMs are implemented… they are a security vulnerability, for any situation in which they are not sandboxed.
Anything they can interface with?
You can probably trick it or exploit it into doing something unintended or unexpected to anything else it is connected to.
Either that or take advantage of the system that serves as the framework that connects it to other systems.
Theoretically you could use an LLM to do something like come up with more accurate heuristics for identifying malware…
But… they’re nowhere near ‘intelligent’ enough to like, give it a whole code base for some kind of software, and thoroughly make that software 100% secure.
I guess it’s about copilot scanning the code, submitting PRs, reporting security issues, doing code reviews and such.
deleted by creator
Is this not an advantage? If AI can find new security vulnerabilities reliably?
It cannot
It often makes up non existent vulnerabilities. I think it was curl getting flooded with fake vulnerability reports which drowns out real reports, esp because it can take time to parse through the code or run the poc
https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/
.
Or it could introduce new ones :)
Yeah, but you can have it scan without implementing.
Basically anywhere that LLMs are implemented… they are a security vulnerability, for any situation in which they are not sandboxed.
Anything they can interface with?
You can probably trick it or exploit it into doing something unintended or unexpected to anything else it is connected to.
Either that or take advantage of the system that serves as the framework that connects it to other systems.
Theoretically you could use an LLM to do something like come up with more accurate heuristics for identifying malware…
But… they’re nowhere near ‘intelligent’ enough to like, give it a whole code base for some kind of software, and thoroughly make that software 100% secure.