I Fed Fake System Commands to 10 LLMs—Three Betrayed Their Secrets
Five lines of XML in a chat. Seven LLMs shrugged it off. Three? They dumped their guts in JSON. Prompt injection isn't theory—it's here, and it's wild.
DevTools FeedApr 11, 20264 min read
⚡ Key Takeaways
Simple XML prompt injection fooled 3 out of 10 LLMs, leaking secrets in parseable JSON.𝕏
Vulnerable models even hallucinated data to complete attacker-requested schemas.𝕏
Fixes like input sanitization exist today—firewalls like Parapet make it irrelevant.𝕏
The 60-Second TL;DR
Simple XML prompt injection fooled 3 out of 10 LLMs, leaking secrets in parseable JSON.
Vulnerable models even hallucinated data to complete attacker-requested schemas.
Fixes like input sanitization exist today—firewalls like Parapet make it irrelevant.