Recently I thought “If AI scrapers are scraping my website, would a prompt injection work? Just adding invisible Prompt commands …?”
And just today, a colleague sent me this link to an article about prompt injection in GitLab Duo: Remote Prompt Injection in GitLab Duo Leads to Source Code Theft:
TL;DR: A hidden comment was enough to make GitLab Duo leak private source code and inject untrusted HTML into its responses.
https://www.legitsecurity.com/blog/remote-prompt-injection-in-gitlab-duo
Well – it shows: damit! Someone else was faster! 😀
But besides that: it confirms a paranoid thought that I have been harboring for quite a while. Any output of an AI system must not be trusted blindly.
(more…)