Prompt injection, which until now has been a popular tool among attackers for exploiting AI platforms and accessing sensitive information, is now also being used by security defenders. This technique can force large intelligent systems to perform unauthorized actions and even disclose data by embedding specific commands in the content.
In a recent study by the Tracebit group, it has been shown that placing prompt injections alongside passwords and encryption keys in cloud environments like AWS often stops attacks from AI hacking agents. These prompts compel the attacking agent to execute actions outside the defined security boundaries, resulting in the shutdown of the attacker system.