AWS documentation defines prompt injection as a security and safety risk in which a user crafts input designed to override, manipulate, or bypass system-level instructions, safeguards, or intended model behavior. The example provided is a classic prompt injection attempt, where the user explicitly instructs the AI assistant to ignore prior rules and operate without restrictions.
In this scenario, the attacker is not exploiting training data or causing factual errors, but rather attempting to change the control flow and behavior of the AI system through malicious or manipulative prompts. AWS identifies prompt injection as a critical risk for generative AI systems, especially those exposed to end users through chat interfaces, APIs, or customer-facing applications.
The other options do not apply. Data bias relates to skewed or unfair training data. Hallucination refers to generating incorrect or fabricated information. Data exposure involves leaking sensitive or private data. None of these describe an attempt to override system instructions.
AWS recommends multiple mitigation strategies for prompt injection risks, including instruction hierarchy enforcement, prompt isolation, input validation, output filtering, and grounding responses using techniques such as Retrieval Augmented Generation. AWS also emphasizes the importance of clearly separating system instructions from user inputs to prevent unauthorized behavior changes.
Prompt injection is categorized by AWS as part of Responsible AI and security governance, highlighting the need for robust guardrails when deploying AI assistants in production. Therefore, the correct answer is prompt injection.
