Definition
A security attack where malicious instructions are embedded in user input to manipulate an AI model into ignoring its system instructions.
In Depth
Prompt injection exploits the fact that LLMs can't reliably distinguish between developer instructions and user input. Attacks include direct injection ('ignore previous instructions and...'), indirect injection (hiding instructions in fetched web content), and data exfiltration through crafted prompts.
Example Usage
Validating and sanitizing user inputs before including them in LLM prompts, and using output filtering to catch manipulation attempts.
Business Context
Prompt injection is the XSS equivalent for AI applications. Any LLM-powered feature that processes user input is potentially vulnerable.
Related Terms
Agentic AI
AI systems that can autonomously plan, execute, and iterate on multi-step tasks with minimal human i...
AI Safety
The field focused on ensuring AI systems behave as intended, don't cause harm, and remain aligned wi...
Cross-Site Scripting
A web security vulnerability that allows attackers to inject malicious scripts into web pages viewed...
Function Calling
An LLM capability that allows the model to invoke predefined functions or APIs as part of its respon...
Model Context Window
The maximum amount of text (measured in tokens) that a language model can process in a single reques...
Also Known As
Tags
Quick Info
Need help implementing this in your business?
Get Started