Detection
Monitoring AI tools for signs of prompt injection
Prevention is partial, so detection matters. Log what goes into and out of your AI tools, and treat odd tool activity as a signal.
Last reviewed against NCSC (UK), OWASP Gen AI Security Project and UK Government (DSIT). Sources
What to log
The records that help
The NCSC's advice is to keep records detailed enough to reveal suspicious activity, possibly down to the complete text going into and coming out of the model, plus which tools and APIs it used [1]. It also advises understanding how an attacker might corrupt the system, what they might aim for and what normal behaviour looks like [1].
Inputs
What the tool was asked and what content it pulled in, including documents and pages.
Outputs
What the tool produced, especially anything containing links, addresses or encoded text.
Tool and API calls
Which actions the tool attempted, and whether they succeeded.
What to look for
Signals worth a review
Repeated failed tool or API calls can be an early sign of an attacker refining an attack [1]. Other signals to define with your team:
- Actions that do not match the user's request
- Requests to unfamiliar addresses
- Output that echoes unusual instructions found in content [4]
- Sudden use of tools a user rarely touches
Who reviews it
Make it someone's job
Logs nobody reads do not help. Decide which team owns AI tool alerts, and add the tools to your incident plan. The NCSC advises leaders to understand the impact if an AI system's integrity, availability or confidentiality is compromised, and to have a response plan [3].
Pair this page with the plan to reduce prompt injection risk.
Keep privacy in mind
Prompts and outputs can contain personal data. Check retention and access with your data protection lead before enabling full logging.
Keep reading
Read next
Timeline
Dated events on prompt injection, from NCSC guidance and the OWASP ranking to EchoLeak, the lethal trifecta and OpenAI's browser warning.
Read the guide
Questions
Sourced answers on prompt injection: what it is, whether it can be fixed, what indirect injection means and what UK organisations should limit.
Read the guide
Am I exposed?
A checklist for UK organisations: which AI tools read outside content, hold private data or can act, and so could be manipulated by prompt injection.
Read the guideHow CyPro can help
Need a hand with any of this?
Patching is one job. Keeping a vulnerable system safe until it is patched, and proving it, is several more. CyPro's consultants, testers and SOC analysts can take any of them off your plate.
Vulnerability scanning
Find every internet-facing system you run, including the forgotten ones, and see which are still on a vulnerable build.
See the scanning service
Hardening and segmentation
Our consultants put the controls in place that limit the damage when something is missed: firewall rules, network isolation and tighter admin access.
More on cypro.co.uk
24/7 security monitoring
Our SOC analysts watch your systems day and night for the signs of an attack in progress: unexpected processes, new outbound connections, logins that do not fit.
More on cypro.co.uk
Penetration testing
A focused test of your systems and the controls around them, so you know they hold before an attacker finds out for you.
More on cypro.co.uk
Incident response
If you may already be compromised, our responders contain it, preserve the evidence, find out what was touched and get you running again safely.
More on cypro.co.uk
Fractional CISO for the board
A senior security lead who advises the board on what to fix first, what to accept and what to tell customers, and owns the decision record.
More on cypro.co.uk
Know your exposure
Not sure you are protected?
Prompt injection cannot be patched away, so the useful question is how far a manipulated tool could reach. Testing your AI-connected systems the way an attacker would shows where the limits on data and actions need tightening.