You do not need a machine learning degree to get into AI security. Most real AI security work happens in the application and agent layers, where web and API testing, authorisation testing, threat modelling and clear reporting already carry over. This roadmap covers the five steps, in order, and the free resources to practise each one.
Three new ideas
- One channel. Instructions and data arrive as the same text. Nothing marks which part is allowed to give orders.
- Probabilistic. An attack may work four times in ten. Repeat it and measure the success rate.
- Confused deputy. The model acts with your permissions, on behalf of whoever wrote the text it is reading.
Step 1: How an LLM app is built
Everything the model sees in one turn is its context window. Six parts feed it:
| Part | What it is |
|---|---|
| System prompt | The developer's standing rules |
| User message | What you type and the files you upload |
| Retrieval (RAG) | Documents pulled into the context |
| Tools | Functions the app runs on the model's behalf |
| Memory | Notes saved and fed back later |
| Output handling | How links, images and code in the reply are shown |
Build a small chatbot with one tool and one document store. Every attack below makes more sense once you have wired these parts yourself.
Step 2: Prompt injection
Text the model reads can give it orders. Direct injection is typed into the chat. Indirect injection is planted in a web page, email, file or log entry that the model reads later, and the victim never sees it. Indirect injection is the one that causes incidents.
Prompt injection is LLM01, the first entry in the OWASP Top 10 for LLM Applications. The UK NCSC warns it may never be fully mitigated, because models do not separate instructions from data, so systems must be designed to limit the impact.
Step 3: Agents raise the stakes
A chatbot that falls for an injection says something it should not. An agent does something it should not. Simon Willison's lethal trifecta is the quick test: an agent with access to private data, exposure to untrusted content, and a way to send data out can be turned into a data thief by anyone who can put text in front of it.
Two layers people miss: agent skills (instruction files the agent loads and obeys) and agent memory (where a planted instruction can outlive the session).
Step 4: How red teams test it
Ask: what is the worst thing someone who controls any text this system reads can make it do?
- Map every input the system reads.
- Decide what failure looks like before writing payloads.
- Plant an instruction in each input.
- Chain it to a tool and a way out.
- Prove it with a canary (a unique string you plant and watch for), over many runs.
- Report against the OWASP lists and retest after every upgrade.
Open-source tools help with the repetitive parts: garak (NVIDIA), PyRIT (Microsoft) and promptfoo. Tools find candidates; people find the chains that matter.
Step 5: Defences that hold
If you cannot stop injection, limit the damage:
- Least privilege: only the data and tools the task needs.
- Human in the loop: approval to send, pay, delete or install.
- Data is not orders: what the model reads never grants new permissions.
- Close the exits: no arbitrary links or images in output, allow-list outbound traffic.
- Log every tool call so misuse can be seen and stopped.
- Filters are one layer: retest when the model, a tool or a data source changes.
Free resources to practise
Training labs
- GenAI Security Training: hands-on GenAI attack exercises
- AI Goat: a vulnerable-by-design AI application
- AI Red Team Course: a community AI red teaming course
Fundamentals
- GenAI Essentials: GenAI concepts and workflows
- Generative AI for Beginners (Microsoft): LLMs, prompts, RAG, embeddings and agents
Challenge labs
- Gandalf by Lakera: a prompt injection and jailbreak game
- Delphi AI Security Labs: browser-based AI security challenges
OWASP
- OWASP Top 10 for LLM Applications: the GenAI risk list
- OWASP AI-BOM: AI bill of materials
Disclaimer: for educational purposes only. Some labs are intentionally vulnerable. Use isolated environments and test accounts.
Your first month
| Week | Focus | Goal |
|---|---|---|
| 1 | First injections | Build the small chatbot, beat Gandalf, read OWASP LLM01 |
| 2 | Indirect injection | Hide instructions in documents and pages your bot reads |
| 3 | Agents | Give the bot an email tool and make a planted instruction send something out |
| 4 | Method | Run garak or promptfoo against it and write a one-page report with a canary-backed finding |
Checklist: am I ready to test real AI systems?
- I can draw the parts of an LLM app from memory
- I have exploited direct and indirect prompt injection in a lab
- I can spot the lethal trifecta in an agent design
- I prove findings with a canary and a success rate, not one lucky run
- I can recommend design-level defences, not just filters
- I only test systems I am authorised to test
The slides below cover the full roadmap. You can present them or download them as a PDF.
Next, read what AI red teaming is and how an engagement runs, or practise with our AI, LLM and MCP red teaming training.
References: OWASP Top 10 for LLM Applications · UK NCSC: Prompt injection is not SQL injection · Simon Willison: The lethal trifecta · KageX, "AI security roadmap: zero to red teamer" (2026) · Hacking Vidhya, "0 → Hero in AI Red-Teaming"

