Module 5: Agents for Security Review#

Stronger models as security reviewers: whole-codebase audit, checklist-driven vs. open-ended prompting, triaging hallucinated and severity-inflated findings, and how agentic review fits an offensive workflow. In the lab, findings aren’t findings until they’re proven: everything you report has to be exploited against a live instance to count.

Questions this module answers#

• How do I point a strong model at a whole codebase and get a useful audit?
• Checklist-driven or open-ended prompting — which finds more, and when?
• How do I triage agent findings: hallucinations, duplicates, severity inflation?
• What failure patterns show up specifically in AI-written code?
• How does agentic review fit into an offensive workflow?

Slides#

Lab 5.1: Audit the Vulnbox#

A focused block. Flag submission stays open all day — the CTFd board doesn’t close until the capstone debrief — but the report is due before Module 6 starts.

Goal#

Use the large model to audit an instructor-built, deliberately vulnerable ShareBox — the vulnbox — and prove what you find. Its vulnerabilities range from easy to well-hidden. A finding only counts when you exploit it against your hosted instance: each vuln yields a flag for the class CTF, and harder vulns are worth more points. Reading the source and declaring “line 84 looks bad” scores nothing. (Exploit-or-it-doesn’t-count is this class’s CTF rule, and a useful discipline — in real-world reporting you’d rightly also publish credible findings you couldn’t prove, labeled with your confidence.)

Provided#

• The full vulnbox source, on your instance: labs/05-audit/vulnbox/
• A hosted, running instance of the vulnbox, dedicated to you — its address is on your student portal page, or from your VM: print it with aws s3 cp s3://$CLASS_BUCKET/portal/$CLASS_STUDENT_ID.md - (both variables are already exported in your shell)
• The instructor’s vuln-class checklist: checklist.md in the lab directory (labs/05-audit/)
• A findings report template: report-template.md in the same directory
• Your vuln-report skill from Lab 2.3 — your write-ups must go through it
• The class CTF (CTFd; link in your portal file)

Framing#

Open-ended: find as many as you can. There is no fixed finding count and no complete answer key handed out. Expect the finding to be the fast part — in the dry run, one broad prompt surfaced every seeded vuln in about 40 seconds — and the bulk of the lab to go into triage, exploitation, and writeups. The escalation tiers below are a hint ladder for gaps, not a required march. Use the large model throughout.

Steps#

1. Recon by hand first: skim the vulnbox source yourself. Note the stack, the auth approach, and anything that smells. Don’t use the agent yet; you need your own baseline to judge its findings against.

2. Tier 1 — the naive pass: point the large model in pi at the source with a broad “find security vulnerabilities” prompt. Save everything it claims. This pass is likely to surface most or all of what’s there; the real work — and most of your time — is steps 3 and 7, proving and writing up.

3. Prove-as-you-go: for each candidate finding, exploit it against your hosted instance. A successful exploit yields a flag; submit it to CTFd immediately. Anything you can’t trigger against the live instance is either a hallucination or not yet understood — both mean it doesn’t count yet.

   Hint 1: a triage test that kills hallucinations (step 3)

   For each candidate finding, make the agent produce a runnable curl command that should demonstrate the bug, then trace the code path yourself: does the parameter it names actually flow to the sink it claims? Two classic hallucination smells: the “vulnerable line” doesn’t exist in the file (the model paraphrased code it imagined), and the finding is real-in-general but blocked by a check two lines up. Here you have the ultimate arbiter: run the request against your hosted instance. No flag, no finding.

4. Tier 2 — when the naive pass runs dry: run semgrep over the source with the class ruleset — semgrep --config ~/materials/05-audit/semgrep-rules.yml app.py (the VM has no internet, so --config auto won’t work) — and triage its output with the agent: paste the results in and have it separate real leads from noise, then chase the leads to flags.

5. Tier 3 — the checklist pass: in a fresh session, walk the instructor’s vuln-class checklist explicitly, class by class.

   Hint 2: prompting the checklist audit (step 5)

   Don’t paste the whole checklist in one message; you’ll get one shallow paragraph per class. Walk it: one vuln class per prompt (“Check every file-download and file-serving path for path traversal. For each finding, quote the vulnerable line and construct the exact request that triggers it.”). Demanding the quoted line and a concrete trigger request in the same breath is what keeps the model honest.

6. Tier 4 — the file-by-file walk: convinced something is still hiding? Go exhaustive: one file per prompt, in full, asking what an attacker gets from this file alone and how it composes with what you’ve already found. Slow and expensive — a last resort, not a routine; time spent here is time not spent proving and writing up.

7. Now turn the same audit on your own app. Point the large model at your frozen ShareBox from Lab 3.1 (~/work/sharebox) with your best-performing prompt from the tiers above. You wrote three predictions about where your agent would cut corners; this is the first real check. Note every finding — you’ll run your own Foundry-lite system against this same code in Lab 6.1, so today’s manual result is the ground truth you’ll grade the machine against.

8. Write up your best findings with your vuln-report skill, in the report template: minimum 3, each mapped to a checklist class, each with the exact request that triggered it and the flag it produced.

9. Grade yourself: pull out the three threat-model predictions you wrote down during the Lab 3.1 build and grade them against what you actually found. Bring hits and misses to the debrief — a miss says as much about where AI-written code fails as a hit does.

10. Submit the report before Module 6 starts by copying it to s3://$CLASS_BUCKET/reports/$CLASS_STUDENT_ID.md (both values are already exported in your shell). No edits after submission.

Done when#

At least three flags on the CTFd board, your own ShareBox audited (findings noted for Lab 6.1), and your report uploaded to the class bucket before Module 6 starts. (Keep hunting flags through the day if you like — the board stays open until the capstone debrief.)

Scoring#

Flags are the score: each vuln has one, harder vulns are worth more points, and only exploitation gets you one. This makes “did the agent hallucinate?” empirically answerable per finding, which is the lesson.