When EDR catches the operator on the first lateral move, the CISO’s next question is usually quiet and pointed: what did we just pay $40,000 for? The operator never left the initial access box. The report came back with LLMNR poisoning and a Nessus export dressed in a Word template. The board deck needs a slide on security posture, and there is nothing in that report that helps write it.
Pentesting works fine. The scoping, the assumptions, and the outcome definition are what failed here. Serious offensive work in an EDR-heavy environment is achievable. It just requires a different conversation before the SOW is signed.
Why CISOs cancel pentests (or stop buying them)
The pattern repeats. An organization buys a red team engagement. The operator runs a phishing campaign, lands a shell, hits Defender, pivots to a noisier technique, hits Defender again, and delivers a report that lists CVEs and recommends patching. The CISO looks at the report and cannot answer the question their CFO will ask: “Did we learn anything we didn’t already know?”
The failure modes are consistent:
- Constrained tradecraft with no constraint acknowledgment. The SOW said “no destructive testing” and “no social engineering of executives,” which is reasonable. But nobody discussed what the operator would do when those constraints removed the two most realistic attack paths. The operator improvised. The findings don’t reflect the actual threat model.
- No line-of-sight to risk. A finding that says “Kerberoastable service account with weak password” is accurate. A finding that says “Kerberoastable service account with weak password, which we used to access the billing database containing 2.4 million customer records in 18 minutes from initial access” is evidence for a decision.
- The report is a scanner dump with a narrative wrapper. Automated tooling finds real issues. But if the report reads like Nessus output with a paragraph of context per finding, the buyer paid $40k for something a $15k tool subscription could have approximated.
- No detection feedback. The SOC either didn’t know the test was running (full black-box, which has its place) or knew and watched passively. Either way, nobody learned whether the detections fired, what they missed, or why. The blue team got nothing actionable.
What the buyer actually wanted was validation: does our security program stop a real attacker, and if not, where and why? What they got was a list of vulnerabilities and a clean bill of health on the things the operator didn’t try.
What $40k should actually buy
Forty thousand dollars is a real budget. It is not enough for a six-month adversary simulation against a Fortune 100. It is enough for a tightly scoped, objective-driven engagement that produces decision-quality evidence. The difference is in how the engagement is designed before anyone opens a terminal.
A well-scoped engagement at this price point should include:
- A defined objective the engagement is built around. “Demonstrate whether an attacker with initial access to a developer laptop can reach production customer data” is an objective. “Perform internal network penetration testing” is a technique category. A good objective survives EDR; a technique checklist gets blocked at the first control and stalls.
- Explicit assumption documentation. If you’re running assume-breach (operator starts with a foothold), say so and document what the foothold represents. A phishing-to-initial-access simulation and an assume-breach lateral movement exercise test different things. Both are valid. Conflating them produces a report that satisfies neither.
- Purple-team touchpoints. At minimum, a mid-engagement sync where the operator shares what fired and what didn’t. Ideally, a structured detection validation pass at the end where the SOC and the operator walk through the attack timeline against the SIEM. This is the part that makes the blue team better, not just the report.
- Clear stop rules and constraint documentation. Every constraint (no exec phishing, no production data exfil, no weekend testing) should be listed alongside its implication. If no exec phishing removes the most realistic initial access path, the report should say so explicitly, and the engagement should be scoped around the next most realistic path.
- An executive readout built around “so what.” The technical report goes to the security team. The executive readout answers three questions: What did we find? What would a real attacker have done with it? What are we doing about it, in what order, and who owns it?
Engagement design: picking the right model
Not every engagement needs to be a full adversary emulation. The right model depends on the organization’s maturity, the threat model, and what question needs answering. Here is a rough decision table:
| Model | Best for | What it answers | Watch out for |
|---|---|---|---|
| Full emulated APT (black-box, multi-phase) | Mature orgs, strong detection baseline, executive buy-in for a realistic scenario | Can a motivated external attacker reach the crown jewel? | Expensive; operator may burn time on blocked paths; SOC gets no real-time feedback |
| Assume-breach lateral movement | Post-EDR rollout validation; orgs confident in perimeter but unsure about internal segmentation | Given a foothold, what can an attacker reach and how fast? | Skips phishing/initial access realism; findings may feel hypothetical to leadership |
| Tabletop + technical validation hybrid | Compliance-driven orgs, limited budget, IR team that needs exercise reps | Does our response process work when the scenario is realistic? | Technical findings are shallower; not a substitute for a real red team |
| Continuous purple team | Detection engineering teams that want ongoing TTP coverage, not a point-in-time report | Are our detections keeping pace with attacker technique evolution? | Requires internal SOC bandwidth; not a red team replacement |
The model you pick should match the question you need answered, not the model your last vendor defaulted to.
What good looks like on the report
The report is the artifact that outlives the engagement. It goes to the CISO, the board, the auditor, and the engineering team that has to fix the findings. It needs to work for all of them, which means it needs structure, not length.
A report from a serious engagement includes:
- An attack path narrative that reads as a story, not a finding list. Something like: operator landed here, pivoted here, reached this asset, and here is what they could have done with it. The narrative makes the risk legible to someone who doesn’t know what BloodHound is (a graph-based Active Directory attack path tool, worth defining here because it’s load-bearing).
- Control failures ranked by blast radius. A CVSS 9.8 on an internet-facing system with no sensitive data downstream is less urgent than a CVSS 6.5 on an internal service that touches your payment processor. Rank by what an attacker can actually do with it.
- Detection hits and misses, with log sources. For every technique the operator ran, the report should note whether it fired an alert, what the alert said, and what log source caught it or missed it. This is the section the detection engineering team will actually use.
- Remediation owners attached to every recommendation. “Disable LLMNR” is a recommendation. “Disable LLMNR via Group Policy; owner: IT Operations; estimated effort: 2 hours; retest criteria: operator re-runs Responder with no captures after change” is a ticket.
Anti-patterns worth calling out by name: a report that lists LLMNR poisoning, SMB relay, and AS-REP roasting as its top three findings, with no attack path connecting them to a business impact, is a checklist wearing an assessment cover page. A report that is 80% scanner output with a paragraph of context per finding is a Nessus export with a cover page. Both are common. Neither justifies the budget.
A sanitized engagement: tight constraints, real value
One composite engagement illustrates how constraints don’t have to mean shallow findings.
The client was a mid-size SaaS company. Their EDR was mature, their phishing controls were solid, and the SOW excluded social engineering of employees entirely. By the standard playbook, that’s a constrained engagement with limited attack surface. The operator had two weeks and an assume-breach starting point: a compromised developer credential with no elevated privileges.
The objective was specific: determine whether an attacker with that foothold could reach production customer data.
What the operator found in the first three days was not a vulnerability in the traditional sense. It was a workflow. Developers routinely spun up short-lived cloud environments for testing, and those environments were created with an automation script that pulled credentials from a shared Slack channel. The channel was private, but accessible to anyone with a developer role. The credentials rotated weekly, but the rotation script had a bug: it updated the channel message but not the underlying secret in the CI/CD pipeline. So the pipeline was running on credentials that were two months old and had never been rotated out of the production IAM policy.
The operator used those credentials to access a production S3 bucket containing customer exports. No exploit. No EDR evasion. A credential in a Slack channel and a rotation script with a bug.
The finding read: “Developer workflow leaks production-scoped credentials into a shared channel; rotation process has a documented failure mode; blast radius is full read access to customer export data.” The remediation owner was the platform engineering team. The fix took four hours. The retest confirmed it two weeks later.
The constraints focused the work. The operator spent the two weeks finding the path a real attacker, with realistic access, would actually take.
What we’d do this week
If you’re evaluating a red team engagement or prepping for one, here are the questions and actions that matter before the SOW is signed:
- Define the objective in one sentence before you write the RFP. “Can an attacker with a phishing foothold reach our payment processing environment?” is an objective. “Test our internal network” is a task category. If you can’t write the one-sentence objective, the engagement isn’t ready to scope.
- Ask the vendor: what happens when your primary technique is blocked? A serious operator has an answer. The answer should describe alternative paths, not a request to loosen the constraints. If the vendor says “we’ll need broader scope” every time you describe a control, that’s a signal.
- Require a detection validation section in the deliverable. Not a recommendation. A requirement. The SOW should specify that the final report includes a technique-by-technique breakdown of what the SOC detected, what it missed, and what log source was or wasn’t present. If the vendor says this isn’t standard, it should be.
- Schedule the executive readout before the engagement starts. The readout format (three slides: what we found, what it means, what we’re doing about it) should be agreed upfront. This forces the operator to think about business impact framing from day one, not as an afterthought in the report.
- Run an internal prep session with your SOC before kickoff. Not to brief them on the test (that may break the realism you need), but to make sure your logging is actually working. If your SIEM isn’t ingesting endpoint telemetry from the systems in scope, the detection validation section will be empty. That’s a problem to find before the engagement, not during.
