What a roomful of threat hunters taught us about detection engineering

By Nomad Security

·

We expected a vendor dinner. Steak, slides, a soft pitch for a platform renewal. That is the pattern, and after enough of them you stop expecting anything else. Intel 471 broke the pattern.

Their Unconference skipped the deck entirely. Instead of a sales pitch in exchange for a meal, the Intel 471 team gathered a large group of infosec and threat-hunting professionals from the local community into a single room and ran a live Capture the Flag exercise focused on real threat intelligence. No product demos. No feature roadmaps. Just practitioners working a problem together.

It was one of the most interactive and genuinely useful vendor-hosted events the Nomad Security team has attended. Here is what happened, what we learned, and why the industry needs more of it.

The CTF: hunting TeamPCP across the supply chain

The exercise centered on TeamPCP, the threat group behind Shai-Hulud and CanisterWorm, two malware families that have been causing serious damage across NPM, PyPI, and other dependency supply chains. Participants were tasked with info-gathering: tracing the group’s infrastructure, mapping indicators of compromise, and building a picture of how the campaigns moved through package registries and into downstream consumers.

The format was straightforward. Teams had access to intelligence sources and a set of structured questions that guided the hunt. The clock was running. The work was real; the threat actors, the IOCs, and the attack patterns were drawn from active campaigns, not lab scenarios cooked up for a training exercise.

What made the CTF valuable was not the competition. It was what came after.

Every team found something different

When the room reconvened to discuss results, something interesting happened. Every team had approached the same data set and arrived at different conclusions. Not wrong conclusions; different ones. Different pivot points, different attribution chains, different assessments of which indicators mattered most for detection.

One group had focused on registry-level signals (package naming patterns, publish cadence, maintainer account age). Another had followed network infrastructure and landed on a completely different cluster of C2 domains. A third had concentrated on the payload delivery mechanism and mapped the post-install execution chain in detail that the other groups had only sketched.

The discussion that followed was the most valuable part of the event. It became a working demonstration of a problem every detection engineering team faces: the difficulty of chaining attack styles and indicators of compromise into a proper system for detection. The CTF did not just test whether people could find the threat. It demonstrated, to a room full of experts, just how hard their jobs actually are.

Not just for themselves. Across the entire room. We are all in the same boat, and we all came up with different maps of the water.

The complete picture only exists in the room

Here is the part that matters for anyone running a threat research or detection engineering program.

No single team had the complete picture. But the room did. When the groups started sharing their findings, overlapping where one team’s blind spot was another team’s strength, a much more complete model of TeamPCP’s operations emerged. Indicators that one group had dismissed as noise turned out to be critical pivots for another group’s analysis. Attribution paths that felt tenuous in isolation became convincing when a second team had arrived at the same endpoint through a completely different route.

This is not a new observation. Threat intelligence sharing has been a talking point for years. But there is a difference between reading an ISAC report and sitting in a room where fifteen teams just independently hunted the same threat group and are now comparing notes in real time. The Unconference made the value of shared intelligence visceral in a way that no slide deck or policy document ever has for us.

We walked away with concrete improvements to our own threat research process, specific indicators we had missed, and a sharper understanding of how other practitioners in our community approach the same problems we face. That is not what vendor events usually produce.

Why the industry needs more of this

The model Intel 471 used deserves to be copied. Put practitioners in a room. Give them a real problem. Let them work it. Then let them compare notes. The vendor’s role is to facilitate, not to pitch. The value proposition is implicit: if you run an event this good, people remember who made it happen.

For threat hunting and detection engineering teams specifically, the format addresses a gap that conferences and training courses do not fill. Conferences give you talks. Training gives you labs. Neither gives you the experience of working the same live problem as a dozen other teams and then discovering, in the debrief, that your blind spots are systematic and that the person sitting three tables over found the thing you missed.

Nomad Security has always believed that the best security work happens in communities, not silos. Events like the Unconference reinforce that conviction. We plan to bring similar formats to our own community engagements, because the lesson of the day was clear: the threat is hard enough that no one team, no matter how good, sees all of it. The complete picture requires the room.

What we’d do this week

  1. Run a team-vs-team threat hunting exercise on a real, active campaign. Pick a threat group your org cares about (TeamPCP’s supply-chain work is a good candidate if you consume open-source dependencies). Give two groups the same starting intelligence and compare results. The gaps will be instructive.
  2. Audit your detection coverage for supply-chain threats specifically. Do you have detections for suspicious package naming patterns, anomalous maintainer behavior, or post-install execution from dependency managers? If the answer is “we rely on SCA tooling,” pressure-test whether that tooling would have caught Shai-Hulud or CanisterWorm.
  3. Join or build a local threat intel sharing circle. ISACs are one mechanism, but informal practitioner groups (local ISSA chapters, DEF CON groups, Slack communities) often share faster and with more operational detail. If your team is not plugged into one, start there.
  4. Attend (or host) a vendor event that is not a sales pitch. If a vendor invites you to something that looks like it might be a working session rather than a demo, go. If none of your vendors are doing this, ask them why not. The best vendor relationships produce shared intelligence, not just invoices.

Nomad Security

From the editors

Need help applying this to your environment?

Nomad Security helps engineering and security teams find and fix the issues attackers actually exploit. Penetration testing, vCISO advisory, secure code review, and threat research, sized to mid-market budgets.