Risky Bulletin – "How AI Turbocharges SOC Analysts"

Podcast: Risky Bulletin
Host: Tom Uren (Risky.biz)
Guest: Edward Wu (CEO & Founder, DropZone AI)
Date: October 19, 2025

Episode Overview

This special episode features Tom Uren interviewing Edward Wu of DropZone AI about their new, first-of-its-kind study on how AI impacts Security Operations Center (SOC) analysts. The conversation dives into empirical findings on accuracy, speed, and fatigue when SOC analysts use AI tools, explores perceptions of trust, and discusses the broader impact of AI on cybersecurity career paths.

Key Discussion Points & Insights

1. Purpose and Structure of the Study

• DropZone AI, in partnership with the Cloud Security Alliance, conducted a study involving 148 real-world SOC analysts of varied experience.
• The analysts were split into two groups:
  • Manual Group: Investigated two security alerts without AI assistance.
  • AI-Assisted Group: Investigated the same alerts using DropZone's AI product.
• Metrics compared: accuracy, speed, completeness, and fatigue resistance.
  (01:15–02:10)

“We essentially had them... split into two groups. One group investigated security alerts manually, the other group... with the help of our product. At the end, we compared... accuracy, speed, completeness and fatigue resistance.”
— Edward Wu, (01:15–02:10)

2. Quantitative Results: Speed, Accuracy, Fatigue

• With AI assistance:
  • 22–29% more accurate
  • 45–61% faster investigation times
  • Significantly less fatigue, measured by a smaller decrease in completeness on the second investigation
    (02:18–02:47)

“With AI support, analysts were 22 to 29% more accurate and finished their investigations 45 to 61% faster...”
— Edward Wu, (02:18–02:32)

• Results may be understated compared to the real-world, as test analysts were new to the tool, unlike DropZone’s existing customers who benefit more after onboarding and customization.
  (03:15–04:34)

“The measured differences were actually smaller than what we have seen across our early adopters... once [users] get to customize and tune the system... the actual impact... is much larger.”
— Edward Wu, (03:17–04:09)

3. Analyst Sentiment: Efficiency vs. Trust

• Analysts mostly found the AI "efficient and helpful", but only ~35% said they fully "trusted" it.
  (04:34–05:28)

“Almost all of them said it was efficient and helpful, but the number for trust was actually relatively low. I think it was something like 35%... I thought it was really interesting that you could have a technology that you think is helpful, useful, not actually that trustworthy.”
— Tom Uren, (04:41–05:22)

• Trust is "earned over time" with repeated, high-quality performance across many investigations, not just a few in a single session.
  (05:28–06:11)

“A lot of people say trust is earned, right? It’s earned over a period of consistent performance... once a security team has experienced consistent, high quality investigations ... they do, over time, start to trust the system more.”
— Edward Wu, (05:28–06:11)

4. Impact on SOC Roles and Career Development

• The aim is not to replace analysts but to automate repetitive toil and elevate humans to "general" or "special forces" roles.
  (06:44–06:59)

• Future without tier 1 SOC analysts:

  • Automation raises the bar for entry; new analysts may skip "tier 1" and start at more complex, tier 2 roles.
  • Concern exists about this higher entry barrier, but Edward Wu compares it to automation in other industries (like airline pilots) that have ultimately improved training and skills.
  • AI acts as a personal coach: Fast-tracks training, letting new staff "compare notes" with expert-level guidance, shrinking the learning curve from years to months.
    (06:59–10:58)

“I do anticipate a world where in the future there is no such role as a tier one SOC analyst... because of the increasing automation, the barrier of entry has been raised... But now... it actually increases and accelerates the learning process... it’s kind of like everybody having your own personal coach... [you] can quickly up level... maybe just within six months, instead of having to go through the tier one phase for a couple years.”
— Edward Wu, (07:08–10:53)

Notable Quotes & Memorable Moments

• On trust and usefulness:

  “Almost all of them said it was efficient and helpful, but the number for trust was actually relatively low.”
  — Tom Uren, (04:41–04:54)

• On AI augmenting— not replacing—humans:

  “What AI agents are going to do is they are going to handle the toil, they are going to handle the manual repetitive work. And by the virtue of doing that, they can up level the existing human engineers or human analysts to be more like generals or special forces.”
  — Edward Wu, (06:48–07:05)

• On accelerated learning:

  “It’s kind of like everybody having your own personal coach... a security analyst intern can now compare notes against one of the best human security analysts on the planet on every single alert.”
  — Edward Wu, (10:12–10:31)

Timestamps for Important Segments

• [00:38–02:10]: Study setup: methodology and structure
• [02:18–02:47]: Headline findings (accuracy, speed, fatigue)
• [03:15–04:34]: Results compared to real-world deployments
• [04:34–05:28]: Analyst perception: efficiency vs. trust
• [06:44–10:58]: Impact on security job roles, barriers, accelerated learning

Closing

The episode provides key first-hand metrics on how AI can measurably boost SOC analyst performance. Yet, questions around trust and workforce implications persist, with Wu championing AI as an accelerator for skill development rather than a replacement for human expertise.