We recently co-hosted a virtual roundtable event, with one of our vendor partners, on a topic that’s been particularly prevalent in security this year.

The event, provocatively titled, ‘Detection Sucks’, proposed for discussion the multiple problems modern security teams face in recognizing and responding to threats within their ecosystem.

“I have lived through the days of collecting all the logs but monitorization of logs has changed and become expensive. Even if you have the logs, you may not find the needle in the haystack for the important log if you need it. So, focus your efforts on what you should log.” CISO, Financial Services

Millions of dollars have flooded out of security programs in 2021 on a new generation of tools designed to recognize a seemingly exponentially growing library of malicious code. One of the biggest takeaways from the conversation was a growing sense of frustration at these shiny, new, bells and whistles. Not because they don’t work. They generally work very well. But because their end purpose doesn’t address the core issue most security programs face in battening down the hatches.

“We cannot continue to monitor as we have been. The number of incidents outnumbers the global population. There are 20x the number of devices as there are people. The reality is there are incidents every day and although we catch the vast majority, CISOs cannot be expected to prevent them all. We are focused on the wrong thing and the narrative around detection needs reframing. You wouldn’t expect the Fire Department to prevent every fire, they are there to deal it – responding quickly and effectively.” CISO, Global Technology Company

The average SOC receives ten thousand plus alerts per day, analysts report a 50% plus false positive rate, security analysts spend 25% of time chasing the wrong alerts, and 83% of companies don’t get around to triaging 50% of alerts.

Attention is a commodity and a zero-sum game, and the focus on detection – sadly – often comes at the expense of the end-to-end incident response process.

“I don’t think investment in a new shiny tool is always the answer. We should be utilizing more of our people’s ingenuity and using the tools we do have better.” CTO, MSSP 

There comes a point reading this blog where you must wonder what perspective a recruiter can possibly have on the theme of detection. When it comes to developing solutions, my ability to recognize keywords and sketch out org charts runs out of track fairly quickly.

What I do see, however, is the relative insignificance of the budget allocated for talent versus what companies will shell out for deals with software vendors.

How many of you reading this blog have gone back and forth with your human resources department begging for more than $160k + 10% bonus for a Detection & Response Engineer with experience in cloud?

“Breaches are often articulated as a “failure to patch”, or at best a failure to “manage applications”. The reality is security failures are far broader. Effective control comes from the management of the end-to-end incident response process, not the logging” CISO, Financial Services

There’s a reason all roads in IR lead to Threat Hunting. The ability of detection systems to identify real, business-impacting threats, zero-day or not, has not yet completely superseded the human element, and yet that’s where the money goes.

There is a space for detection. There is a big space for detection. But looking for what can hurt you is just one part of a machine that needs oil (and love) from end-to-end to recognize, contain and eliminate genuine threats to your business.

Security is getting better (whether the threats are getting better at a greater speed is another topic, entirely), but every industry finds itself in rabbits holes occasionally, and detection, at least the kind that swallows up your resources like a supermassive black hole, is one of them.

We’ll say it again, (we’ll say it again, too), invest in tools, but invest in talent, more.

If your system pings, but nobody is around to hear it, did it ever ping at all?