Strong security depends on more than technology; it also depends on focused security analysts (SA). Focus weakens as stress increases, and security analysts today are increasingly stressed as their jobs near the level of overwhelming. The stress is largely driven by fear of missing incidents (FOMI). Â
Organizations must recognize the possible outcomes because of the loss of productivity and make the necessary changes: changes that prevent high security analysts turnover and weakened incident response.
Importance of Focused Security AnalystsÂ
Security analysts review a continuous flow of alerts from across the network: both internal and in the cloud. These alerts include both false and true positives. They sort through the alerts and make decisions about how to respond. Â
Quick, early detection of an incident is an essential part of a robust incident response process. If analysts begin to miss accurate indicators of compromise (IoC), an attacker can effectively bypass security controls, compromise one or more systems, and remain active for months. Â
Dan Swinhoe writesOpens a new window that the cost of an attack not quickly detected and dealt with averages $3.86 million (based on an IBM-funded Ponemon Institute study). The cost decreases by $1 million if detected and managed earlier.
The Stress of Fear of Missing Incidents
In my experience as a director of IT security, FOMI increases as the number of alerts increases. Alerts do not always come during the business day. Instead, they occur 24/7 and require continuous attention. Stress and frustration increase as the number of false positives grows. Â
In this articleOpens a new window , the author writes that about 45% of alerts are false (based on a FireEye IDC survey). This means that analysts must try to identify what IOC is meaningful. As alert fatigue and FOMI increase, many alerts are sometimes simply ignored as they just assume they are false. Â
Learn More: Why Automation Should Be the New Modus Operandi for Battling Alert Fatigue in SOCÂ
Negative FOMI Outcomes
As shown in Figure 1, the first step in incident response is detection (post preparation) is detection and analysis. Because almost half of the alerts are often false positives, detection becomes difficult. Â
Figure 1 (from NIST SP 800-61Opens a new window )
For each alert, security analysts must figure out what is causing the IoC and where affected devices are. If it takes considerable time to decide the validity of an alert, IT management must decide whether to implement incident response processes. Failure to respond fast enough or respond when an alert turns out to be false can cause problems. When is containment necessary, and when will it be considered overreaction by management?
Some years ago, I had dealt with three serious incidents involving a large company for which I served as director of information security. We had to shut down the entire network for a national health care company for containment during each incident. In two cases, the shutdown lasted up to one week for some locations as eradication and recovery took place.
In all three cases, security was blamed for the incidents: even though security was not responsible for system patching that would have defended against the attacks. The stress associated with managing the incidents and dealing with post-incident challenges was intense. Â
Because of the stress associated with incidents occurring, both from recovery and management displeasure, we became quite sensitive to any alerts. This became a big problem for my analysts as we realized that we needed to look at alerts 24/7 because we did not have a 24/7 security operations center (SOC). On-call security analysts had to respond to alerts at all hours, causing a lack of sleep and reduced effectiveness.
At one point, we received an alert that looked like we were experiencing an attack like those that caused long-term downtime. In post-incident meetings for previous incidents, we learned that quick implementation of our revised response processed significantly reduced downtime. Â
Although we received many false positives, we needed to assess the alert’s validity even after we had tuned our detection controls. However, we also feared what would happen if we did not react quickly. I decided to implement containment until we thoroughly analyzed the IoC. Containment resulted in a loss of access for many users across the enterprise.
When we finally determined the alert was false, we restored functionality. We had only been down for 20 minutes, but my team and I were criticized for acting on a false alert.This did not help our overall stress and FOMI going forward.
Finally, FOMI stress can also cause high analyst turnover. According to this articleOpens a new window , a Critical Start survey revealed that alert overload could cause up to over 50% turnover. With the lack of experienced security professionals in the workforce, replacing security analysts is difficult. Further, retraining new analysts and acclimating them to the tools used and protected environments results in lower detection and response effectiveness.
Learn More: 5 Best Practices for Running a Security Operations Center (SOC)Â
Managing FOMI
The first step in dealing with FOMI is for management to understand that it is real and that it influences how well the organization manages information security risk. Once this happens, several recommendations can help.
First, security teams must ensure all security appliances are finely tuned. In many cases, internal teams handle tuning. However, bringing in third-party experts that focus only on adjusting controls is usually a better approach. There is always a balance between false and positive alerts but getting alerts to manageable levels is necessary.
Second, management should consider moving initial alert triage to a managed security service provider (MSSP). If the organization is not committed to creating a full-time, 24/7 security operations center, then FOMI is lowered by removing nighttime calls to on-call analysts. Further, the SOC analysts are only responsible for alerts; they are not responsible for other crucial internal security tasks.
One important consideration when using an MSSP is complete transparency into how they man the SOC. MSSPs are also subject to FOMI. Organizations must ensure they are not simply transferring reduced efficiency to another entity.
The third recommendation applies to both internal teams and contracted MSSPs. Consider using AI quickly to assess the validity of alerts and to find threats on the network. Eddie Segal writesOpens a new window that AI is not a replacement for traditional alert management. It works best when integrated with conventional approaches.
The fourth recommendation falls on the shoulders of the analysts and their managers. They must always inform management of the residual risk associated with how security is managed. They should ensure managers actually understand how their decisions affect the probability of bad things happening. If management refuses to accept any residual risk, it is time for affected analysts to find another employer.
Finally, management should carefully review security analysis spending. What is the risk of paring teams down to minimums? Is it worth the savings?
Learn More: Do You Need a Security Operations Center?Â
Parting ThoughtsÂ
FOMI is a real challenge to achieving optimum incident response. Associated stress could cause ignoring positive alerts, extending attack consequences. It can also result in high turnover.
Solutions to FOMI begin with management realizing there is a problem. Management understanding is followed by ensuring proper staffing and security appliance configuration. Business and IT management, especially at the C-level, must understand that there is always a residual risk. When an incident happens, it is not always someone’s fault.
Do you think it is important to mitigate alert fatigue in the security operations center? Comment below or let us know on LinkedInOpens a new window , TwitterOpens a new window , or FacebookOpens a new window . We would love to hear from you!