Beyond the Technical Glitch: Understanding the Cost of Misconfigurations
System outages have led to several notable disruptions over the past decade.
A striking example occurred in 2021 when Facebook experienced a worldwide outage impacting around 2.9 billion users. Similarly, in 2017, Amazon Web Services suffered a significant service disruption affecting a vast number of websites and online platforms. Both examples were due to misconfiguration errors; however, these are not the only ones that could be cited - even for these two repeat offenders.
In fact, misconfigurations have been a chronic, recurring issue for well over 20+ years and continue to plague businesses with unpredictable disruptions and delayed initiatives as the IT systems we depend upon evolve rapidly and become more complex.
In addition, misconfigurations aren't just impacting uptime. They are seen as a critical issue in cybersecurity based on recent publications by the Cybersecurity and Infrastructure Security Agency (CISA). Misconfigurations are often exploited by malicious actors and have been identified as contributing significantly to the vulnerability of organizations - even those with mature cyber security postures. CISA goes as far as to call this a 'systemic weakness' in organizations – echoing the above-referenced blog.
The True Cost and Impact of Misconfiguration
Misconfigurations are more than just part of the job that needs to be handled. Misconfigurations can cause a variety of problems that end up as an outage or a security breach. They can also go unnoticed, leaving the company vulnerable, until they start to cause significant hits to system stability, team member productivity, client experience, etc. By then, trust, compliance - and the organization's bottom line have already been compromised. In this fashion, it's not just the cost of handling the outage or security incident that should be considered, but also the indirect costs, such as dealing with public relations rather than focusing on core business initiatives and profitably that add to the impact and cost.
In this way, the seen and unseen costs of misconfiguration can be difficult to quantify. This calculation depends on a variety of factors, such as the severity of the misconfiguration, the size and complexity of the organization, and initiatives important to your business within your industry.
As an example, a 2022 report states that a network misconfiguration, on average, costs an organization 9% of its annual revenue. That can easily be hundreds of millions of dollars. The cruel truth - this figure is an 'average' and a year old and doesn't take into account stalled initiatives that represent the alternative ways cybersecurity or error budgets could be spent. That means that this number is likely much higher, especially for larger and more complex organizations.
Some additional studies to consider as you think about the true cost of a misconfiguration in your environment:
- 2022 Microsoft released its 2nd Edition of Cyber Signals, stating that over 80% of ransomware attacks are traceable back to common misconfigurations found in software and devices. That's 80%, and most misconfigurations are mistakes by humans – not all – but a majority. So, what percentage of these could be caught or prevented?
- According to research by Comparitech, in 2023, ransomware attacks have cost financial organizations $32.3 billion in downtime over the past three years. This breaks down to approximately $8,662 per minute. Other reports, not related just to ransomware, state up to $9000 per minute for downtime, which is approximately half a million dollars per hour. This can add up fast and puts a lot of pressure on the CIO and his/her team to prevent and resolve issues quickly.
- A 2023 study by Pingdom, additionally found that the average cost of downtime or unplanned IT outage for high-risk industries (such as financial services, healthcare, government, retail, etc.) could reach as high as $5,000,000 per hour. Financial Services had the largest cost per hour at $6.48 M. This same report also notes that hourly downtime costs have risen 32% over the past 7 years due to the reliance on the digital marketplace to meet customer expectations. Talk about pressure.
- A 2023 study by Ponemon Institute found that the average cost of a data breach reached an all-time high of $4.45 M this past year. The study also states that the healthcare industry is still the most impacted by cyber threats. The study also makes many recommendations, including embracing AI and automation to increase the speed and accuracy of detection as well as root cause analysis.
- And lastly, Gartner provided a report earlier this year that anticipates that through 2027, over 99% of cloud breaches will be traced back to preventable customer errors, misconfigurations, or account takeover. That is 99% - and preventable.
To showcase the potential magnitude and how quickly these things can add up:
- In just 12 hours, in 2015, Apple lost approximately $25 million in an outage caused by a DNS configuration error. That's a little over 2 million per hour.
- In just 14 hours, in 2019, Facebook lost nearly $90 million due to a server configuration change. That is over $6.4 million per hour.
- In just 6 hours, in 2021, Facebook again experienced a giant outage that knocked its 3 popular social media apps offline for billions of users, likely costing the company over $60 million – all due to a configuration change on backbone routers. And this one is believed to have cost Facebook $10 million per hour.
The impact and cost of a misconfiguration is real. It is also important to understand that misconfigurations are not projected to do anything except continue to grow in occurrence and cost due to:
- The increasing complexity of IT systems and networks.
- The growing number of cyber threats.
- The increasing reliance of organizations on disparate IT systems.
"What's clear from this research is that misconfiguration risks are impacting the bottom line. Senior network professionals are prioritizing compliance and feeling confident about network security but delivering on it at scale and continuously is a major challenge," said Phil Lewis, CEO of Titania.
How Do You Conquer Misconfiguration
Gaining intelligence about configuration risk should be considered a key requirement for IT – from ITOps to CloudOps, to Security, to DevOps. By understanding the true financial and organizational impact, you can strategically plan, budget, and tackle this challenge head-on within your organization.
Traditional solutions were not built to detect configuration changes across your hybrid estate, let alone assess their risk to your enterprise. These solutions typically only monitor or observe your environment for "symptoms” that an issue has occurred (in its proprietary environment) and then present that evidence to you to evaluate via logs, traces, or metrics. Despite these traditional approaches, unauthorized changes and misconfigurations have continued to haunt the best and most dedicated IT departments around. Unfortunately, the lack of granular change information, only provided after the fact, still leaves IT teams scrambling to investigate the root cause and restore services.
This very reason is why Evolven was created. Evolven was purpose-built to do one thing – know everything about the configurations across your hybrid environment. Evolven's advanced technology performs a deep data collection of all configurations and then watches for any change in the state of your environment. See Figure 1.
Figure 1. Evolven’s Deep Data Collection includes all environments and configuration types with granular details.
With clear visibility into your entire infrastructure, its real-time configuration state, and prioritized, AI-based risk analysis of any change, your team is armed with the right information across all departments to take action, prevent incidents, and recover from incidents that do occur quickly if caused by:
- Unauthorized changes
- Configuration drift
- Certificate expiration
- Non-compliance to policies
- File integrity violations
- Blind spots
- And more
To find out more about how we help some of the largest banks minimize these “technical glitches” to increase their security posture, compliance, and infrastructure stability, contact us.
Increase your configuration risk intelligence and squash downtime costs caused by misconfiguration.