Are Security ROI Figures Meaningless?

Recently, security expert Bruce Schneier expounded security ROI figures were meaningless! Is it true? Well, yes and no.

The brutal truth.

Well respected information security expert Bruce Schneier recently provided a stark opinion regarding the value of ROI's.

Follow this link to see the story:

In brief, Bruce stated security because numbers can be manipulated to justify anything.

tHe explained that the amount spent on a product can change significantly by simply playing with the equation.
t"If the chance of you being attacked is one in a million and I change it to one in two million... I have halved the amount of money you should spend.
t"I can make an ROI model say whatever I want. I could justify or not justify anything based on these very, very rare and very, very damaging events," he said.

Tell me it is not true!

I believe Bruce is both right and is delivering a message which is a little incomplete. His general message is accurate and shocking enough to garner the right level of attention. Most of the information security ROI's I have read were speculative, could not be validated, were impossible to reproduce, and had great latitude to provide results which benefit the desires of the author. Nowadays audiences are being provided ‘information' under the auspices of ‘fact', when in reality they are more of an opinion. Such valuation assessments are based on qualitative data versus quantitative metrics.

I blogged about the back in August 2007

Awareness must be raised. I applaud Bruce in helping to make this happen. His message, as brutal as it sounds, is bringing to light a shadowy area in our industry. I think the follow-up message for audiences is to scrutinize and apply common sense to any ROI they come across. Understand the methodology and if it makes sense in their context. Lifting the curtain can quickly reveal a puppet master pulling the strings to artificially show value.

Like Bruce, I too have a jaded perspective. I have seen some WILD ROI's. Much of what I have read from security vendors is pure folly. However, just because most are fiction, it does not mean all methodologies are without merit.

Intel published a which is applicable to some situations. This method, far from being a silver-bullet, is a good start and has proven its truthfulness.

For any method, the accuracy should be scrutinized. Can it be validated, repeated? Was the method exclusively developed solely for self serving purposes from someone trying to sell something shiny? Does it make sense? These are the questions I ask myself.

On the bright side, many bright sharp people are working very hard to make the industry better and develop more rigid processes to insure both accuracy and confidence.

In the end, there is much work to be done in the information security valuation space. In the meanwhile, savvy consumers should be aware of the challenges and dive deeper into prospective ROI's and determine if they are ‘meaningless'.

Published on Categories Archive
Matthew Rosenquist

About Matthew Rosenquist

Matthew Rosenquist is a Cybersecurity Strategist for Intel Corp and benefits from 20+ years in the field of security. He specializes in strategy, measuring value, and developing cost effective capabilities and organizations which deliver optimal levels of security. Matthew helped with the formation of the Intel Security Group, an industry leading organization bringing together security across hardware, firmware, software and services. An outspoken advocate of cybersecurity, he strives to advance the industry and his guidance can be heard at conferences, and found in whitepapers, articles, and blogs.