Measuring security must be done in a manner which is a benefit to the organization. Yes, it is difficult to obtain data, determine key factors, calculate value estimations, analyze results, conduct sanity checks, and translate the information to the intended audience. Yes, even the most expeditious professional can be consumed for weeks, months, and even years due to the complexities, lack of data, and sheer desire to make it a little more accurate. But this exercise has a purpose and a window of applicability. Taking six months to conduct a ROI for a project, which management wants integrated in 4 months, is a waste. Every request is different and the resulting analysis should flex to meet its intended purpose.
I know what you are going to say: "You can have it fast, cheap, or accurate, just pick two". This is very true and must be taken into account when tackling the ugly job of measuring security. In the example of the 4 month project, setting an expectation of a 1 week ROI to give ball-park accuracy may be entirely acceptable to management. They get what they need to make a go/no-go decision and the analyst does not waste effort on over-kill.
Beware the frustration inherent in trying to achieve accuracy to the second decimal place (or any other ridiculous granular measure). It is a mirage you will never grasp. Methods in measuring information security value are still in their infancy. No silver bullet exists which delivers precise results and applies to all situations. Know the situational limitations and align the analysis with the business decision trying to be made.
Understanding what is needed is the first step of any security measurement endeavor. Having discussions early on regarding the scale of accuracy, how the output will be formatted (dollars, MTTR, compliance to regulations, etc.), and a timeline for completion will set clear expectations and avoid the "bring me a rock" situations.
My advice is to apply the Security Judo mantra:
|"Exert the minimum amount of enercy necessary to achieve the security business objective"|
Principles of good planning and project management apply to measuring security. Don't go overboard and calculate the exact strength of a hurricane if management only wants to know if they should take an afternoon pleasure cruise.