Security metrics are a highly discussed topic within IT these days but mainly due to the need of understanding how well security controls are working to protect against threats to Confidentiality, Integrity, and Availability or the (CIA) Triad. But one of the most important aspects of such effort to collect data and present it properly is to understand the goal or purpose of the effort. If data collection is just for the sake of collection, there can be a great deal of wasted time into something that is not beneficial.
Good metrics is the difference between information and data. Good information is more important than a large amount of data that is not meaningful. Too often, there is a bias towards an expected result that comes out of security metrics to provide information on demonstrating the need or justification for a certain control. This can actually be detrimental to a security program. Security related metrics can be used not only for the purpose of providing good information about security events, but also to help make better risk-management decisions. Security Metrics benefit the organization if they are well understood, used, and provide value and insight. Benefits can describe a business value for what is being spent on an IT Security controls or identifying a security process that can use some improvement.
Taking a simplistic approach to defining security metrics, one may use a simple 3 step process known as Goal-Question-Metric or GQM. The GQM concept has been taken from software engineering metrics collection that can also be used for security metrics to provide for a direct link between metrics and goals. The goal setting effort also provides a clear understanding from all stakeholders when a requirements of data collection for metrics crosses business group boundaries.
Goal – A goal or objective is specific and should relate back to some system, process, or characteristic of your security program. It should also relate to something measurable and verifiable. A goal should be defined before the start of any effort to collect metrics so that all stakeholders can provide input and agree that the purpose is clearly understood.
- Outcome (decrease, understand)
- Element 1 - Malware detections (Anti-Virus software)
Question – A goal can be translated into a series of questions that allow for the information to develop into attributes and targets. This enables the components of the goal to be achieved or evaluated for success. There could be multiple questions with multiple metrics for a specific goal.
- What is the current percent of systems with malware detections?
Metric – After questions have been developed to define the goal operationally, the goal can begin to be characterized at a data level, and metrics can be assigned that will provide answers.
- % of systems with malware detections found during weekly Anti Virus scans over one month period.
This concept can also help for an annual review of security metrics and indicators to ensure they all are well understood, used and provide insight and value to the organization.
After Defining the Goal, Question, and Metric, it will be the next steps in the process that should include the data collection, analysis and reporting strategy. Those concepts will be my next blog on this topic.