The relevance of training as a soft mitigation measure: why the existing Capability Maturity Models are not adequately addressing the issue.
Introduction
Training of the persons, or better cyber-training, in the organisations is a soft mitigation countermeasure, useful to reduce the cyber risk. Cyber-training has some unique characteristics versus training that needs a broader discussion. It is one of the most effective solutions today because it focuses on the root cause of most security-related incidents: humans. Today, it is a well-known fact that almost 95% of attacks involve humans in the early stages[1]. At the same time, it is also a well-known problem that training people with an efficient process and with lasting results (which means having a long retention time and long-lasting behavioural changes) is a challenge[2]. How does the HERMENEUT cyber-risk assessment methodology integrate with the desired maturity level of the cyber training strategies and the expected mitigation performances? To answer this question, we need an analysis of the current state of the art in the existing Capability Maturity Models (CMMs), especially those that include training and especially cyber-training.
During the discussions of the HERMENEUT project, the importance of linking the cyber risk estimations to a proper CMM methodology emerged clearly. However, the existing CMMs have some issues when it comes to training, and a few of them adequately address the issue. Unfortunately, even less (actually almost zero) addresses cyber-security training, which is a particular area of training.
What are CMMs and their relation to the HERMENEUT results?
As cybercrime creates continuously and rapidly evolving risks, the tools and techniques to defend and recover must evolve and become mature just as rapidly. Today’s cybersecurity training market still misses a scalable, measurable, but most of all, open and accepted maturity model. This methodology helps organisations and their supply chains reach the correct maturity level, which addresses the specific risk maps of organisations at a reasonable cost. Existing CMMs propose a “one-size-fits-all” training approach that does not fit all the possible specificities of the organisations.
A good yet brief explanation of Maturity Models is[3]: “Maturity models establish a systematic basis of measurement for describing the “as is” state of a process. A process’s maturity can then be compared to management’s expectations or contrasted with the maturity of other similar processes for benchmarking purposes. Insights also can be derived from the model for determining improvement options that help a process to satisfy its intended objectives over time.” CMM and maturity, in general, is a hot topic in today’s industry and, recently, also in cybersecurity processes and eLearning; even if no unified, formalised, widely-accepted standard exists for training on cybersecurity, there are several separate propositions[4]. However, there are still no fully defined CMM propositions to cover the training in cybersecurity. CMMs are composed of different elements: (1) levels, (2) components, (3) expectations and (4) supporting tools. A CMM describes process components that lead to better outputs and better outcomes when applied throughout an organisation. A low level of maturity implies a lower probability of success in consistently meeting a specified objective, while a higher level of maturity implies a higher probability of success. The correct maturity level is that level that addresses the specific risk map of the organisation at a cost aligned to the organisation’s expectations and investment priorities.
The HERMENEUT output directly feeds the CMM model for training because it defines the company’s exposure to cyber risk. As a project and cyber risk evaluation mechanism, HERMENEUT gives the CMM the right metrics and the correct indication of costs and KPIs to define the desired maturity level accurately.
Limitations of the existing CMMs
Existing CMMs which include training and their problems: Several training CMMs exist, but their focus is not on cybersecurity training, for example:
- Too general approaches to training: Bloom’s Taxonomy[5], Phillips ROI Process Model[6], Six Sigma[7] plus DMAIC[8];
- Addressing training in the Industry context, but too generic or for other types of skills (e.g., management): Carnegie Mellon CMM, CMMI[9], Organisational Project Management Maturity Model (OPM3)[10];
- Vendor-related and not specific for cybersecurity training: Gartner’s eLearning Maturity Model[11], Zeroed-In Technologies[12], Human Capital Contribution Model (HCCM)[13];
- Cybersecurity-related CMMs: do not cover training or cover it as a too generic process: SANS CMM for Endpoint Security[14] and Cyber Security CMM[15].
Generically speaking, therefore, some of the common problems of the existing CMMs (e.g., CMMI, SEI, and Common Criteria) are:
- “One size (mis)fits all”: current approaches concentrate on generic training and/ or eLearning to drive down costs[16];
- The frameworks concentrate on the training processes (often seen as an industrial process that must produce constant quality) rather than readiness outputs and agile processes;
- No assessment methods other than the traditional Q&A (often multiple choice) and attendance assessment rather than readiness level;
- Do not consider the use of the immersive business environment (e.g. nomadic working style) that can overcome the traditional metrics;
- Applicable to single organisations and not to supply chains.
One of the most evolved CMMs in cybersecurity training is openSAMM[17] (including the supporting community, the tools and the stakeholders of OWASP). It avoids the common problems discussed above, being agile, flexible and supported by a complete set of support tools. Unfortunately, it does not deliver a full solution for cybersecurity as it focuses on issues related to secure development, and when dealing with cybersecurity training, it exhibits similar issues[18].
Do we need to create a CMM specific for cybersecurity training?
Education and training across an organisation are essential aspects of all organisational initiatives for change; the curriculum is usually driven by analysing the opportunities and risks associated with any change plan. Cyber risk is currently generally addressed outside the broader risk register. Still, the consensus within the risk professional community and the more forward-thinking cyber community is that cyber risks will fold into the general risk processes in a relatively short period. Moreover, Cybersecurity training has some unique characteristics compared to training in general:
- Cyber Security Training is one of the few situations where training is not only a method to increase employees’ skills or sustain the business goals, but also a method of defence of the organisation: on the longer term, improved cybersecurity training improves the overall security of the Organisation;
- The metrics used to measure cybersecurity training success effectivelyare different from other training CMM. In particular, it is a form of training that must be applied to every individual in an organisation or across a supply chain, the form of training may be different for different groups but the objectives are common — defence of the organisation / the supply chain. Metrics must be applied to the individual and the organisation (and ideally the supply chain) as a whole and must link to a demonstrable measure of risk reduction.
A CMM for training in cybersecurity is essential because it integrates and extends existing approaches: some of the questions of cybersecurity training are identical to those of general training (e.g., reuse of Bloom’s taxonomy, Legacy of Kirkpatrick or Philips ROI methodology, etc.). Cybersecurity training has specific, and to a degree organisationally relevant, metrics as part of a Capability Maturity Framework that can be specific to an organisation, when driven from the risk assessment. A way to push the existing solutions to another level is a CMM framework for creating, managing and measuring an organisational cybersecurity training program. An ideal CMM should, therefore, use business and cyber risk-assessment processes, such as that done with HERMENEUT, to specify what a given organisation needs as the desired maturity level and then determine the journey through education and exercises to reach that level.
The CMM and the supply-chain role, a still not integrated combo
The supply chain has a role in this scenario, too, and its actors should have a maturity level adequate to the cyber risk of the leading organisation. The CMM must, therefore, be applied to supply-chain actors. However, what is missing from the currently available CMMs are the risk-driven controls, methods and metrics to measure the supply chain, too. The intention is to create consistent expectations across the supply chain, for example, specifying a security baseline or standard in contracts or service level agreements. A source of inspiration on this matter is the Information Assurance Maturity Model (IAMM) developed by CESG and now included in the National Cyber Skills Centre in the UK policies[19].
The EU context demands more research.
The EU is standardising its education frameworks (e.g., e-CF[20] is now a formal standard, EN 16234), with the final aim to adequately and efficiently integrate also the ICT Security workers and employers in the European e-Skills market. Unfortunately, the offering of certified tracks in ICT Security remains underdeveloped. The European Qualification Framework (eQF) is following a similar strategy[21], fostering social dialogue to find common sectorial agreements and job-matching approaches between workers and enterprises and establishing governance mechanisms based on continuous improvement and quality labels. Security is still not wholly included in these European frameworks because of the profound and highly dynamic changes in society and cybercrime that impact the qualification profiles of ICT security professionals[22].
The open challenge is, hence, a more profound rethink of how to integrate cyber-risk estimation solutions (such as HERMENEUT), dedicated maturity level CMMs and soft mitigation strategies.
References
[1] The DOGANA project (www.dogana-project.eu), among others, demonstrated the impact of the human-related attacks within the current cybercrime strategies, see the document “D2.1 The role of Social Engineering in evolution of attacks”
[2] For example, A. Vishwanath, “Why Most Cyber Security Training Fails and What We Can Do About it”, Blackhat.com, 2017. [Online]. Available: http://ubm.io/2wy8Ugc.
[3] J. Rose, “Selecting, Using, and creating Maturity Models: a tool for assurance and consulting engagements”, 2017. [Online]. Available: http://bit.ly/2wyuWPV.
[4] H. Wagenstein, “A capability maturity model for training & education. Chapter one: background and rationale”, PMI® Global Congress 2006 — North America, 2006 [Online]. Available: http://bit.ly/2wyc9Eh.
[5] while not a normative system, Bloom’s Taxonomy is highly regarded by many trainers are a mature way to think about educational programs, and should be included then in the total range of resources behind training maturity, https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
[6] “The ROI methodology”, http://roiinstitute.net/wp-content/uploads/2014/03/The-ROI-Methodology.pdf
[7] https://www.isixsigma.com/
[8] DMAIC stands for Define, Measure, Analyse, Improve and Control, and is the core tool used to drive Six Sigma projects.
[9] CMMI Institute, http://cmmiinstitute.com/
[10] Ibid. note 7
[11] “People CMM: Quality and Maturity for Workforce Practices”, http://gtnr.it/2w3SSZ0
[13] See http://www.cedma-europe.org/newsletter%20articles/KnowledgeAdvisors/KA0603/KAJeffBerk.pdf
[14] Ibid. note 5
[15] Ibid. note 4
[16] As an example, OPM3 has sections dedicated to training, but not specific for cyber security: section 5200 provides Project Management Training and Section 5210 for continuous training in the use of tools, methodology and deployment of knowledge.
[18] For example, this is the definition of training processes in the OpenSAMM Education & Guidance section: “increasing security knowledge amongst personnel in software development through training and guidance on security topics relevant to individual job functions”.
[19] https://www.ncsc.gov.uk/articles/hmg-ia-maturity-model-iamm
[20] European e-Competence Framework, http://www.ecompetences.eu/
[21] As the first sector-specific implementation of the European Qualifications Framework (EQF), the e-CF fits for application by ICT service, user and supply organisations, multinationals and SMEs, for ICT managers, HR departments and individuals, educational institutions including higher education and private certification providers, social partners, market analysts, policymakers and other organisations in public and private sectors.
[22] Cybercrime is highly multi-disciplinary, and responding to Cyber issues implies cyber tracks that span both technical and human sciences.
