Cybersecurity: testing the responsiveness of the incident management team with the Full Spectrum Vulnerability Assessments
In the book The Black Swan, Nassim Taleb describes the links between the concepts of risks and perception: if a person has not experienced a certain event, he will probably tend to underestimate the possibility that the event will touch him directly. As hypothesized by the sociologist Ulrich Beck, theorist of the risk society, it is possible to anticipate dangers and overcome fears only if we understand that risk is at the center of the life of each of us. This lesson applies to risk in its most general and broadest sense, but it can also be applied to cyber risk and IT security incident teams.
Understanding risk (including digital risk) is fundamental in order to know how to orient oneself in a period of crisis never experienced before, above all to understand in which directions to move.
Since risk cannot be eliminated, it must therefore be understood and anticipated. Anticipating the risk serves to consider that the worst may happen but also to question one’s own certainties, beliefs, prejudices, ideologies, etc.
Perceiving a risk means anticipating the potential disaster in the making.
Cefriel has developed a new method of testing information security. The methodology goes alongside other testing activities of computer systems (the normal technological vulnerability assessments) and of people (simulated social engineering tests, which we call social driven vulnerability assessment –SDVA- [1]). In this case, the objective of the test is the security incident management teams.
The methodology developed by Cefriel is called Full Spectrum Vulnerability Assessment (FSVA): a simulation of a “complete” attack, albeit simulated, which, acting in the ways and methods of real cyber-attacks, has the purpose of testing the “responsiveness” of the team that manages cybersecurity in a company. What is stimulated, therefore, is not the response to a specific problem, but the ability to adapt and think independently when the worst happens.
Recent cybersecurity incidents or disasters, also linked to the rise in cybercrime following the COVID-19 pandemic, have demonstrated the insufficient understanding of how cybersecurity operators react in the event of crisis situations, with implications for policy design and implementation, for example in the form of preparedness plans.
The forced digital transformation of recent months, which took place on both fronts of cybercrime and cybersecurity, following COVID, means that security must also be transformed. Long before the current pandemic, the training of incident response teams used to be approached in conditions of relative peace of mind compared to the real cyberattack landscape: exercises and courses were often not aligned with the trends actually used in cyber-attacks or were simulated games (e.g., cyber ranges) usually run on systems or PCs not used for the business or production.
In other words, looking at the Figure 1, the exercise takes place outside of the “period of intensity”. In general, incident response plans are a complicated part of a company, they are used and tested only in the face of a real attack.
Recent statistics [2] report that, despite the dizzying pace of breaches, about one third of organizations let a year or more pass by between cyber crisis management exercises.
Cefriel’s proposal goes in the direction of providing a method to increase the frequency of training making it at the same time less onerous for people, involving a wider range of stakeholders in the company. This involves realistic, more frequent exercises that can be understood by everyone, from public relations and legal teams to HR.
Simulating an attack in this way, also known as micro-drilling [4], helps team members build vital ‘muscle’ memory or cognitive agility, which will teach them the necessary instincts (the Observe, Orient, Decide, and Act -OODA-) to answer when the worst will happen [3] (figure below).
As can be seen from the Figure 2, on the other hand, FSVAs are a particular testing tool because they are placed halfway between the technical and the organizational, addressing the human element both as a means of offense and defense, allowing to verify the KPIs of organizations and define improvement strategies, at various levels.
How an FSVA is mapped in respect to other types of Assessments
As clarified by the previous figure, in many respects, an FSVA covers activities that are usually carried out by Black, Red, Purple or Gold Team Assessment. An FSVA differs from these types of tests on three macro-elements.
- First of all, the objective, as mentioned above, is not to test the correct technological configuration of the defenses, but to understand the preparation of the IT security team and in general to stimulate its “muscle memory” (or cognitive agility) in stressful situations, perceived as a real attack.
- Second, the ability of an FSVA to link corporate KPIs with test results.
- Third, an FSVA is designed to test a security procedure that typically consists of four elements: policies, tools, procedures, and people.
The figure below shows a mapping of the various types of Assessment with respect to the questions already listed and where conceptually an FSVA is positioned.
FSVAs are a particular type of test that is configured in a unique way, compared to other types of Assessment, precisely by virtue of the results it wants to obtain.
With reference to the Figure 3 above, it should be noted that an FSVA differs from a Purple Team Assessment in that it tests both processes and people. Compared to a Gold Team Assessment, on the other hand, an FSVA has a technical dimension that allows you to create much more realistic tests. Compared to a Black Team Assessment, the main difference is the absence of a reconnaissance phase, which transcends the purposes of an FSVA, not to mention that it is often a decidedly time-consuming and privacy preserving problematic activity. Finally, an FSVA also differs from a Red Team Assessment (whose purpose is to test the security of systems by acting, obviously on authorization, like a malicious hacker): an FSVA focuses more on the human element of defense and the combination of procedures, processes and people, and less on purely technological and configuration aspects.
The theoretical foundations of an FSVA, the security ceremonies
The theoretical foundation of an FSVA falls within the field of security ceremonies (first introduced by Ellison [5]). The term is used to describe the set of computer systems, protocols and human beings that interact for a specific purpose.
A security ceremony can be described as a protocol in its context of use, made up of machines, protocols, applications and humans (see Figure 4).
Many protocols (for example, organizational or governance) that have been shown to be safe in theory, are insecure in practice when deployed in the real world. Security tests often test protocols in isolation, they do not consider the wide range of possible attacks when the perimeter of investigation is extended, including social engineering, interfaces / interference with other protocols and the environment.
Security ceremonies are a superset of normal security protocols, an extension that includes operations that are not normally considered or are taken for granted (e.g., the psychological reaction of operators to an external stimulus). For example, did who designed a SOC intervention protocol, in the face of a cyber threat, have in mind an operator with specific responsiveness and skills? And are operators fitting to this profile?
The inclusion of human interaction, and consequently of behavior and cognitive processes, is a characteristic of the ceremonies, which is normally difficult to consider because it is outside the limits of the current modeling systems of operational protocols.
FSVAs are a solution that Cefriel has designed to test the security of a particular type of security ceremonies, those related to incident management response teams.
The theoretical foundations of FSVA, the difference between awareness, training and learning.
Three terms often used as synonymous in common language, have very different meanings in cybersecurity: awareness, training and learning. While all three contribute to the definition of both people’s capabilities and an organization’s defense techniques, it is good to understand the difference.
The term awareness is the simplest and refers to the brief knowledge of a phenomenon without the need for its complete understanding. Awareness is for example a anti-phishing campaign. Conversely, training is the provision of information and knowledge, through speech, written word or other demonstration methods, in order to statically instruct the trainee. Learning is the more complex issue of the three cases as it is the process of absorbing information in order to increase skills and abilities and thus make use of it in a variety of contexts [6].
From this definition it follows that training is something that arises from the outside, preparing a person to respond to predetermined operations, (usually training for a specific task / activity / topic); while learning takes place inside (in the mind), and prepares a person for new and unexpected situations.
In the literature the need to make the transition from training to learning has been stressed several times in the context of cybersecurity [7].
In this sense it can be said that an FSVA is a learning method and not a training method. In particular, a learn-by-doing method, but in an off-guard context. The strong point of an FSVA is the realistic simulation of an attack, more or less complex depending on the specifications, but performed outside of a simulation, with procedures, mind-sets and dynamics completely similar to reality.
Comparison between FSVA and cyber ranges
Cyber Ranges (CR) are a typical learn-by-doing methodology for cybersecurity teams, primarily used today to increase incident management security teams’ preparedness. Today, the cyber range market is quite mature, and several offers are ranging from off-the-shelf automatic simulations to more complicated (and expensive) setup. Figures 5 and 6 present an overview of the main differences between an FSVA and a cyber range. Without going through the rows one by one, it is useful to underline the most significant difference. In an FSVA, people perform the activities on the production machines without being aware that they are part of a simulation. The off-guard requirement is a fundamental aspect of an FSVA. Consequently, the second element reported in Figure 5 is the so-called 7th layer of security (the human element). The human element is fundamentak characteristic of an FSVA, while often it is not in a cyber range.
The core concept of an FSVA is to simulate a real(istic) attack in the production context, concentrating on people more than systems. This characteristic leads to several differences with cyber ranges. In our opinion, cyber ranges are ideal for improving the security incident teams’ preparedness and self-confidence, in a controlled context. Still, the definitive test must be in a real(istic) and stressful condition, implemented with something like an FSVA.
Expected benefits
There are thee main ways to stimulate the spider sense of incident management teams:
- Operational capabilities are not sufficient to keep the pace of cybercrime.
- The human element cannot be removed from both attacks and defenses strategies.
- Need to increase the level of automation within cyber-defence capabilities, but it is anyway impossible to completely substitute the human judgement. e.g., “AI Stats News: Humans Plus AI 20X More Effective In Cybersecurity Defense than Traditional Methods”, Forbes, November 2019
This leads to the following main business benefits:
- Improve an organization’s resistance to attack in ways not yet encountered.
- Train the company’s incident management team to handle advanced and persistent attacks.
- Benchmark the performance of security operations, for example against KPIs or corporate policies
- Understand and gain confidence in the resilience of the organization.
- Plan organizational, governance and technical mitigations.
Conclusions
Through an FSVA, a targeted simulation is carried out, using specific evaluation parameters, measuring the real effectiveness of an organization in the management of cyber security threats. Using an active approach, an FSVA exposes critical vulnerabilities of internal business processes by simulating multi-vector cyber-attacks. An FSVA simulates a realistic attack, impersonating hackers, other malicious agents or even insiders, in a proactive way, that is, before a real attack takes place. All this without any impact on users or infrastructure. With an FSVA, an organization can proactively test its cyber security posture against cyberattacks.
In summary, the focus of an FSVA is on processes, governance, management KPIs and growth of the operational capabilities of the staff, allowing at the same time for a clear indication of how an attack similar to the simulated one would have acted on people and machines.
References
[1] https://link.medium.com/ehVN4A3x4bb
[3] https://sbscyber.com/resources/top-5-most-common-incident-response-scenarios
[5] Ellison, C.: Ceremony design and analysis. Cryptology ePrint Archive, Report 2007/399 (2007), http://eprint.iacr.org/
[6] B. Olaniran, ―Culture, learning styles, and Web 2.0, Interactive Learning Environments, 2009
[7] C. Di Gregorio, “Law enforcement training and learning: a comprehensive “capacity building” approach,” European Law Enforcement Research Bulletin, vol. 3, pp. 31–44, 2017.
