However, how to model these different FAIR input data?  How to compute with these data? Are there tools to simplify their collection or estimate their quality, and what efforts do they require to be implemented?

Having seen previously how trustworthy the risk quantification method was in its processes, let’s now see how the inevitable part of subjectivity can be isolated, and which facilitators can help to obtain reliable results.

The FAIR fuel: data

The risk analysis proposed by FAIR (according to the standardization document published by openGroup)[3]  is carried out in four stages:

• At first, in a fairly conventional way, it is a question of specifying the scope of the examined risk : what is the asset (subject to risk), what is the threat context (agent and scenario), and what is the loss event (the dreaded event in terms of losses);
• The second step (called Evaluate Loss Event Frequency) aims at collecting all the frequency data related to the loss event (and thus intimately linked to the threat agent). This consists of collecting the values for the left branch of the arborescence below.
• The third one (called Evaluate Loss Magnitude), because it assesses the loss, is focused on the asset. It is then a question of estimating the various primary losses (i.e. the inevitable loss in case of risk occurrence) and secondary (or possible loss, i.e. not occurring systematically when the risk advent). Its goal is to collect the values of the right branch in the tree below.
• Finally, the last step (called Derive and Articulate Risk) consists in merging the collected data as defined in the FAIR tree by the various calculations, to obtain the result in the form of usable outputs.

Link between FAIR analysis and taxonomy

Without detailing more the taxonomy, already discussed in the article presented before2, one can note that the standard analysis of a single risk requires seven data  (corresponding to the elements at the base of the tree):

1. Contact frequency;
2. Possibility of action;
3. Threat capability;
4. Resistance strength;
5. Primary loss magnitude;
6. Secondary loss magnitude;
7. Secondary loss event frequency.

It should be added that FAIR invites to decline losses (primary and secondary) into six categories (in order to ease and accurate estimate of the loss):

• The production losses: related to the interruption of the service produced by the asset;
• The response cost: related to the incident response;
• The replacement costs: related to the replacement of damaged constituents of the asset;
• The fine/judgement costs: related to fines, court fees and legal proceedings;
• The financial impact on competitive advantage: related to the impact on the organization in its sector;
• The reputation costs: related to the impact on the public image of the organization.

How do we correctly model risk uncertainty?

Furthermore, it is good to ask the question of what a FAIR data is actually.

Indeed, it is too reductive to define a data by a single numerical value. For example, lets consider a ransomware attack: it would be incorrect to say that an occurrence of this risk would cost exactly 475k €[4] (illustrated by the blue curve on graph 1).

Graph 1: A distribution, a more realistic model than a single value

However, adding uncertainty to this data by accompanying it with a minimum value (which could be  1€ in our example) and a maximum one (of  300 M€ in the same example), while keeping the most likely value stated above, would allow to model much more accurately the reality (purple curve of graph 1).

A data is then defined by a minimum, a maximum and a most likely value (corresponding to the peak of the distribution). We can also, note that such a probability distribution is independent of the kind of values considered: it may as well be a loss in any currency  (cf. the previous example), than an occurrence (for example, between once a year and once every 10 years, and a value more likely around once every two years), or even a ratio (between  30% and 70%, more likely 45%). Hence, we can use these distributions to model all the  data of the FAIR taxonomy.

Another advantage of predicting uncertainty through distribution is that it is possible to fine-tune the degree of confidence in the most likely value, via the kurtosis coefficient of the curve. The higher it would be, the greater the data will be trusted (corresponding to a very marked peak, see the green curve on graph 2). On the other hand, an unreliable data will be modelled by a much more homogeneous distribution (see the red curve on graph 2).

Graph 2: Reflecting the level of trust through distributions

However, using distributions rather than fixed values is a problem when it comes to combine them, which will necessarily be the case when we will make the computations of the FAIR tree. As we can indeed see on graph 3 (the addition of the green distribution and the red one giving the violet), the addition of two distribution does not allow to obtain a distribution as ‘simple’ as the previous ones (it no longer follows a log-normal distribution). This is also the case in the context of a multiplication (the result of which is also complex).

Graph 3: addition of two distributions.

To obtain a mathematically consistent result, game theory gives us a simple way: The Monte Carlo simulations. It is in fact a matter of dissecting the distributions (the green and the red of the graph 3), in a predefined number of random values (called number of simulations), distributed in such a way as to correspond to the given distribution. We can then combine the distributions thus dissected by performing the calculations on pairs of values of each distribution. The new distribution can then be approximated, and will be all the more precise as the number of simulations will be large.

Hands on toolboxes to automate FAIR…

To make these calculations and obtain a numerical value of risk, solutions have emerged (mainly from the FAIR method). We will therefore address here the pros and cons of these tools, which are also cited in the previous article1.

The OpenFAIR Analysis Tool

The first we can cite hire is the OpenFAIR Analysis Tool[5]. While this tool has a pedagogical purpose, it nevertheless helps to understand how FAIR works. It is thus possible to have a first concrete application of the method, and to obtain simply results (only for the analysis of a single risk). Developed by the University of San José (California) in collaboration with the OpenGroup, this tool relies on an Excel sheet to obtain a risk assessment from a predetermined number of  simulations, scrupulously respecting the FAIR taxonomy.

OpenFAIR Risk Analysis Tool: a tool that is first and foremost educational

Very useful to have a first contact with quantification, this tool remains however very limited in terms of use. Finally, one should note that Excel is needed, and it is only accessible with an evaluation license limited to 90 day.

Riskquant

For a larger scale use, Netflix’s R&D department has developed Riskquant[6] solution. It is a Python programming library, relying more particularly on tensorflow (a specialized python module for massive statistical calculation). Riskquant’s particularity is to propose a quantification of risk inspired by the FAIR taxonomy, but with a great freedom in its approach and its implementation. Developed to facilitate the use on containers, it would allow by its design very fast evaluations from csv files.

Riskquant: an original approach but lacking maturity

However, keeping of FAIR taxonomy only a single loss value and a single frequency makes it not very usable, especially in the context of an organization that would seek to precisely scope its risks. In addition, it provides so far only a few exploitable results and clearly lacks maturity. Finally, it seems to have been dormant since May 1^st, 2020 (the date of the last commit on the GitHub page of the solution).

PyFAIR

To conclude on this paragraph on solutions that can be used for a basic implementation of FAIR, the PyFAIR library is available on the official python repository (downloadable via the pip tool). Now mature, the tool allows a decomposition of risk according to the FAIR taxonomy. It also allows the feed of the FAIR tree with intermediates values, or the aggregation of data that can be used for several risks (e.g. allowing groupings by asset or threats). It is capable of calculating overall and global risks, and provides easily usable distributions (exploitable with other simple python modules), but also gives access to advanced charts and HTML pre-formatted reports.

PyFAIR, a complete and efficient library in Python

Although it remains a programming toolbox, hence requiring an appetence and time to develop and maintain a Python solution, PyFAIR is well-designed. It facilitates the implementation of FAIR by staying very close to the taxonomy, and provides functions facilitating implementation and the exploitation of the results. Suitable to be operated on multiple levels (i.e. using it only to calculate results by influencing the fine settings of FAIR and Monte Carlo, or by exploiting its high-level reporting functions), it makes it possible to envisage a use of quantification technically facilitated and on a large scale.

‘Turnkey’ platforms to make data acquisition easier:

Nevertheless, the main difficulty of FAIR remains, as we have seen before, obtaining the data and their trust level. To deal effectively, the most efficient solution is to rely on a platform that integrates a CTI database.

These platforms provide risk threat statistics (very few company-dependent). They also support in deploying and implementing the quantification method in the organization, which includes a guidance in obtaining the appropriate loss data.

RiskLens

The first of these solutions is the RiskLens[7] platform. This solution, directly derived from the FAIR methodology, was co-founded by Jack Jones. It is used as technical support for the development of the method, linked to the FAIR Institute. Emphasing on a technical approach of the method, it focuses on the respect of the standards of analysis  in general  and the definition of the perimeter (first  step  of FAIR) in particular.

RiskLens, FAIR’s application to the letter

Nevertheless, it should be noted that, on the one hand, this solution requires advanced notions in the FAIR methodology to be easily operable. Indeed, the platform does not provide a consequent help in obtaining data (which, as we have seen, remains the keystone of quantification), on the basis that the definition of the perimeter is enough to define precisely the data, and thus to obtain it easily. On the other hand, it is an American platform, which implies that the interface (quite unintuitive) is only available in that language, and that the data collected is also subject to U.S. regulations.

CITALID

The second platform we will mention here is the French startup CITALID, whose approach is fundamentally different. Indeed, it has been founded by two ANSSI analysts, who wanted to link the CTI to the risk management. Thus, using FAIR as the tool to make this link, it makes its effort on the conception and the maintenance of the database, made of solid figures kept up to date, to closely monitor the local and international cyber geopolitical situation.

CITALID, a high value-added database

The CITALID platform provides real support in the definition and the collection of the FAIR data, thus allowing to identify precisely where is the remaining part of subjectivity undeniably linked to risk. Available in French and English, it facilitates the management of cyber risk by taking into account all the parameters of the organization (location, size, sector of industry, level of maturity, compliance with standards, etc.), to provide data originating from appropriate contexts. Furthermore, and in addition to an interactive explanation of each of the platform’s fields, the startup supports its customers in collecting the needed inner data of their organization.

First step with FAIR…

Anyhow, the difficulty will always be to succeed in the transition from qualitative to quantitative estimation. Even if solutions can facilitate this shift, leaving a controlled qualitative method for a new unassimilated assessment method remains a challenge, despite all the benefits the new method promises.

If three points were to be highlighted to pursue on the quantitative way, they could be:

• First, to make sure the required maturity is reached. Quantification requires a good understanding of the level of security of the concerned IS, and a pre-existing and well-established risk management method. If quantification provides solutions to assess the cost of a risk, provision it or estimate  the  ROI  of a measure, it is however useless  (or even counterproductive) to embark on this path too early (at best it will be a waste of time, at worst it will degrade the existing risk management process).
• Then, to have a gradual approach in the deployment of quantification. In a mature IS with stable risk management, it is preferable to gradually adopt the quantitative method. This allows to gain confidence in the estimates produced (potentially by making it coexist with the elder qualitative estimation method) and to assimilate the methodology, while ensuring its integration into the existing risk management workflow.
• Finally, rely on existing experience in collecting cyber risk data. As the difficulty stays confined in obtaining reliable data, it is crucial (to be confident in the method) to have trusted figures. It then seems appropriate make use of a platform that can provide data of quality, and a support in the collection of our own data. It will furthermore have more experience deploying the methodology to various customers. The quality of the provided results will then be the key element in the confidence that the organization will have in the quantitative method.

[1] https://www.riskinsight-wavestone.com/en/2020/11/quantified-risk-assessment-1-2-a-quantification-odyssey/

[2] https://www.riskinsight-wavestone.com/2020/06/la-quantification-du-risque-cybersecurite/

[3] https://publications.opengroup.org/c13g

[4] https://www.sophos.com/fr-fr/medialibrary/Gated-Assets/white-papers/sophos-the-state-of-ransomware-2020-wp.pdf

[5] https://blog.opengroup.org/2018/03/29/introducing-the-open-group-open-fair-risk-analysis-tool/

[6] https://netflixtechblog.com/open-sourcing-riskquant-a-library-for-quantifying-risk-6720cc1e4968

[7] https://www.risklens.com/

Cet article Quantified risk estimate (2/2): What data, what tools? est apparu en premier sur RiskInsight.

It is therefore useful at this point to take a look at the existing methods and the theories that could lead us to concrete results. In the big bang of cyber risk quantification, what are the theoretical foundation for the development of a method? Which ones have succeeded, which ones seem mature? Can we expect in the short or medium term, alternatives to the current quantitative assessment methods?

Roadmap: Risk analysis and quantification:  what can we expect of it?

To locate the quantification in the field of risk management, let’s start by clarifying what we are looking for. Within the risk management process, the primary objective is to define an efficient numerical value, illustrating a level of risk (usually a financial cost).

It is therefore, according to the ISO27k standard, only a new risk assessment. Indeed, preceding phases of risk contextualization and identification have no reason to be affected by quantification. The phases of risk treatment, acceptance, supervision or communication, while they will benefit from the results of the quantitative analysis, are unchanged in their workflow. Simply put, it is only question of changing the way each risk is estimated and computed.

This point, rather trivial but crucial, allows us to ensure that, even if they are fundamentally different from the qualitative methods in their results, the quantitative ones will in any case support pre-existing methods. So, we can be reassured that, although it is necessary to use them to have a mature risk management process, it will also be the basis for the quantification (that will thus exploit the pre-existing risk identification phase).

Now that we have framed the contribution of quantification in an organization’s overall risk analysis, let us specify what we would expect (regardless of the possibility of achieving these assertions):

• On the one hand, it is imperative for this method to be more precise in its result, compared to the qualitative method that it has to replace. This means above all that, from the first occurrence and without having previous results records, it must give a precise numerical estimation (which may as far as possible contain several values: maximum risk or probable risk in particular).
• We may also want it to be faster to achieve (or at least to be carried out in an acceptable time), in order to be able to completely replace the qualitative estimate in the long-term. We are here talking about the time it would take to implement the analysis, without worrying a lot about the time it would take for computations (which can now be efficiently delegated, especially via the cloud). In the end, correlating this with the previous point, it is only question of having a better efficiency than the qualitative evaluation.
• Furthermore, we wish the quantitative assessment to be based on concrete data, in order to gain credibility in the results that will be produced. Indeed, since the workflow of a quantitative method is based on mathematical theories, only an incorrect implementation could introduce subjectivity into the values obtained. This last point would justify that in a time equivalent to qualitative analysis, we have finer results.
• Finally, and this stems from the previous point, we need to have a precise taxonomy, for the collected data to be clearly defined (regardless of the kind of risk). Indeed, if the quantitative estimate is based on proven mathematical theories, the quality of the data produced will then depend only on the quality of the data used as input, and in particular on the relevance and the consistency of the data, depending largely on its definition..

At the core of the galaxy: moving from theory to practice

Having specified what are the characteristics of quantification, let us now see what mathematical theories would take into account the hazard associated with a risk.

Consider, for example, the fuzzy sets theory. This mathematical theory is based on the principle that an element, instead of classically belonging or not to a mathematical ensemble, may only partially belong to it, according to a stated degree. This could be useful to highlight the occurrence or the impact of a risk with the degree of belonging of that risk to ensembles. This theory, while interesting, has not led to concrete applications.

Another approach, which could be called correlative, would be based on the use of self-learning neural networks, to determine from CTI data what the level of risk of a company would be, according to its characteristics. This theory has benefited from the current popularity for artificial intelligence. This led to academics’ studies comparing different modes of machine learning (notably BP[2] or RBF)[3], in order to be used in cyber risk analysis. However, to date, it does not appear mature enough to lead to a realistic method.

Finally, the only mathematical solution that has paid off has been the statistical analysis (and game theory, which offers the means to combine statistical distributions, see the “Risk Quantification and Data: Advice and Tools”[4] article about this subject). The principle of statistical analysis is to rely on statistical observations to estimate the level of a risk. The hazard of risk is then, in large part, taken into account by the distribution of the statistics.

Based on these statistics, two approaches are practicable:

• The first is illustrated by a method proposed by the IMF[5]. It proposes to assess a cyber risk by a detailed statistical analysis. However, it is highly computational and inaccessible for regular use or as a part of a quantified risk estimate. However, it retains an undoubted interest in an analysis of a level of cyber risk on several entities that would have similar data, which may be useful for an insurer or in the banking community. However, it remains confined to this use. Reduced to the already limited scope of entities with acceptable cyber maturity, this method does not seem to be able to offer in the short or medium term an exploitable solution for the IS level of an organization.
• The second is to break down any cyber risk based on common characteristics. This is in particular the approach of the FAIR methodology: it proposes in its taxonomy (see ‘how to apply the FAIR method’1) a dissociation of risk according to its occurrence and the estimated impact, from a financial point of view. FAIR then proposes a declination of these two parameters which, because of their universal nature, may therefore be applied to any cyber risk. This type of method has the advantage of proposing an identical process for the analysis of any cyber risk, facilitating its use in an organizational context (that can then compare cyber risks of distinct natures).

The galaxy of quantification

The FAIR method: a supermassive black hole

Currently, only the FAIR method has risen to applicated quantification solutions for a company. Its monopoly in the field is such that it has become an inescapable reference for a solution or methodology to remain credible. Like a black hole, it attracts to it all the current solutions of quantification. We can, for example, illustrate this with the Risquant Library, developed by Netflix’s R&D department[6]. This one clearly announces that it relies on the FAIR methodology. Nevertheless, he takes great freedom in the interpretation of taxonomy and analysis, but the fact of quoting it allows him to be more easily accepted and recognized.

This hegemony of FAIR can be explained quite easily:

• To begin with, it’s a pragmatic method by design. Its inventor, Jack Jones, set it up when he was an RSSI of a large American group, and was asked to justify cyber ROI. It was therefore initiated for operational purposes, then refined and gained credibility by relying on mathematical tools and theories. This concept of development  (i.e.  the fact that the method was born out of a need, and then mathematically justified) makes of FAIR a method particularly appreciated by the first concerned, that are the CISO and the other cyber-risk managers.
• Then, it was particularly visionary, as she preceded all other methods. Appeared in 2001, the first book about the method was published in 2006, detailing its operation and taxonomy. As time went on, a community was made up around Jack Jones and his method: the FAIR Institute. This community continued the maturation and thz diffusion of the method. More precisely, it helped developing the efficiency of the method by placing facilitators to make it ever usable.
• The FAIR method also has a particularly solid basis: in addition to the publication mentioned above and which was the subject of an enriched reissue in 2016, it is based on two  standardization documents, published by the OpenGroup (the consortium behind the architecture standard of SI TOGAF). The OpenGroup also offers certification to the method, based on its two standards, and which add to the interest laying on the method.
• Finally, FAIR is strongly supported (particularly across the Atlantic): the community that drives it is particularly active, and contributes as much to its evolution as to its promotion: the links between the OpenFAIR and the FAIR Institute, both mentioned above, are substantially close. The strength of his ties is ensured by the fact that Jack Jones, father of the method, plays a central role in both organizations.

Thus, in the world of cyber-risk quantification, the only operational solutions to date all rely on the FAIR methodology, with a more or less large but still displayed parentage.

If the maturity of this method seems now acquired, its monopoly in the field of quantification allows with little doubt to envisage, at least for next years, that it will remain the only method of quantification. In order for another method to be equal, and in addition to the fact that it will have to establish its conceptual credibility, it will above all have to make a place for itself  alongside the hegemony of FAIR, while proving that it is more efficient.

[1] https://www.riskinsight-wavestone.com/2020/06/la-quantification-du-risque-cybersecurite/

[2] Back-propagation

[3] Radial basis functions

[4] See the 2nd article on Risk Insight

[5] https://www.imf.org/en/Publications/WP/Issues/2018/06/22/Cyber-Risk-for-the-Financial-Sector-A-Framework-for-Quantitative-Assessment-45924

[6] https://netflixtechblog.com/open-sourcing-riskquant-a-library-for-quantifying-risk-6720cc1e4968

Cet article Quantified Risk Assessment (1/2): A Quantification Odyssey est apparu en premier sur RiskInsight.