From static detection methods to behavioral analysis

As attacks evolve more and more rapidly and in an increasingly sophisticated way, the SOC (Security Operations Center) is forced to review its approach and existing tools as static detection mechanisms become obsolete:

• The historical approach uses the recognition of known behaviors and footprints (e.g. malware signatures). This method, called misuse-based, provides explicit alerts that are easy to analyse for operational staff, but only attacks that have already occurred and been detected can be recognized.
• The new approach aims to analyse actions that deviate from the behavior normally observed, without having to explicitly and exhaustively define a malicious act (e.g. the behavior of an individual who deviates from that of his colleagues). This anomaly-based approach makes it possible to detect attacks that are not directly run through the tools but require high volumes of data.

The anomaly-based approach exploits the correlation capabilities of unsupervised learning algorithms that highlight links between unlabeled data (i.e. not categorized as normal or abnormal).

Recipe: detection of anomalies on a machine learning bed

To know if Machine Learning is appropriate for its context, the best solution is to create a PoC (Proof of Concept). How do you implement it? What are the key points to look out for? Here are the key steps in our development.

Starter, main or dessert: define the use case

Doing Machine Learning is good, knowing why is better. Defining a use case is like answering the question ‘What do you want to observe?’ and determining the means available to respond.

In our context, a use case is a threat scenario involving one or more groups of accounts (malicious administrators, exfiltration of sensitive data, etc…). To evaluate them, several criteria must be taken into consideration:

• Utility: what would be the impact if the scenario were to happen?
• Data availability: what are the available sources of useful data?
• Data complexity: is the available data structured (numbers, tables) or unstructured (images, text)?

We have chosen to work on the compromising of service accounts: some may have important rights, and their automated actions generate relatively structured data. In the context of a PoC, a limited scope, and homogeneous and easily accessible data sources are essential to obtain concrete and exploitable results, before considering more ambitious use cases.

Ingredient weighing: determine the data model

In order to make the best use of the data, it is necessary to define a behavior to be modeled based on available information. This is where business expertise comes in: can an isolated action be a sign of compromise or should a series of actions be considered for detecting malicious behavior?

First, we defined a model based on the analysis of unit and family logs (e.g. connections, access to resources, etc.) to evaluate the overall functioning. However, a model that is too simple will ignore weak signals hidden in action correlations, while a representation that is too complex will add processing time and be more sensitive to modelling biases.

Selection of tools: choose the algorithm

Several types of algorithms can be used to detect anomalies:

• Some try to isolate each point: if a point is easy to isolate, it is far from the others and therefore more abnormal.
• Clustering algorithms creates groups of points that look alike and from this it calculates the center of gravity of each one to create the average behavior: if a point is too far from the center, it is considered abnormal.
• Less common, auto-encoders are artificial neural networks that learn to recreate normal behavior with fewer parameters: behavior reproduction errors can be considered as an anomaly score.

Other approaches still exist, including the most exotic artificial immune systems that mimic biological mechanisms to create an evolving detection tool. However, it should not be forgotten that a simple and well optimized tool is often more effective than an overly complex tool.

The k-means clustering algorithm was selected in our case: used in the detection of bank fraud, it simplifies re-training which allows the tool to remain adaptable despite changes in behavior.

All these algorithms can also be enhanced, depending on the chosen behavior model, to consider a series of actions. Thus, convolutional or recurrent neural networks can be added upstream to take into account time series.

Preparation of ingredients: transforming data

Once the algorithm has been selected, the raw data must be processed to make it usable. This process is carried out in several steps:

• Cleaning: correction of parsing errors, removal of unnecessary information and addition of missing information.
• Enrichment: adding data from other sources and reprocessing fields to highlight information (e. g. indicate if a date is a public holiday…).
• Transformation: creation of binary columns for qualitative data (e.g. account name, event type, etc.) that cannot be directly transformed into numbers (one column for each unique value, indicating whether the value is present or not).
• Normalization: reprocessing the values so that they are all between 0 and 1 (to prevent one field from taking over from another).

Due to the variety of possible events and the complexity of the logs, we have chosen to automate this process: for each field, the algorithm detects the type of data and selects the appropriate transformation from a predefined library. The operator can then interact with the tool to modify the choice before continuing the process.

Seasoning: test and optimize the tool

Once the model has been defined, the algorithm chosen and the data transformed, the tool developed should be able to raise alerts on anomalies. Do these alerts make sense or are they false positives?

In order to evaluate the performance of the tool, we performed two types of tests:

• Intrusion simulation by performing malicious actions to check if they are detected as abnormal (this approach can also be handled by directly adding “false” logs to data sets).
• Analysis of anomalies by checking whether the alerts raised actually correspond to malicious behavior.

Many parameters can be adjusted in the algorithms to refine detection. Performance optimization is achieved through an iterative process; changing parameters and observing the effect on a set of validation data. Manually time-consuming, it can be improved by the AutoML approach which seeks to automate certain steps by using optimization algorithms.

However, parameter optimization is not enough: the results of our PoC have shown that the quality of detection based on behavioral analysis depends largely on the relevance of the behaviors defined before the algorithm is developed.

ML or not ML: that may not be the question

Despite its undeniable advantages, Machine Learning is a tool to be used in a rational way: frameworks are becoming increasingly accessible and easy to use, but the definition of the use-case and the behavior model are still crucial steps that exist. These choices, where business expertise is essential, will irreversibly influence the choice of data, the selection of the detection algorithm and the tests to be performed.

The question is no longer ‘Where can I put Machine Learning in my SOC? ‘, but rather ‘Of all the approaches available, which is the most effective to address my problem?’.

To find out, there’s only one solution: light the fires!

To go further…

… here are the tools used during our PoC:

• IDE
  • Pycharm: clear and practical development environment with efficient library management
• Language
  • Python: a language widely used in the field of Data Science with many powerful libraries
• Libraries
  • Scikit-learn: complete Machine Learning library (supervised, unsupervised…)
  • Pandas: complex processing of data tables
  • Numpy: handling of matrices and vectors
  • Matplotlib, Seaborn: display of graphics for visualization

Cet article Detect cyber incidents with machine learning: our model in 5 key steps! est apparu en premier sur RiskInsight.

Against a backdrop of an acute shortage of cybersecurity skills, these issues cannot be addressed solely by increasing the size of SOC teams. The use of new tools, based on four strategic areas, is essential in enabling SOCs to stay ahead of threats.

Here, extending the scope of detection enables the protection of new areas of the IS that are not sufficiently secure (such as the cloud) and of resources that are increasingly being chosen as targets (through ransomware attacks on terminals, targeted attacks using ADs, etc.).

At the same time, new approaches need to be adopted to detect targeted attacks (0-day, “low signal”, etc.), whose increasing sophistication is undermining existing security measures.

In addition to these new detection tools, an advanced knowledge of threats and attackers can improve existing detection capabilities, help prioritize incidents to be dealt with, and increase the effectiveness of the response.

But SOC teams are already struggling to process the events generated by existing tools. As a result, it’s essential to standardize and automate interactions between teams and systems, and, wherever possible, the sequence of analysis and response.

Follow our series on the topic and learn how to tool up in these four strategic areas!

Extending the scope of detection to new perimeters

A unique solution to secure all clouds: CASB

Cloud Access Security Brokers (CASBs) address an area of the IS that is poorly served by traditional security measures: the cloud. The very nature of the cloud means that protection in this area requires a different approach to that used for a conventional IS; there is little or no control of resources (infrastructure, OSs, or applications—depending on the type of offering), assets are located outside the IS, etc.

CASBs aim to centralize and ensure that security policies are applied. Some cloud providers offer their own CASB security services (for example, Microsoft’s Cloud App Security); but, depending on the needs, it may be preferable to use third-party solutions, even though there is a cost to adding in another player. While CASBs aim to ensure security levels in the cloud, relying on the cloud service providers to perform this monitoring role can be counterproductive: it’s preferable to make use of a “trusted third party”.

In all cases, CASBs offer a diversity of solutions that can include a very large number of services—their degree of maturity depending on the solution’s publisher, the cloud provider, and the type of hosting (IaaS, PaaS, SaaS, etc.).

On the one hand, CASB solutions make it possible to deal with specific cloud issues, by addressing the lack of visibility in these environments (through shadow IT detection, usage statistics, etc.) and ensuring that they are compliant (verification of configurations, etc.).

On the other hand, they play a part in the application of traditional security measures in this cloud. In particular, data security issues (such as DLP and encryption measures, which are of special concern to regulators) and threat detection (centralization of cloud logs for transmission to SIEM, detection of abnormal behavior using UEBA (see our dedicated article on this), etc.) are parts of a CASB traditional capabilities. In addition, some stakes associated with IAM can also be addressed by these solutions (SSO, access contextualization, etc.).

There are two main modes of deployment when putting these features in place, each with its advantages and disadvantages. Proxy-type solutions are placed between users and the cloud service.

In contrast, when using API-type solutions, which are sometimes called “out-of-band”, the cloud service’s consumers communicate directly with it. Each time it’s accessed, the service queries the CASB’s APIs to evaluate the risks and authorize (or prohibit) the consumption of the service. However, to operate, API solutions rely on the interfaces offered by the cloud provider, which may limit the options.

At present, CASBs are relatively new and immature solutions, and their deployment is limited. However, given the increasingly broad adoption of cloud services (already well advanced), CASBs undoubtedly have a bright future. They’ll enable SOC teams to extend their surveillance to this area, which will soon represent a large proportion of any IS.

Examples of CASB publishers:

The new Swiss army knife for securing terminals: endpoint detection and response (EDR)

Endpoint Detection and Response (EDR) solutions are set to enhance SOC’s detection and response capabilities for terminals (PCs, servers, etc.).

As the name implies, EDRs play a part in detecting attacks. In fact, they are plugging the gaps in anti-virus solutions (and other HIPSs) which make use of specific attack signatures and are therefore unsuited to detecting certain attack types—in particular advanced attacks (APTs). EDRs are based on other detection methods, with publishers generally offering a combination of techniques commonly used elsewhere.

Among these techniques, a large number of solutions detect the exploitation of known vulnerabilities or attack patterns (the opening of suspicious ports to dubious addresses, etc.), the analysis of files using a sandbox (local emulation, submission in the cloud, etc.), and behavioral approaches based on Machine Learning (in particular UEBA solutions—see the dedicated chapter on this). Depending on the SOC’s needs, the alerts produced can be integrated as SIEM sources, or made available directly from the solution management console.

In addition to their advanced detection capabilities, EDR solutions also result in a considerable increase in visibility on devices: lists of processes and services launched, lists of files in certain system directories, as well as other information that facilitates investigation in cases where an alert is raised. Some solutions go beyond mere recovery of the state of the terminal at the time of the request, enabling its history to be recovered too: generation of logs, recovery of deleted files, etc.

But EDRs’ features don’t end at the detection and analysis phase. In fact, these solutions enable remote remediation actions to be performed, and the complexity of these depends on the publisher: deleting or quarantining files, ending processes, quarantining the terminal from the network, modifying registry keys, etc.

EDRs, thus, are comprehensive solutions that come into play at every stage of the process: from detection, through analysis, to response. However, they are not intended to replace anti-virus solutions: it’s always more effective to block known attacks, even though publishers are increasingly offering solutions that combine these two types of functionality.

For more details on EDR solutions, read our dedicated article here.

Examples of EDR publishers:

Protecting the keys to the kingdom: Active Directory supervision

Directories are among an IS’s most critical components. They provide the authentication and authorization functionality for almost all IS resources—both technical and business function—including the most critical ones. It’s therefore not surprising that compromising the AD is one of the most frequent attack methods used, since it opens numerous doors to an attacker.

Despite this criticality, and the fact that AD architectures are well known and have evolved little in recent years, their security has scope to improve. This is due, in particular, to their specific mode of operation (OUs, domains, trees, forests, users, etc.), which renders traditional protection and surveillance methods ineffective; a significant concern given that any vulnerability can represent a major risk for the rest of the IS.

AD surveillance solutions aim to overcome this problem by supervising (in real time, or during an audit) the specificities of directories (configuration, status of accounts, etc.) and detecting vulnerabilities that could result in them being compromised. To do this, AD supervision solutions have a highly detailed knowledge of how ADs function, and, in particular, the associated security issues.

When the solution detects a vulnerability, it raises an alert (via the SIEM, or directly) and can provide remediation advice to facilitate the work of the teams responsible for rectifying the problem.

AD supervision tools also enable the SOC to detect any changes in configuration (legitimate, accidental, or malicious) and continuously assure security levels for these critical components. In doing so, they make the task of numerous attackers decidedly more complex.

In addition to directly strengthening the AD’s security levels, such solutions can also be used to ensure compliance with standards or regulatory requirements (for example PCI DSS, etc.).

These solutions are not widely applied today, and their use is generally limited to one-off audits. However, given the considerable security improvements associated with the provision of detection and remediation advice, and their ease of use, such solutions have strong potential and are likely to find their place among the tools used by SOCs.

Examples of AD supervision publishers:

You can find our second article in the series here.

Cet article SOAR, UEBA, CASB, EDR and others: which tools do you need for you SOC? (1/3) est apparu en premier sur RiskInsight.