This article presents four complementary approaches to strengthening end-to-end backup security: 

1. Continuously ensuring the availability of usable backups 
2. Strengthening the security of the backup infrastructure against attacker takeover 
3. Protecting backups against logical destruction 
4. Identifying residual risks in light of the measures implemented 

This article is published in two parts: the first focuses on approaches 1 and 2, followed by a second publication covering approaches 3 and 4. 

The recommendations presented do not replace those set out in ANSSI guidelines, which define the fundamental principles of backup [2] practices. 

Figure 1: Strengthening Backup Security Through Four Approaches 

1. Continuously ensuring the availability of usable backups 

To guarantee the availability of usable backups, it is essential to apply fundamental best practices. 

Ensuring backup completeness and consistency 

In the context of a ransomware attack, the primary objective of backups is to provide a reliable data source enabling the reconstruction of the information system. Backups are truly effective only if they contain all the elements required for full recovery. This notably includes businesscritical data, configurations of business applications and systems, installation sources, as well as critical operational data such as password vaults, licenses, and operational documentation. 

Backup completeness alone is not sufficient. The need for data consistency points across backups originating from different sources (e.g., a document management system (DMS) database and its associated files) must also be taken into account. Conducting a preliminary analysis helps facilitate data resynchronization across different repositories during the recovery phase. 

In addition, it is necessary to maintain backups of the infrastructure itself to enable identical reconstruction. These backups must include the backup catalog, software installation sources, encryption keys, and all other required secrets. A copy of configuration parameters should be stored in a separate location, such as an offline environment, distinct from the primary infrastructure, in order to limit the risk of a shared compromise. 

According to the Cyber Benchmark conducted by Wavestone across more than 170 assessed organizations, approximately 90% of the organizations observed perform regular data backups. 
Among organizations that perform regular backups: 

• Approximately 65% conduct restoration tests 
• Approximately 20% perform business data consistency checks 

In this context, various controls must be defined and implemented on a regular basis. 

Testing Backup Reliability Through Regular Controls 

A first level of control aims to ensure that backups are effectively performed and remain usable. This can be based on the application of daily verification procedures relying on evidence such as reports, logs, and alerts. These checks may be manual or (semi) automated. However, an additional human review remains necessary to ensure that indicators and alerts are not misleading, particularly in the event that monitoring and control mechanisms have been compromised or disabled by an attacker. 

This first level also includes periodic restoration tests, carried out on representative scopes, in order to verify, where possible with the involvement of application or business subject-matter experts, the integrity and completeness of the data required for business recovery. 

The second level consists in ensuring that first-level checks are properly applied. It relies on independent controls or formalized processes. Dashboards may be used to centralize confidence-level indicators by correlating the results of daily operational checks with restoration test outcomes. 

Once the reliability of backups has been established, restoration processes should be optimized by regularly testing them and ensuring their effectiveness. 

Industrializing Restoration Processes to Optimize Recovery Time in the Event of a Compromise 

To reduce recovery time following a compromise, it is essential to industrialize restoration processes at scale in order to support mass recoveries. This requires preparing these processes in advance, testing them regularly, and adapting them to different destruction scenarios. 

As the restoration phase of an information system may extend over several weeks, or even several months, it is necessary to increase backup retention periods for the data to be restored, in order to prevent their loss through overwriting or premature deletion. 

Restoration processes must also include mechanisms to rapidly assess the state of backedup data by identifying, based on indicators of compromise, data that has been compromised, modified, or corrupted, so as to effectively target the appropriate restoration points. 

Integrating the Risk of Backup Compromise into the Restoration Strategy 

To ensure reliable recovery following a compromise, it is essential to account, within the overall restoration strategy, for the risk of alteration or manipulation of backedup data. This involves addressing the risk of data alteration or manipulation occurring upstream of backup processing by the backup agent, for example: 

• Being able to rely on full backups created prior to the attacker’s intrusion, as identified during the initial investigations. In such cases, the backedup data can be considered uncompromised and used to rebuild systems and applications. 

When restoring unaltered application or system components that are not reinstalled from trusted sources, the restoration process must also include the application of security patches and hardening measures to prevent any subsequent compromise. 

The backup process alone cannot prevent potential data compromise before the data is handed over to it. Depending on the context, additional measures may be implemented, such as: 

• Protecting data integrity through system-level mechanisms and/or cryptographic means; 
• Detecting data alteration through application-level validation, monitoring of “canary files” data, or the use of an EDR (Endpoint Detection and Response) solution. 

These topics must be addressed in addition to backup protection measures. 

Extending Backup and Restoration Best Practices to Cloud Environments 

Finally, the backup rules defined for onpremises environments must be replicated and adapted to cloud environments. 

According to the Cyber Benchmark conducted by Wavestone, approximately 25% of the organizations observed have a regularly reviewed and updated backup policy covering both onpremises and cloud environments. 

In addition, around 29% of organizations externalize a backup of their cloud data to another region or to an onpremises environment, ensuring resilience against cyberattacks and regularly testing this process. 

Beyond the usability of backups, securing the infrastructure that hosts them represents an equally critical challenge, one that is sometimes insufficiently addressed. 

2. Strengthening the security of the backup infrastructure against attacker takeover 

Before considering more advanced mechanisms, it is important to recall that effective backup protection first relies on best practices for securing the backup infrastructure, notably those documented by ANSSI3. A compromise of this infrastructure could indeed result in the alteration of backups (encryption, destruction, etc.). 

Ensuring Defense in Depth for the Backup Infrastructure 

These best practices include segregating production and backup environments, using dedicated administrative accounts, and hardening infrastructure components, particularly through the application of ANSSI hardening guides applicable to Windows, Linux, and other systems. They also apply to backup agents, which may constitute a propagation vector toward production systems. 

In addition to hardening measures, the backup infrastructure must be subject to both technical and cybersecurity monitoring. 

Implementing technical and cyber monitoring of backup infrastructures 

Technical monitoring of backup infrastructures helps ensure their proper operation and detect any anomalies. The effective handling of detected anomalies must be regularly reviewed. 

Cybersecurity monitoring of the backup infrastructure relies on appropriate logging and traffic analysis. It must be capable of detecting the main attack techniques observed in the wild. 

Maintaining Threat Intelligence Focused on Backup Systems 

Threat intelligence specifically targeting backup systems must be maintained, beyond the technical vulnerability monitoring performed as part of maintaining the backup infrastructure in a secure operating condition. This threat intelligence should cover attack techniques and tactics used against backup infrastructures, in order to anticipate potential attacks and adapt protection, detection, and response capabilities accordingly. 

Despite the measures implemented to prevent the compromise of backup infrastructures, the risk of logical destruction remains and must be anticipated. 

Reference

[1] Wavestone, CERT

[2] ANSSI, Sauvegarde des systèmes d’information

[3] ANSSI, Sauvegarde des systèmes d’information

Cet article Backups : The Last Line of Defense Against Ransomware – Part 1  est apparu en premier sur RiskInsight.

Wavestone assists many victims in managing their crisis triggered by ransomware attacks. Often, poorly trained IT and management teams make incorrect decisions in response to these unexpected situations.

This list of the top ten pitfalls to avoid in ransomware crisis management is based on the feedback from over 5 years of supporting victims.

#1 Paying the ransom will speed up your back up process

Although the French National Agency for the Security of Information Systems (ANSSI) recommends never paying a ransom, this question will always arise for certain stakeholders, especially the decision-makers, who are not completely aware of these issues. Beyond the fact that paying the ransom encourages attackers to continue their activities, it should also be noted that paying the ransom does not always lead to the recovery of the decryption key.

In cases where a ransom payment allows the decryption key to be obtained, the decryption time is often very lengthy. It can take several hours or even dozens of hours per server or workstation, depending on the size of the encrypted files. When there are large numbers of computers, strict coordination is necessary for processing all the systems. In comparison, it will be faster to restore systems from backups.

In addition to the time required, the encrypting or decrypting process is rife with errors, according to past experience. Thus, even with the decryption key, certain files, particularly the largest ones, are sometimes modified and cannot be restored as they are.

In short, we are far from the widely held notion that a simple click can restore a functional information system after ransom payment. There is no point in considering a ransom payment to restart services if there are healthy and functional backups. In the case of data theft, the issue gets more complex.

#2 Reopening the information system too quickly

To limit operating losses due to inactivity, the crisis unit is often urged by the business to restart the information system as quickly as possible. During the reconstruction phase following a cyber-attack, it is necessary to control the following two main risks:

• Resumption of the attacker’s attack and re-encryption of the reconstructed systems
• Theft of new information by the attacker who restarts the data exfiltration

Isolating the information system from the outside world is the initial security measure that must be implemented to reduce these risks significantly (internet access in and out, links with partners, etc.). This measure eliminates the connection between the attacker and his malicious tools, and thus drastically reduces the likelihood of a repeated attack. Certain external flows can be opened individually for the most critical activities (from or to a controlled server, on a given port), but reopening a wide range from or to the outside increases the risks.

Restoring servers is the first step in the rebuilding process. However, since servers are not always backed up simultaneously, resynchronisation of applications and data are often necessary before the servers are put into service. As an example, a hasty restart of a payment application during a crisis has already resulted in a double payment of hundreds of thousands of euros, which had already been transmitted to the bank before the attack.

#3 Wait for an extreme level of security before reopening

As cyberattacks can have a significant impact, fear of repeated attacks can be traumatic for crisis teams, leading them to impose an extreme level of security before restarting. The challenge here is to find the right balance between security and rapid recovery.

There are two possible strategies for rebuilding servers:

• The green zone strategy- It prioritises security over the speed of recovery. It involves creating a new network zone in which only the rebuilt machines are hosted. The technical components (Active Directory, DHCP, DNS, etc.) are dedicated to this secure zone. However, changing the addressing plan can have unintended consequences and it will slow down the reboot.
• The grey zone approach- It favours the speed of recovery over security. It consists of rebuilding or restoring servers in their initial zone. Compromised machines can be found alongside the restored machines. This strategy will represent a risk when the attack is propagated from server to server.

The choice of strategy must be made in the light of the investigations and techniques used by the attacker. As mentioned earlier, isolation from the outside world remains as a key measure in reducing the risk.

#4 Denying a restart in degraded mode

In most companies, IT has become indispensable for all their operations. In the exceptional situation of a total IT shutdown, it is often initially inconceivable to continue working without IT. This is the common argument put forward by the IT and business managers during cyber crisis.

By working in degraded mode, it is possible to limit the impact in many instances. This includes implementing previously defined resilience and business continuity plans. To consider implementing these plans when all the activities are ceased during a crisis, special emphasis must be given to when these activities were stopped during the crisis, as well. Managers often underestimate the mobilisation and creativity of the teams to work differently (disengaging non-essential processes, transferring to a partner, carrying out operations manually, etc.).

It is essential to find a good balance between restarting the first application in degraded mode within two days, rather than waiting two weeks to have the complete application chain.

#5 Wanting to restart all at once

Quite often, the information systems of large organisations contain hundreds or thousands of applications. During cyber crisis situations, it is impossible to spend an enormous time on all these applications. Thus, it is essential to prioritise them by defining a restart order.

Each department often tends to consider that its own activity is the most important in the company. Arbitration by general management is often necessary to establish a restart plan that can be utilised by everyone. Some departments, countries, or regions will see their applications getting started later than others too.

It should also be noted that there are many dependencies between applications, and these are not always known by IT teams. But this must be considered in the reboot plan.

#6 Executing major changes in a hurry

It is sometimes tempting to want to take advantage of the situation of a system shutdown to carry out major changes in the information system.

Security teams see this as an opportunity to carry out projects that the IT department has turned down in the past because of their perceived complexity and impact: network partitioning, upgrading operating system versions, Active Directory tiering, multi-factor authentication, etc.

If these projects are necessary to reinforce a level of security, it is advisable to avoid making too many changes during the crisis period. Thus, the teams can concentrate on actions that are necessary to restart within a controlled level of security.

For example, partitioning the network requires a review of the addressing plan, the addition of network equipment, and the modification of configurations. These actions often cause additional problems to be dealt with (IP hardcoded in the application, blocking of flows necessary for the proper functioning of the application, etc.).

#7 Waiting until investigations are complete

Investigations are essential to understand the attacker’s techniques and identify the vulnerabilities they exploited to carry out the attack. These vulnerabilities can be corrected during the reconstruction to avoid a new attack. However, investigations are complex and time-consuming (sometimes several weeks).

A standard mistake is to wait until the end of the investigation to launch the reconstruction. In reality, it is essential to start the reconstruction operations before the end of investigations. According to the conclusions from the investigative teams, the plan for secure reconstruction will be regularly revised.

In many instances, it will not be possible to identify patient zero or reconstruct the attack’s chronology. Indeed, the traces (log) maintained by the information system are not always precise (verbosity) or it does not permit going back far enough (retention time). They have sometimes been deleted by the attacker itself, while erasing his traces.

Finally, decision-makers frequently ask whether the data has been exfiltrated from an information system. It should be emphasised that unless the organisation has advanced security systems, it is rarely possible to respond precisely to this issue (DLP, for example).

#8 Not anticipating human resource management

Cyber crises are situations where employees are intensely mobilized. Some members of the teams can be so determined to settle this matter, that they do not wish to quit at all! Concurrently, it is quite common to see cases of burnout, hospitalization, or sick leave during a poorly managed crisis.

In the case of a large-scale ransomware attack, intensive team mobilisation will take at least three weeks. It is essential to organise team rotations from the start of the crisis ensuring that key resources are maintained over time.

It is often appropriate to call on external service providers for expertise (digital investigations, Active Directory reconstruction, backup systems, etc.) to multiply efforts (reinstallation of workstations, upgrading of operating system versions, coordination of tasks, etc.).

#9 Hiding the situation from employees, partners, customers

More than five years ago, companies that suffered a cyberattack but failed to take adequate precautions had their reputations severely harmed. Due to the rise in cyber-attacks, an increasing number of organisations are falling victim to cyber-attacks, and firms will be evaluated not just on their ability to manage a crisis, but also on their status as victims.

One of the first reflexes of management teams is often not communicating with stakeholders (employees, partners, and customers) in the hope that the attack will go unnoticed. It is clear that a major attack will always end up being communicated. In these situations, social networks and the media will communicate before the victim, who will then have to adopt a defensive posture in response. It is recommended to adopt a posture of transparency with the stakeholders, who will be reassured to know that the situation is under control. 

Finally, there must be appropriate communication between partners. Without specific information, many of the attackers tend to cut all the links with the victim, and thus the reopening could be longer and more complex. Moreover, s will be inclined to reopen if the victim has proactively warned them.

#10 Failing to structure crisis management

Since ransomware attacks have a significant impact on information systems, they inevitably cause chaos within an organization. Thus, no longer we can rely on the current communication and decision-making procedures.

Some people think that a crisis situation allows approximate management: improvised meetings, follow-up of perfectible actions, lack of formalisation, etc. On the contrary, decisions must be precise, rapid, formalised, and communicated to all stakeholders involved. This discipline is the key to successful crisis management, where everyone finds their place and is utilised wisely.

In certain large-scale crises, such as those involving international groups, keeping a PMO crisis unit for several dozen individuals is advantageous. It will be their responsibility for consolidating precise inventory, organising crisis committees, communicating, and following up on the decisions taken by the crisis unit. This unit of crisis management professionals is an indispensable asset for effective crisis management.

A ransomware attack is a sudden event that has a very significant impact on the business’s operations. There are various pitfalls to avoid in crisis management to quickly regain a functional situation. To optimise operations and gain valuable feedback, it is strongly advised to surround yourself with cyber crisis management professionals.

Cet article Successful Ransomware Crisis Management: Top 10 pitfalls to avoid est apparu en premier sur RiskInsight.

MAUI

Sources:

https://www.cisa.gov/uscert/ncas/alerts/aa22-187a

https://stairwell.com/wp-content/uploads/2022/07/Stairwell-Threat-Report-Maui-Ransomware.pdf

https://securelist.com/andariel-deploys-dtrack-and-maui-ransomware/107063/

Ransomware Activity

Presentation of the figures collected by our tool on the data given by the RaaS platforms about their successful attacks. This graph gives an estimation of the number of victims by the most active RaaS groups, by month.

Number of announced victims by the most active Ransomware-as-a-Service (RaaS) for the past 10 months:

Noticeable change:

• Lockbit2.0 disappears to make room for Lockbit3.0
• Conti’s number of victims is dropping after May 2022. It does not mean that the individuals stopped their activities since the organization could have been divided into several groups after the events related to the Russian situation in April 2022 and come back under other names.
• The activity has decreased during the summer of 2022, except for Lockbit 3.0.

   Number of announced victims by the 15 most active RaaS groups for the past 10 months:

It must be taken into account that the data is based on the RaaS declaration of victims, the graphs are therefore an estimation of the reality.

CERT-W: FROM THE FRONT LINE

The First Responder Word

VULNERABILITY OF THE MONTH

Sophos Firewall

SEE YOU NEXT MONTH!!

Cet article CDT Watch – September 2022 est apparu en premier sur RiskInsight.

Last February, a Ukrainian researcher released more than 60,000 messages from inside conversations between different members of the group. Through these discussions, several revelations are made about their operations, allowing us to understand the RaaS ecosystem (Ransomware-as-a-Service).

Through this article, let’s take a look at how a Ransomware platform operates, then let’s question the organizational structure and the benefits generated by former CONTI group.

Ransomware platform ecosystem

The proliferation of articles on the Ransomware threat over the last few years gives the impression that the sector is flourishing. There are several players involved, and data theft amounts to hundreds of millions of dollars per year. For instance, CERT-Wavestone shared that about 60% of its incident responses in 2021 were for ransomware attacks.^[2]

As described in the figure, ransomware platforms do not work alone. It receives help from different service providers or other platforms and offers its services (in the form of ransomware) to different groups of attackers. Finally, the platform can also directly extract data from its victims: individuals, companies, states…

These platforms have fueled the growth of a RaaS economy. Approximately $5.2 billion of BTC transactions have been identified by the US Treasury with the payment of the most commonly reported ransomware platforms.^[3] This makes it a highly profitable business.

At the same time, it is also an activity where there is a significant number of established groups of players that seem to frequently appear and disappear and which generally last several months. Behind these multiple platforms usually hide the same individuals. If the CONTI franchise, supposed successor of the Ryuk^[4],has only survived 2 years, its former members still seem to be active.

In this fragmented and complex environment, it is difficult to retrieve consistent information on the functioning of the platforms. The internal discord that followed the war in Ukraine and the publication of the Conti Leaks allowed us to investigate the functioning of this secretive group before its dissolution.

Conti enterprise ?

On February 27th, 2022 we discover the underside of CONTI organization. The disclosures are made within a few days and soon reveal :

• Well-established recruitment processes, that can even go as far as advertising on legitimate recruitment sites

• A vertical organization with its own HR, financial and operational departments

• A salary policy established according to the different functions in the organization, including bonuses per position after a successful ransom (See. Full study)

This thoughtful and efficient organization seems to be one of the secrets to the profitability of the group.

A highly profitable business model

We have endeavored to recreate the cash flow statement of CONTI from scratch.

Each line of profit and cost is researched to be estimated as accurately as possible over a year.

According to The 2022 Crypto Crime Report^[5], in 2021, CONTI and its affiliates held at least $180M in bitcoins for ransom from addresses traceable on the blockchain. It is estimated that 70% is paid back to its affiliates, and the remaining $45M is the group’s revenue.

CONTI then has to deal with traditional expenses, which are common in traditional companies. The main ones are: money laundering, which represents up to 50% of the earnings, purchases of third party services and maintenance costs, and finally.

CONTI would therefore generate a net profit of $16M (if we assume that the group does not pay Russian taxes). The ROI of the group is estimated at about +163%, according to the same information.

Conclusion

The Conti Leaks enabled to better understand the organization and ecosystem of Ransomware-as-a-Service platforms through the study of a group. This work thus provides a solid foundation for the popularization of the RaaS threat.

About our method

All of the information presented in this study comes from Wavestone’s field observations as well as multiple sources – both public and private such as: the National Cyber Security Center, Check Point Research, Palo Alto Networks, Breachquest and Chainalysis.

Sources :

[1] « Conti ransomware shuts down operation, rebrands into smaller units », Bleeping Computer, May 2022

Conti ransomware shuts down operation, rebrands into smaller units (bleepingcomputer.com)

[2] « Cyberattacks in 2021: ransomwares, still threat n°1 », Risk Insight, October 2021

https://www.riskinsight-wavestone.com/2021/10/cyberattaques-en-france-le-ransomware-menace-numero-1/ 

[3] « US links $5.2 billion worth of Bitcoin transactions to ransomware », Bleeping Computer, October 2021

https://www.bleepingcomputer.com/news/security/us-links-52-billion-worth-of-bitcoin-transactions-to-ransomware/

[4] « Le rançongiciel Ryuk », ANSSI, September 2021

CERTFR-2020-CTI-011.pdf (ssi.gouv.fr)

[5] « THE 2022 CRYPTO CRIME REPORT », Chainalysis, Février 2022

Crypto-Crime-Report-2022.pdf (chainalysis.com)

Cet article Ransomware: Inside the former CONTI group est apparu en premier sur RiskInsight.

CERT-W: FROM THE FRONT LINE

The First Responder Word

For more information for vulnerability detection and remediation, contact Wavestone CERT-W!

Reading Of The Month

To learn more about the evolution of cybercrime, we recommend reading the Internet Organized Crime Threat Assessment 2021 of Europol. This report focuses on changes and developments of cybercrime threats during the last 12 months.

Internet Organized Crime Threat Assessment 2021, Europol

Cet article CDT Watch – December 2021 est apparu en premier sur RiskInsight.

Reminder of the state of the threat

ANSSI states in ÉTAT DE LA MENACE RANÇONGICIEL – À L’ENCONTRE DES ENTREPRISES ET INSTITUTIONS[1] published on 05/02/2020: «  Since 2018, ANSSI and its partners have observed that more and more cybercriminal groups with significant financial resources and technical skills favour the targeting of particular companies and institutions in their ransomware attacks. ».

Faced with this observation, it is more necessary than ever to secure information systems. This involves applying the fundamentals of security: applying patches, managing accounts and passwords, managing network segmentation etc. As a reminder, the application of these initial measures permits a significant reduction in the probability that an information system will be subject to a ransomware but can in no way guarantee that this will not happen.

Specificity of the industrial sector

However, even though new defensive solutions are continually being developed, the cost and complexity of deploying some of them ultimately make them little used. This is truer in an industrial environment, where their integration can be complex, as some systems are fixed in a functional configuration. Moreover, the budgets allocated to IT security in an industrial environment, although increasing in recent years, are still not sufficient for many sites.

Furthermore, an industrial information system shares a common base with a conventional information system and is therefore subject to the same attacks. Of course, attacks such as Stuxnet, Triton, or BlackEnergy (on a smaller scale) require additional skills. However, it is always worth remembering that the targets of interest for groups possessing this type of means are generally already subject to regulatory obligations (LPM in France, NIS directive etc.), which if respected, greatly limit the risks of a successful attack against them. However, these systems are not invulnerable, and must therefore also be prepared to respond to an attack.

Inevitable attack on industrial systems: how to minimise the impact and restart operations quickly?

It therefore appears that:

• Protecting oneself from the threat is often limited to the application of basic security measures if there is no regulatory obligation applicable to the target information system;
• Identifying the sources of threat and detecting an attack before it reaches its objective requires in most cases resources that are too important in relation to the budgets of current industrial information systems.

If the probability of an information system undergoing a successful cyber-attack, and more specifically a ransomware, is almost certain, the following question arises: “How can we prepare for a major cyber-attack, maintain critical activities in a degraded mode, while rapidly regaining confidence in the industrial information system? ».

The answer to this question is covered by the last two pillars of computer security according to the NIST framework: respond and recover. An attempt to answer this question is presented in this article.

Note: the first part of this article “How to respond to an attack before it is too late?” is not necessary to implement the recommendations detailed in the second part “How to recover after an attack if it could not be contained? ». Although the implementation of network filtering measures is highly recommended, it may be interesting for sites where the implementation of such filtering measures takes too long to implement, to start with the preparation part of the remediation of a cyber-attack, which is easier to implement.

How to respond to an attack before it is too late?

Involving industrial teams

Before talking about the measures that can be put in place to respond to a digital security incident, it may be interesting to remember that industrial staff are used to crisis management.

Indeed, many industries regularly organise crisis management exercises (fire, chemical risk, natural disasters, etc.). On many sensitive sites, procedures are therefore already available to respond to this type of incident, under the direction of a dedicated manager. In addition, autonomous physical protection is generally available: pressure relief valve, non-return valve, sprinkler etc., although these are sometimes replaced by connected instrumented safety systems.

The context is therefore appropriate for adding a new procedure in order to respond to a computer attack. This will generally consist of isolating the industrial information system from the outside via a procedure known as the “red button”. In order to draw up the associated procedure, the involvement of site personnel will be essential, particularly to ensure that the application is not more harmful than the attack itself.

A prerequisite for the implementation of the isolation posture: the control of its flows and the implementation of network partitioning/filtering.

It is necessary to measure the impacts generated using the “red button”. To do this, it is necessary to list the interconnections of the industrial site with other systems.

List the interconnections with other information systems.

It may be interesting to start by listing the flows between the industrial information system and the outside. First of all, it is necessary to define what this system contains. In a basic case, it includes the PLCs, the supervision, as well as the equipment necessary for the interconnection of the first two.

Other equipment can then be added: an Historian server, client stations for supervision, a NAS, etc. This network, later called an industrial network, is generally connected with other networks in order to share information with the equipment of the latter.

It is possible to mention:

• Exchanges with the company’s ERP (whether an MES – Manufacturing Execution System is present or not), generally located on the office network;
• Exchanges with partners: regulation of electricity, water and gas networks, etc.;
• Exchanges with service providers: weather, cloud solutions for energy optimisation, predictive maintenance, etc.

These flows, although useful to simplify operations, can generally be temporarily cut off or replaced by alternative means (telephone call to indicate production levels for example).

Moreover, each industrial site is different, and therefore manages these interconnections differently. It is common to see MPLS networks dedicated to industrial sites when the company owns several of them. In other cases, the office network will be used to federate them. It is also true for the connection needs between these industrial networks and the Internet, which sometimes pass first through the office network, or benefit from a direct output.

List its internal flows

After listing the interconnections between the industrial network and the outside, the internal flows remain to be listed. Most of these flows should be strictly necessary for the proper functioning of the industrial process, such as those between supervision and PLCs. Cutting off these connections would therefore require stopping the industrial process, or at least making it safe.

It may then be interesting to separate the equipment and associated flows into several zones:

• Supervision;
• Field network;
• Others (supervision client stations, historian server, etc.).

Setting up these zones allows the exposure of these components to be drastically reduced. Indeed, only the supervision should have access to the field networks, while the “Others” category should only have access to the supervision.

Other zones may be necessary to implement such as:

• An administration zone: which could also be used to program the PLCs according to the distribution of roles and responsibilities on site;
• A DMZ: which can accommodate a relay server so that equipment outside the industrial site does not connect directly to the supervision system to retrieve production data, etc.

Depending on the services offered (WSUS server, antivirus server, Terminal Server for remote access etc.) other zones can of course be added.

Evaluate the real need for these flows

After listing all these flows, it is interesting to identify the real need for each of them. For example, is it necessary to be able to access e-mails from a supervision server?

In order to limit the exposure of the industrial network to the outside, it could also be necessary to take some equipment out of it. For example, if a database accessed from the office network is fed by the supervision, but not useful to it, hosting it directly on the office network may prove simpler than trying to limit access.

Once the necessary flows have been clearly identified, the associated filtering rules must be configured in detail (source IP address, destination IP address, destination port). This work generally requires a significant human investment, mainly from the teams in charge of the industrial site, as well as a significant material cost to acquire security equipment. However, it is a prerequisite for setting up the fallback postures described below. In an ideal case, application filtering (level 7 of the OSI model) could also be implemented.

This work, although essential to the implementation of isolation postures, is also one of the fundamental actions to be carried out within the framework of securing an information system (industrial or not). Indeed, each flow cut off is a flow that does not need to be monitored, as well as one that is less exploitable by an attacker.

Preparing fallback postures

Complete isolation of all the equipment in an industrial information system is not always desirable, even in the event of an attack. After having listed these flows, it may be interesting not to set up a single isolation posture, but several fallback postures, allowing in some cases to continue working almost normally.

Preventive fallback posture: isolate the plant in the event of an attack on an external network

After identifying the flows between the industrial network and the outside, it is possible to create an associated fallback posture in order to deactivate them if necessary. The objective of this posture is to cut all interconnections of the industrial network with the outside in order to prevent any propagation of an attack. A proven solution is to group these flows on a few dedicated Ethernet ports. Thus, it is sufficient to indicate in the associated procedures to disconnect the associated cables to activate the fallback posture. This also avoids having to intervene on the configuration of firewalls in the event of a cyber security incident.

In addition, it is also necessary to define the cases in which this posture should be activated. If it can be activated without posing any problem to production, or adding too much work to the site staff, the question may arise as to whether these flows are necessary.

If this posture does have an impact on the site’s industrial activities, a good balance must be found between triggering it too early (as soon as the antivirus software on an office workstation raises an alert), or too late (after the first industrial workstations have been encrypted). This will also depend on the context of the company and its resources (dedicated or non-dedicated security monitoring team, etc.).

Specificity (distributed sites, non-autonomous sites, etc.)

If all flows with the outside do not have the same destination, it may also be interesting to define several specific fallback positions. Indeed, if the service provider in charge of managing the site’s cameras warns that he is undergoing a ransomware attack, it seems more optimal to disconnect only the flows between this service provider and the factory network, rather than all the flows, including those to the ERP.

In the case where the industrial process is distributed over several sites (production and distribution plant in particular), the activation of the preventive fallback posture should not cut off the flows between these different sites. Indeed, specific links should be dedicated to this. If this is not the case, use of the office network to ensure these connections, for example, a project to overhaul the industrial network is probably to be expected (deployment of a dedicated VRF, or a SDWAN network for example).

Finally, it is always good to remember that each factory is different, so a local study will have to be carried out on each one to understand its specificities.

Last resort fallback position: switch off the information system in the event of a proven attack on the plant

Finally, it may be interesting to prepare a last resort fallback posture. This should consist of isolating each VLAN (if defined, preferably with a local HMI per VLAN to ensure a degraded mode) or each piece of equipment (turn off the switches) in order to prevent the attacker from continuing his actions, which in the most advanced cases of attack, could directly target the site’s industrial process.

The objective is then to secure the site or ensure its essential services. The activation of this posture implies working without an information system and should only be applied in the event of proven compromise of at least one piece of equipment on the site, since it leads to the same immediate result as a ransomware, if not worse.

An upstream work with the operators will be necessary in order to list all the actions to be carried out when this posture is activated and to define degraded modes. Indeed, this will generally require the activation of on-call duty in order to manually perform certain tasks: checking the correct operation of equipment, especially on remote sites, use of local HMIs, etc. Moreover, some industrial processes are no longer manually controllable today, and will therefore have to be stopped since no degraded mode is available.

In order to estimate the impacts of activating such a posture, it may be interesting to look at the impacts listed in the event of fire or a general power failure. Moreover, only a real test of this posture can ensure its operational impacts.

How to recover after an attack if it has not been contained?

In some cases, the activation of fallback postures may not be sufficient to protect the entire industrial information system, especially if they are activated too late. It is then essential to be able to proceed with the reconstruction of all or part of the said system in a sufficiently short time to limit the associated impacts.

The main prerequisites for restoring an industrial information system are listed below.

What must be backed up to be able to restore its PLCs?

In order to be able to restart the factory, it is necessary in most cases to start restoring PLCs, which requires two main elements.

Having an up-to-date copy of your PLC programs

PLCs are spared in most current attacks, probably because targeting Windows workstations is enough for attackers to achieve their intended objectives. However, attacks are likely to be increasingly targeted, and most PLCs currently in use are not secure (unencrypted and unauthenticated communications, default passwords, administration functionality that cannot be deactivated, etc.).

It is therefore necessary to save these programs, which is already generally the case, particularly on the programming station (sometimes belonging to a service provider) used when the device is commissioned. It should be noted that these backups should be stored on at least one off-line medium, so that they are not encrypted in the same way as the workstation hosting them.

These observations remain valid even for the new generations of PLCs, which, although benefiting from a level of security that is far superior to that of their predecessors, are not invulnerable.

Save a means of downloading these programs to the PLCs

Many PLCs require dedicated software to be programmed. This is even the case if you just want to download an already written program. It is therefore advisable to have a copy of these programs.

In some cases, a programming station disconnected from the network and reserved for this purpose can be a solution. It should be noted, however, that maintaining such a station in a safe condition can quickly become complex. If this solution is selected, this station could also host the copy of the PLC programs. Keeping a second backup set off-line (external hard disk for example) would however be an additional security measure.

Furthermore, if new generations of PLCs are used, with the latest security features enabled, other elements should be backed up such as: PLC program passwords, certificates used for certain communications (or a means of regenerating them) etc.

These prerequisites are also valid for network equipment (firewalls, switches etc.).

What needs to be backed up to be able to restore essential computer hardware?

Identifying what is really needed

Restoring SCADA system, and associated client workstations, is generally equivalent to restoring a Windows system and associated programs. Several questions must be asked to identify the items to be backed up:

• What equipment is needed? An engineering workstation, a SCADA server, a few operator workstations?
• Is it possible to reinstall the SCADA system from scratch (new installations of Windows and the supervision software) and then deposit a backup of the SCADA configuration? Is this feasible in a sufficiently short time?
• Would not a complete copy of the SCADA server disk be simpler? It would indeed be sufficient to insert the saved disk to reboot.
• Are changes regularly made to the supervision software? If yes, is it necessary to back them all up? In this case, it seems complex to make a complete copy of the disk each time.

Backing up intelligently

In many cases, backups of Windows workstations are made in the same way as those of PLC programs, by copy/paste. It could then be interesting to look at automatic backup mechanisms. However, these are probably to be avoided for factories starting from scratch and not having enough budget to install them serenely. Indeed, implementing this type of solution in a secure manner is generally more complex than making a simple bit-by-bit copy of a hard disk.

Do not neglect documentation and training

However, it is not enough to have complete backups available. It is also necessary to draw up detailed operating procedures for restoring these backups. Indeed, if a crisis were to occur, the stress of the teams and the potential unavailability of some of the knowledge could lead to handling errors in the absence of documentation.

These procedures are not intended to enable a complete restoration of all systems, but at least to enable the essential elements previously identified to be restarted:

• An engineering workstation with the associated PLC programming software;
• A SCADA server;
• Two to three operator workstations;
• The plant’s essential PLCs.

In addition, it is generally recommended to have at least two sets of backups, one to be stored near the equipment concerned, the other to be stored on another physical site, with access limited to a limited number of people. It may be tempting to store an additional set of backups online, but it should be noted that in the event of a cyber-attack, and activation of fallback procedures, it is complex to download these backups and deposit them on the systems to be restored.

Finally, it is essential to test all these procedures to ensure that they are exhaustive. A test could, for example, be the opportunity to realise that the backup of the SCADA configuration does not include the licence key, or that the passwords configured when the complete disk was copied have since been modified without keeping the history.

Conclusion

Crisis management is an important component of the business for many industrial system operators. These same people are also the most experienced in their perimeter. However, they are generally not IT experts. Pragmatic measures, adapted to their context, will therefore be far more useful than a generic 200-page guide containing all the good practices to be applied to an information system.

As in development with the KISS principle (Keep it simple, stupid), fallback postures, as well as restoration procedures, should be kept simple to understand, and stupid to apply.

Furthermore, although the application of a strict network filtering policy can only be advised, it is not strictly necessary for the implementation of backup and recovery actions. Thus, even if the probability of a successful attack is increased, it will still be possible to restore critical systems.

Finally, it should be noted that more and more industrial processes are nowadays operating in a just-in-time mode. In this type of context, the preservation of the industrial system from an attack, or the ability to restore it quickly, would not be sufficient to maintain the level of production if the management of orders or distribution, for example, are unavailable. Cyber resilience must therefore be considered at the company level, and not only at the level of the industrial site.

Key elements

To respond to an attack before it is late, it is necessary:

• To involve the industrial teams (without which it is highly likely that the computer will survive the attack, but without the factory continuing to fulfil its primary mission);
• To control its flows and implement network partitioning/filtering in order to be able to set up fallback postures:
  • Preventive, in order to isolate the factory in the event of an attack on an external network without having too significant an impact on the industrial process;
  • As a last resort, in order to shut down the information system in the event of a proven attack on the factory before the attacker modifies the industrial process.
• To test these fallback postures, in order to ensure that their activation is not worse than the attack.

And in the case where the attack could not be contained, the following elements are generally necessary in order to recover from the said attack:

• Possess an up-to-date copy of your PLC programs;
• Save a means of downloading these programs to the PLCs;
• Have at least one copy of all critical backups on an off-line medium (external hard disk for example);
• Identify its essential computer equipment (in particular so as not to restore the history server before the supervision server, etc.);
• Backing up intelligently, sometimes a bit-by-bit copy of the hard disk is more efficient than an automatic copy on a dedicated server, generally encrypted at the same time as the system whose backups it hosts;
• Don’t neglect documentation and training (otherwise a forgotten license key, or someone on holiday could quickly sign the end of the restore…).

[1] www.cert.ssi.gouv.fr/uploads/CERTFR-2020-CTI-001.pdf

A new version of the threat assessment was published at the beginning of the year: https://www.cert.ssi.gouv.fr/uploads/CERTFR-2021-CTI-001.pdf

Cet article Cyber resilience in an industrial environment est apparu en premier sur RiskInsight.