At the end of June 2017, an image shocked the minds of the cyber security and business continuity world. An open space, filled with workstations, all displaying the same screen: the NotPetya ransomware message. Even today, 90% of the crises managed by Wavestone CERT are caused by ransomware [1]. How, then, is it possible to begin investigations, reconstruction or enable the business to continue working if all workstations stop functioning? What strategy should be developed to integrate the workstation component into continuity plans, which until now have mainly addressed it from the point of view of disasters affecting buildings? 

Define the needs 

To begin with, it’s important to define the cyber scenario you want to protect yourself against. Is it a “total blackout” scenario, where the entire IS is unavailable? Or a basic Windows ransomware scenario where some Windows servers and workstations are compromised, but network equipment and Linux bricks are still functioning?   

Next, and based on the scenarios selected, it is necessary to segment the populations according to their needs: it is not possible to provide for an infinite number of workstations in a given period, and you need to know where to allocate the first workstations that will be made available. For example, we can distinguish between business-critical teams, whose activity cannot be interrupted for more than 4 hours, and less critical business activities, for which activity can be interrupted for 3 days with acceptable impacts for the company in crisis mode. Similarly, the IT and Cyber teams to be mobilized in the very first hours of a crisis to conduct investigations and begin reconstruction.   

Another point to consider is the minimum business functionality required for the rebuilt workstations to be useful. Some business populations use thick clients on their workstations, which can be complex to install and maintain. Likewise, certain professions need to interact with third parties for their vital activities, via dedicated VPNs or an IP whitelist. It is therefore essential to clearly define how many people have these needs, and in what timeframe, to define the technical solutions that can be implemented.   

We won’t necessarily propose the same solution to IT investigation and reconstruction teams – who need access to the internal network – as to business teams, who may have degraded modes of operation outside the company’s information system (IS) for the first few days of a crisis.  

When all is said and done, we tend to distinguish two clearly differentiated phases in the strategy for providing workstations in the event of a ransomware crisis: 

• A first phase during the very first days of the crisis, for a limited population, which will generally rely on solutions with the least possible adherence to the nominal Information System, in order to ensure critical business activities;  
• A second phase when investigations have progressed, with a massive workstation rebuild using the company’s master workstation, which will have been hardened beforehand by drawing lessons from past investigations.  

Adapting the solution to your context  

Several parameters need to be taken into account when planning your workstation rebuild strategy. One solution may work for one company but be unsuitable for another.   

For example, numerous security and access control measures have been put in place in recent years concerning access to the internal workstation network. NAC (Network Access Control) is increasingly widespread, and in recent buildings, Ethernet sockets accessible to each desk are tending to disappear. Office 365 access is restricted via conditional access, and VPN (Virtual Private Network) gateway authentication is based on a certificate on the workstation. When all these constraints exist, a BYOD (Bring Your Own Device) strategy for the first few days of a crisis cannot be the answer – at least not on its own.   

Also, the way in which workstations are managed is a determining factor and does not necessarily mean that the same technical solutions can be implemented for reconstruction. Generally speaking, there are two main approaches:  

• One, a so-called “historical” approach, with fleet management solutions based on classic architecture such as Microsoft System Center Configuration Manager (SCCM), which is the most widespread solution today.   

• Alternatively, a more “modern” approach (i.e. Modern Management) with Cloud-based fleet management solutions such as Microsoft Intune, which has been gaining ground in recent years. 

Reconstruction methodology also needs to be anticipated. There are two possible methods: restoration and reinstallation. Restoration represents a return to a previous state of the environment (OS and/or applications and/or data) thanks to a backup. Reinstallation, as the name implies, means rebuilding the workstation from scratch, losing local documents.   

In the case of workstations, the number of documents stored locally is generally fewer and is therefore a less critical issue. Most documents are now stored on file servers (NAS or Sharepoint) for shared work, or in the user’s personal OneDrive. As a result, users will be more inclined to reinstall workstations from scratch, rather than take the risk of restoring the system to a previous state, where the ransomware may already have been present but not yet activated. Especially as recent ransomware attacks local restore points [2].   

Choosing the reconstruction methods best suited to your strategy 

There are several different ways of providing workstations, depending on the situation and the formalization of needs discussed above. Here is a list of the main solutions we have encountered in the field, and our opinion on the advantages and disadvantages of each solution. 

• Building up a stock of emergency PCs 

A method often applied in conventional emergency plans (for building/site loss scenarios), crisis PCs are placed in Ergotron-type containers, ready for use in the event of a disaster. They are connected to the local network via the Ergotron, and automatically receive updates. Another strategy may be to rely on IT departments’ rolling stock of workstations, or to keep decommissioned workstations as backup stock.  

Our opinion: While this approach is well-suited to resilience scenarios such as the loss of a building/site, it presents a risk in the face of ransomware, as these PCs would be compromised in the same way as others, since they would be accessible and visible on the local network. These PCs would then have to be managed “off-line”, requiring a higher level of MCO (maintenance in operational condition), since the PCs would have to be manually switched on and updated regularly. What’s more, having unused, dormant equipment raises the question of optimizing resources and carbon footprint. This solution should be considered for a restricted population with a very low acceptable downtime. In addition, for populations using thick clients, it is possible to save time by pre-installing them on these dormant workstations. 

• The use of unmanaged PCs, via BYOD (Bring Your Own Device) or the use of “consumer PCs” purchased in the event of a crisis  

This strategy is generally associated with a “Total IT Blackout” scenario, in which the entire information system is considered compromised, and work must be carried out without any link to it. In this case, unmanaged workstations are used, either personal or mobilized in the event of a crisis via a contract with a supplier.     

Our opinion: the functionalities of this solution are limited, as the workstation has no access to the company VPN, and if NAC is deployed, when visiting the site, the PC will not have access to internal resources that are still functional. It can, however, be considered in conjunction with crisis measures that have been planned in advance and will enable the PC’s functionality to be improved (emergency NAC shutdown; temporary modification of O365 Conditional Access with Internet access; storage of business-critical data in a crisis Vault outside the IS, so that work can continue). In most cases, this solution will be reserved mainly for the business community, and possibly for the IT staff in charge of rebuilding – by coupling it with a return-to-site strategy and a lifting of the NAC, enabling physical access to the internal network. This remains a solution that can be highly effective when well anticipated and combined with the crisis measures mentioned above. 

• Nominal existence of workstations under another OS 

In the event of an attack specifically targeting Windows environments (most encountered in the field), the affected computers can be replaced by the solution running on another OS.   

Our opinion: this solution implies an MCO (Maintaining Operational Conditions) of at least two technologies and does not guarantee that users who normally work under Windows will be able to work under Linux or MacOS (non-compatible thick clients, etc.). It is, however, an entirely feasible solution for very specific populations, such as investigation teams. These teams generally prefer to use specific distributions such as Kali Linux, and these are the people who need to have access to the IS in the first hours of a crisis. 

• Remastering workstations on benches   

In the event of a crisis, the teams go to the various sites with mastering benches with their compromised PCs to be remastered. Even in the largest companies, run remastering benches have limited rebuild capacity (a maximum of a few hundred workstations/day per site). To increase this capacity, additional crisis remastering benches can also be provided as part of a contract with an external supplier.   

Our opinion: the remastering method in nominal mode on a bench requires careful preparation to be effective in the event of a crisis, given the volume of substations to be rebuilt. A plan must be drawn up to organize the return of many people to the site at the same time (distribution by site, communication to users on time slots, etc.), based on the remastering capacity of the benches per physical site. 

• Remastering workstations via USB keys   

In the event of a crisis, USB sticks prepared in advance (or to be generated during the crisis using a predefined procedure) with a Windows OS image are used to reinstall a new OS on the machine. This can be a blank Windows OS, or a company-specific image.   

Our opinion: this is a tried-and-tested method for crisis situations, which can save a lot of time if it is anticipated. You need enough USB sticks, with a recent Windows OS image, and a method for quickly cloning the sticks. You also need to define a way of distributing these keys to users (either before the crisis – but this makes updating the keys more complex, and there is a risk of losing them – or during the crisis, by going to an IT kiosk, as with the benches). It is also necessary to be able to boot on external media. If this functionality is blocked in the BIOS, this method cannot work, or at least not without a procedure to lift this restriction. This method can be combined with the use of benches to maximize the number of workstations to be remastered in parallel on site (some of the PCs run on the benches, while others launch the process via USB key). Similarly, if the workstation bootstrap has been compromised, a USB key with a blank Windows can be combined with Intune remastering at a later stage. 

• The use of crisis VDI (Virtual Desktop Infrastructure)   

Users connect to a remote virtual desktop via a browser. This solution must necessarily be combined with another (BYOD, consumer PC purchased for the occasion, or other) as a PC is required to connect to the remote VDI. VDIs can offer more or less advanced functionalities, depending on their link with the company’s IS (access to the internal network, pre-installation of thick clients, etc.).  

Our opinion: This system enables rapidly operational work environments, while limiting the risk of data leakage, since it is possible to prohibit copy/paste from the VDI to the host workstation. What’s more, by relying on VDIs in the cloud, you can achieve a high level of scale-up potential (from 1 VDI to 200 active VDIs very quickly in the event of a crisis). The main risk remains that the more the VDI infrastructure is correlated with the company’s IS, the greater the likelihood that it too will be compromised by the attack. In this case, relying solely on this solution is a risky gamble. Conversely, a VDI that is completely uncorrelated with the IS will function, but will offer limited functionality without any access to uncompromised parts of the company’s IS. 

• Re-mastering from the cloud via Intune 

The master deployed on workstations is externalized to Intune, a SaaS service hosted in the Microsoft cloud. At start-up or after a factory reset, the workstation asks the user to enter his or her Microsoft email address, thus identifying the user as a member of the company. This triggers the automatic download and installation of the master, with no further intervention required. There is one important prerequisite, however: the fleet must be natively managed via Intune to be able to use these methods. 

Our opinion: This is one of the most effective methods, particularly as it is possible to modify the image (in the event of compromise via a vulnerable protocol/patching flaw), then remotely launch a massive remastering of the compromised workstations from within Intune. It is also possible to carry out this self-service remastering on the user’s side, but a prerequisite will then exist: possession of the workstation’s BitLocker recovery key (or other encryption technology if applicable), if the workstation’s hard disk is encrypted as part of the workstation protection measures deployed by the company. For reasons of practicality on the day of the crisis, mass remastering launched from the Intune console is therefore preferable, as it avoids the BitLocker constraint. To do this, however, administrators must be guaranteed access to Intune – and Intune itself must not be compromised. Last but not least, if the ransomware destroys the workstation’s bootstrap, it won’t be possible to remaster it with Intune alone, and you’ll need to add the installation of a blank Windows on the workstation as a prerequisite (via a USB key, for example).  

It should be noted that there are also a few exceptional crisis situations in which, due to limited response and management resources, some organizations may choose to allow employees to work in degraded mode on compromised machines for a set period, if they are still operational. This may be the case, for example, when only office files have been encrypted, when the malware is passive and does not communicate with a Command and Control system, and by removing Internet access from workstations to prevent any remote takeover. 

To sum up, what are the success factors for an office environment resilience strategy? 

There’s no such thing as a “magic” solution for every situation, and every solution meets the need to get a workstation up and running again, but the choice of the best solution depends on several parameters specific to each organization. To ensure an effective strategy, it is important to :  

• Segment the company’s different populations to prioritize the provision of workstations, and propose solutions adapted to the specific needs of each one. 
• Diversify and adapt solutions. Focusing on a single solution can prove dangerous if it fails. The aim is to have a toolbox of technical solutions, which the crisis unit can choose to activate or not, depending on the exact nature of the crisis encountered. 
• Test solutions: whatever solutions and strategies are implemented to rebuild workstations, they must always be accompanied by planned tests. A solution that is not used regularly is a solution that may not work in the event of a crisis. Whenever possible, therefore, the backup solution should be used on a day-to-day basis to remaster PCs, or if VDIs are involved, they should be used on a regular basis. If this is not possible, the solution should be integrated into a business and/or IT continuity test plan, so that it can be tested in real-life conditions at least once a year. 

The solutions most frequently used in the field include mass remastering on the bench, building up a stock of crisis workstations, using Cloud solutions such as Intune and virtual desktops such as VDI coupled with BYOD. But these solutions, taken one by one, may not be enough, because as mentioned in the principle of diversification, putting all your eggs in one basket can cause problems. We could, for example, imagine a crisis where access to the Intune console is impossible and/or the Intune image itself has been altered by the attack. In this case, having a fallback solution such as external VDI or remastering via USB key is essential.  

[1] https://fr.wavestone.com/fr/insight/cyberattaques-en-france-le-ransomware-menace-numero-1/  

[2] https://attack.mitre.org/techniques/T1490/  

Cet article Cyber Resilience: how to define the best strategy for digital workplace recovery  est apparu en premier sur RiskInsight.

Reminder of the state of the threat

ANSSI states in ÉTAT DE LA MENACE RANÇONGICIEL – À L’ENCONTRE DES ENTREPRISES ET INSTITUTIONS[1] published on 05/02/2020: «  Since 2018, ANSSI and its partners have observed that more and more cybercriminal groups with significant financial resources and technical skills favour the targeting of particular companies and institutions in their ransomware attacks. ».

Faced with this observation, it is more necessary than ever to secure information systems. This involves applying the fundamentals of security: applying patches, managing accounts and passwords, managing network segmentation etc. As a reminder, the application of these initial measures permits a significant reduction in the probability that an information system will be subject to a ransomware but can in no way guarantee that this will not happen.

Specificity of the industrial sector

However, even though new defensive solutions are continually being developed, the cost and complexity of deploying some of them ultimately make them little used. This is truer in an industrial environment, where their integration can be complex, as some systems are fixed in a functional configuration. Moreover, the budgets allocated to IT security in an industrial environment, although increasing in recent years, are still not sufficient for many sites.

Furthermore, an industrial information system shares a common base with a conventional information system and is therefore subject to the same attacks. Of course, attacks such as Stuxnet, Triton, or BlackEnergy (on a smaller scale) require additional skills. However, it is always worth remembering that the targets of interest for groups possessing this type of means are generally already subject to regulatory obligations (LPM in France, NIS directive etc.), which if respected, greatly limit the risks of a successful attack against them. However, these systems are not invulnerable, and must therefore also be prepared to respond to an attack.

Inevitable attack on industrial systems: how to minimise the impact and restart operations quickly?

It therefore appears that:

• Protecting oneself from the threat is often limited to the application of basic security measures if there is no regulatory obligation applicable to the target information system;
• Identifying the sources of threat and detecting an attack before it reaches its objective requires in most cases resources that are too important in relation to the budgets of current industrial information systems.

If the probability of an information system undergoing a successful cyber-attack, and more specifically a ransomware, is almost certain, the following question arises: “How can we prepare for a major cyber-attack, maintain critical activities in a degraded mode, while rapidly regaining confidence in the industrial information system? ».

The answer to this question is covered by the last two pillars of computer security according to the NIST framework: respond and recover. An attempt to answer this question is presented in this article.

Note: the first part of this article “How to respond to an attack before it is too late?” is not necessary to implement the recommendations detailed in the second part “How to recover after an attack if it could not be contained? ». Although the implementation of network filtering measures is highly recommended, it may be interesting for sites where the implementation of such filtering measures takes too long to implement, to start with the preparation part of the remediation of a cyber-attack, which is easier to implement.

How to respond to an attack before it is too late?

Involving industrial teams

Before talking about the measures that can be put in place to respond to a digital security incident, it may be interesting to remember that industrial staff are used to crisis management.

Indeed, many industries regularly organise crisis management exercises (fire, chemical risk, natural disasters, etc.). On many sensitive sites, procedures are therefore already available to respond to this type of incident, under the direction of a dedicated manager. In addition, autonomous physical protection is generally available: pressure relief valve, non-return valve, sprinkler etc., although these are sometimes replaced by connected instrumented safety systems.

The context is therefore appropriate for adding a new procedure in order to respond to a computer attack. This will generally consist of isolating the industrial information system from the outside via a procedure known as the “red button”. In order to draw up the associated procedure, the involvement of site personnel will be essential, particularly to ensure that the application is not more harmful than the attack itself.

A prerequisite for the implementation of the isolation posture: the control of its flows and the implementation of network partitioning/filtering.

It is necessary to measure the impacts generated using the “red button”. To do this, it is necessary to list the interconnections of the industrial site with other systems.

List the interconnections with other information systems.

It may be interesting to start by listing the flows between the industrial information system and the outside. First of all, it is necessary to define what this system contains. In a basic case, it includes the PLCs, the supervision, as well as the equipment necessary for the interconnection of the first two.

Other equipment can then be added: an Historian server, client stations for supervision, a NAS, etc. This network, later called an industrial network, is generally connected with other networks in order to share information with the equipment of the latter.

It is possible to mention:

• Exchanges with the company’s ERP (whether an MES – Manufacturing Execution System is present or not), generally located on the office network;
• Exchanges with partners: regulation of electricity, water and gas networks, etc.;
• Exchanges with service providers: weather, cloud solutions for energy optimisation, predictive maintenance, etc.

These flows, although useful to simplify operations, can generally be temporarily cut off or replaced by alternative means (telephone call to indicate production levels for example).

Moreover, each industrial site is different, and therefore manages these interconnections differently. It is common to see MPLS networks dedicated to industrial sites when the company owns several of them. In other cases, the office network will be used to federate them. It is also true for the connection needs between these industrial networks and the Internet, which sometimes pass first through the office network, or benefit from a direct output.

List its internal flows

After listing the interconnections between the industrial network and the outside, the internal flows remain to be listed. Most of these flows should be strictly necessary for the proper functioning of the industrial process, such as those between supervision and PLCs. Cutting off these connections would therefore require stopping the industrial process, or at least making it safe.

It may then be interesting to separate the equipment and associated flows into several zones:

• Supervision;
• Field network;
• Others (supervision client stations, historian server, etc.).

Setting up these zones allows the exposure of these components to be drastically reduced. Indeed, only the supervision should have access to the field networks, while the “Others” category should only have access to the supervision.

Other zones may be necessary to implement such as:

• An administration zone: which could also be used to program the PLCs according to the distribution of roles and responsibilities on site;
• A DMZ: which can accommodate a relay server so that equipment outside the industrial site does not connect directly to the supervision system to retrieve production data, etc.

Depending on the services offered (WSUS server, antivirus server, Terminal Server for remote access etc.) other zones can of course be added.

Evaluate the real need for these flows

After listing all these flows, it is interesting to identify the real need for each of them. For example, is it necessary to be able to access e-mails from a supervision server?

In order to limit the exposure of the industrial network to the outside, it could also be necessary to take some equipment out of it. For example, if a database accessed from the office network is fed by the supervision, but not useful to it, hosting it directly on the office network may prove simpler than trying to limit access.

Once the necessary flows have been clearly identified, the associated filtering rules must be configured in detail (source IP address, destination IP address, destination port). This work generally requires a significant human investment, mainly from the teams in charge of the industrial site, as well as a significant material cost to acquire security equipment. However, it is a prerequisite for setting up the fallback postures described below. In an ideal case, application filtering (level 7 of the OSI model) could also be implemented.

This work, although essential to the implementation of isolation postures, is also one of the fundamental actions to be carried out within the framework of securing an information system (industrial or not). Indeed, each flow cut off is a flow that does not need to be monitored, as well as one that is less exploitable by an attacker.

Preparing fallback postures

Complete isolation of all the equipment in an industrial information system is not always desirable, even in the event of an attack. After having listed these flows, it may be interesting not to set up a single isolation posture, but several fallback postures, allowing in some cases to continue working almost normally.

Preventive fallback posture: isolate the plant in the event of an attack on an external network

After identifying the flows between the industrial network and the outside, it is possible to create an associated fallback posture in order to deactivate them if necessary. The objective of this posture is to cut all interconnections of the industrial network with the outside in order to prevent any propagation of an attack. A proven solution is to group these flows on a few dedicated Ethernet ports. Thus, it is sufficient to indicate in the associated procedures to disconnect the associated cables to activate the fallback posture. This also avoids having to intervene on the configuration of firewalls in the event of a cyber security incident.

In addition, it is also necessary to define the cases in which this posture should be activated. If it can be activated without posing any problem to production, or adding too much work to the site staff, the question may arise as to whether these flows are necessary.

If this posture does have an impact on the site’s industrial activities, a good balance must be found between triggering it too early (as soon as the antivirus software on an office workstation raises an alert), or too late (after the first industrial workstations have been encrypted). This will also depend on the context of the company and its resources (dedicated or non-dedicated security monitoring team, etc.).

Specificity (distributed sites, non-autonomous sites, etc.)

If all flows with the outside do not have the same destination, it may also be interesting to define several specific fallback positions. Indeed, if the service provider in charge of managing the site’s cameras warns that he is undergoing a ransomware attack, it seems more optimal to disconnect only the flows between this service provider and the factory network, rather than all the flows, including those to the ERP.

In the case where the industrial process is distributed over several sites (production and distribution plant in particular), the activation of the preventive fallback posture should not cut off the flows between these different sites. Indeed, specific links should be dedicated to this. If this is not the case, use of the office network to ensure these connections, for example, a project to overhaul the industrial network is probably to be expected (deployment of a dedicated VRF, or a SDWAN network for example).

Finally, it is always good to remember that each factory is different, so a local study will have to be carried out on each one to understand its specificities.

Last resort fallback position: switch off the information system in the event of a proven attack on the plant

Finally, it may be interesting to prepare a last resort fallback posture. This should consist of isolating each VLAN (if defined, preferably with a local HMI per VLAN to ensure a degraded mode) or each piece of equipment (turn off the switches) in order to prevent the attacker from continuing his actions, which in the most advanced cases of attack, could directly target the site’s industrial process.

The objective is then to secure the site or ensure its essential services. The activation of this posture implies working without an information system and should only be applied in the event of proven compromise of at least one piece of equipment on the site, since it leads to the same immediate result as a ransomware, if not worse.

An upstream work with the operators will be necessary in order to list all the actions to be carried out when this posture is activated and to define degraded modes. Indeed, this will generally require the activation of on-call duty in order to manually perform certain tasks: checking the correct operation of equipment, especially on remote sites, use of local HMIs, etc. Moreover, some industrial processes are no longer manually controllable today, and will therefore have to be stopped since no degraded mode is available.

In order to estimate the impacts of activating such a posture, it may be interesting to look at the impacts listed in the event of fire or a general power failure. Moreover, only a real test of this posture can ensure its operational impacts.

How to recover after an attack if it has not been contained?

In some cases, the activation of fallback postures may not be sufficient to protect the entire industrial information system, especially if they are activated too late. It is then essential to be able to proceed with the reconstruction of all or part of the said system in a sufficiently short time to limit the associated impacts.

The main prerequisites for restoring an industrial information system are listed below.

What must be backed up to be able to restore its PLCs?

In order to be able to restart the factory, it is necessary in most cases to start restoring PLCs, which requires two main elements.

Having an up-to-date copy of your PLC programs

PLCs are spared in most current attacks, probably because targeting Windows workstations is enough for attackers to achieve their intended objectives. However, attacks are likely to be increasingly targeted, and most PLCs currently in use are not secure (unencrypted and unauthenticated communications, default passwords, administration functionality that cannot be deactivated, etc.).

It is therefore necessary to save these programs, which is already generally the case, particularly on the programming station (sometimes belonging to a service provider) used when the device is commissioned. It should be noted that these backups should be stored on at least one off-line medium, so that they are not encrypted in the same way as the workstation hosting them.

These observations remain valid even for the new generations of PLCs, which, although benefiting from a level of security that is far superior to that of their predecessors, are not invulnerable.

Save a means of downloading these programs to the PLCs

Many PLCs require dedicated software to be programmed. This is even the case if you just want to download an already written program. It is therefore advisable to have a copy of these programs.

In some cases, a programming station disconnected from the network and reserved for this purpose can be a solution. It should be noted, however, that maintaining such a station in a safe condition can quickly become complex. If this solution is selected, this station could also host the copy of the PLC programs. Keeping a second backup set off-line (external hard disk for example) would however be an additional security measure.

Furthermore, if new generations of PLCs are used, with the latest security features enabled, other elements should be backed up such as: PLC program passwords, certificates used for certain communications (or a means of regenerating them) etc.

These prerequisites are also valid for network equipment (firewalls, switches etc.).

What needs to be backed up to be able to restore essential computer hardware?

Identifying what is really needed

Restoring SCADA system, and associated client workstations, is generally equivalent to restoring a Windows system and associated programs. Several questions must be asked to identify the items to be backed up:

• What equipment is needed? An engineering workstation, a SCADA server, a few operator workstations?
• Is it possible to reinstall the SCADA system from scratch (new installations of Windows and the supervision software) and then deposit a backup of the SCADA configuration? Is this feasible in a sufficiently short time?
• Would not a complete copy of the SCADA server disk be simpler? It would indeed be sufficient to insert the saved disk to reboot.
• Are changes regularly made to the supervision software? If yes, is it necessary to back them all up? In this case, it seems complex to make a complete copy of the disk each time.

Backing up intelligently

In many cases, backups of Windows workstations are made in the same way as those of PLC programs, by copy/paste. It could then be interesting to look at automatic backup mechanisms. However, these are probably to be avoided for factories starting from scratch and not having enough budget to install them serenely. Indeed, implementing this type of solution in a secure manner is generally more complex than making a simple bit-by-bit copy of a hard disk.

Do not neglect documentation and training

However, it is not enough to have complete backups available. It is also necessary to draw up detailed operating procedures for restoring these backups. Indeed, if a crisis were to occur, the stress of the teams and the potential unavailability of some of the knowledge could lead to handling errors in the absence of documentation.

These procedures are not intended to enable a complete restoration of all systems, but at least to enable the essential elements previously identified to be restarted:

• An engineering workstation with the associated PLC programming software;
• A SCADA server;
• Two to three operator workstations;
• The plant’s essential PLCs.

In addition, it is generally recommended to have at least two sets of backups, one to be stored near the equipment concerned, the other to be stored on another physical site, with access limited to a limited number of people. It may be tempting to store an additional set of backups online, but it should be noted that in the event of a cyber-attack, and activation of fallback procedures, it is complex to download these backups and deposit them on the systems to be restored.

Finally, it is essential to test all these procedures to ensure that they are exhaustive. A test could, for example, be the opportunity to realise that the backup of the SCADA configuration does not include the licence key, or that the passwords configured when the complete disk was copied have since been modified without keeping the history.

Conclusion

Crisis management is an important component of the business for many industrial system operators. These same people are also the most experienced in their perimeter. However, they are generally not IT experts. Pragmatic measures, adapted to their context, will therefore be far more useful than a generic 200-page guide containing all the good practices to be applied to an information system.

As in development with the KISS principle (Keep it simple, stupid), fallback postures, as well as restoration procedures, should be kept simple to understand, and stupid to apply.

Furthermore, although the application of a strict network filtering policy can only be advised, it is not strictly necessary for the implementation of backup and recovery actions. Thus, even if the probability of a successful attack is increased, it will still be possible to restore critical systems.

Finally, it should be noted that more and more industrial processes are nowadays operating in a just-in-time mode. In this type of context, the preservation of the industrial system from an attack, or the ability to restore it quickly, would not be sufficient to maintain the level of production if the management of orders or distribution, for example, are unavailable. Cyber resilience must therefore be considered at the company level, and not only at the level of the industrial site.

Key elements

To respond to an attack before it is late, it is necessary:

• To involve the industrial teams (without which it is highly likely that the computer will survive the attack, but without the factory continuing to fulfil its primary mission);
• To control its flows and implement network partitioning/filtering in order to be able to set up fallback postures:
  • Preventive, in order to isolate the factory in the event of an attack on an external network without having too significant an impact on the industrial process;
  • As a last resort, in order to shut down the information system in the event of a proven attack on the factory before the attacker modifies the industrial process.
• To test these fallback postures, in order to ensure that their activation is not worse than the attack.

And in the case where the attack could not be contained, the following elements are generally necessary in order to recover from the said attack:

• Possess an up-to-date copy of your PLC programs;
• Save a means of downloading these programs to the PLCs;
• Have at least one copy of all critical backups on an off-line medium (external hard disk for example);
• Identify its essential computer equipment (in particular so as not to restore the history server before the supervision server, etc.);
• Backing up intelligently, sometimes a bit-by-bit copy of the hard disk is more efficient than an automatic copy on a dedicated server, generally encrypted at the same time as the system whose backups it hosts;
• Don’t neglect documentation and training (otherwise a forgotten license key, or someone on holiday could quickly sign the end of the restore…).

[1] www.cert.ssi.gouv.fr/uploads/CERTFR-2020-CTI-001.pdf

A new version of the threat assessment was published at the beginning of the year: https://www.cert.ssi.gouv.fr/uploads/CERTFR-2021-CTI-001.pdf

Cet article Cyber resilience in an industrial environment est apparu en premier sur RiskInsight.