Managing risks in the long term

Equipment hardening

In addition to secure architecture and administration tools, security levels for each item of equipment should be increased according to the strict necessity principle. A generic hardening guide can be created and then adapted to each of the technologies identified by the industrial IS mapping. This allows some of the vulnerabilities to be remedied at configuration and system levels.

Additional security can be provided by adding complementary solutions, such as:

• Antivirus software, which will cover industrial workstations against the most common viruses, whether connected to the network or not (although the latter will require manual updates);
• Implementing strict rules on local machine firewalls, which can be used to prevent communications, and therefore intrusions, on unused ports, and to filter the origin of flows according to the protocols used – which means attempted attacks can be more easily detected;
• Local administrator account-management solutions (for example, LAPS for Windows) finally make it possible to manage native administrator accounts on workstations in a central and individualized way.

However, sometimes it may no longer be possible to harden equipment due to obsolescence. In such cases, there is a need to work with the relevant business functions on obsolescence management of the equipment – its potential replacement and, as a last resort, options to isolate it from the rest of the IS. On obsolete workstations, configuration blockers can be used to ensure the installation and use of components is limited only to those that are strictly necessary.

It’s important to remember that, while industrial ISs have vulnerabilities, they are, above all, part of the company’s means of production. Dialog with the relevant teams is therefore essential in understanding how equipment is used – in order to resolve the vulnerabilities while limiting effects on the business as far as possible.

Security maintenance

Once equipment has been brought up to the right level of security, a plan will be needed to maintain this over time. A choice of options for managing security patches can be developed to meet the needs of the business (in terms of availability, integrity, etc.) and synchronized with the maintenance of the industrial equipment through:

1. Integration into standard operating processes; for example, an installation’s qualification/quality processes may require that equipment be up to date. The updating and administering of equipment can therefore take advantage of plant shutdowns, especially where recertification is needed.

2. Planning a “hot swap” update process in the event of a critical security breach and a procedure for the preventive isolation of production lines – until it’s possible to interrupt the production process;
3. The identification of redundant or peripheral equipment where interventions can be carried out on the basis of straightforward interaction with production managers.

To put in place these patching processes, the mapping carried out previously must have generated a precise equipment inventory, including:

• The identification of the equipment: type, location, and number of units;
• The industrial processes that each item of equipment is used for, and the associated criticality;
• The version of the operating system and/or firmware, and the tools and configurations deployed;
• The cybersecurity needs of supported processes;
• The availability of redundancy, data buffering, and cold spares;
• The required patching frequency and patching history.

But maintaining security levels isn’t simply about applying patches to equipment, it should also:

• Define the process for updating the security solutions installed on equipment isolated from the network;
• Install removable media cleaning solutions, given that these types of tool remain in widespread use on industrial sites. Here, the use of portable solutions allows such media to be analyzed while moving around the site;
• Ensure the safeguarding of equipment configurations and their integration into the DRP in order to guarantee that equipment can be restarted following an incident while still meeting availability needs;
• Set up monitoring of the industrial IAM[1] to ensure robust physical and logical access control. This can also be used to automate a number of time-consuming activities that are still sometimes done manually.

Detecting cybersecurity incidents

The measures set out above help reduce the likelihood of risks occurring and increase the availability of equipment, which benefits the business. Nevertheless, there will still be a need to prepare for the worst and to have in place the tools needed to detect an incident – to be able to remedy such events as quickly as possible and minimize interruption times.

Putting in place detection

The first step is to activate the IDPS[2] functions on networked equipment to ensure that a first stage of detection, and potentially automatic blocking, is in place.

The next step is to collect information by deploying a concentrator on site. The network equipment and server logs can then be sent to existing or dedicated SIEMs[3] where correlation and detection can take place. SOC[4] and CERT[5] teams can then carry out analysis and detection, and respond, if needed, to an incident, by working through standard scenarios.

Anticipating specific risks

However, detection based on standard scenarios may offer only limited value to the business functions. Considering the entirety of sources (PC, Linux, UNIX, etc.) and setting up dedicated industrial IS probes, capable of interfacing with the SCADA systems, can enhance the detection system. Such solutions, however, can be costly.

The key factor is to ensure a progressive and rapid increase in the maturity and value added by the SOC. Agile methods are a good fit here and involve the iterative application of the cycle described in the text box below.

Planning for remedial activities

Lastly, detecting an incident will only result in effective remediation if the business-function teams are involved. As with equipment updates, emergency stop procedures should be reviewed jointly with industrial IS users. A formal Incident Response Plan enables the actions for an industrial cyber-incident to be planned.

Dedicated industrial IS crisis-management exercises should also be carried out to ensure that teams are optimally prepared and to highlight any shortcomings.

Taking a progressive and participative approach guarantees an initiative’s success

The security maintenance of an industrial IS is a complicated undertaking that can only be successful if it is carried out in partnership with the business functions. A progressive and participative approach should be taken to work with them in each of the following areas:

• Understanding the industrial IS, by mapping and prioritizing the most critical elements;
• Mitigating the risks on the industrial IS, by implementing state-of-the-art secure network architecture and defining the administration processes – due to their criticality, safety ISs must be given particular attention;
• Ensuring an adequate level of safety, by hardening and ongoing security maintenance – in particular, this will involve discussions with equipment suppliers and manufacturers;
• Putting in place the tools needed to detect security incidents – these can have a bearing on production and define the response processes.

The actions above can’t always be carried out in parallel. Defining a clear roadmap will enable such actions to be prioritized. This will aid cost control and maximize the value added for the business functions.

Given that such significant undertakings are often driven centrally, the challenge is to engage the individual industrial sites (which may be spread across the world) to ensure security levels can be maintained in the long term. In general, we observe that companies take a two-stage approach:

1. A multiyear cybersecurity program (typically carried out over three years), with a budget of €10m-15m, aimed at:
   • Creating the industrial IS inventory
   • Raising the security levels of existing assets by putting in place protective measures, often involving separation and filtering, and remedying the most critical vulnerabilities – here, defining procedures is essential;
   • Putting in place an initial network of local cybersecurity coordinators;
2. Create an industrial cybersecurity team and its associated management structures that bring together:
   • A framework of key activities that local players will need to manage;
   • The participative construction of the tools that will help this network of local managers carry out their cybersecurity activities;
   • The development of approaches to manage the increase in security maturity levels and change (such as maturity matrices, site-level budget-modeling tools, the definition of steering indicators, central services that the sites can draw on, etc.).

Implementing the management processes can start immediately after the program and therefore benefit from the initial network of site-level cybersecurity coordinators put in place.

Once constructed, it becomes a question of energizing the initiative and steering progress on the sites and industrial ISs, in terms of both security and maturity levels.

Doing this typically involves:

• A network of local cybersecurity coordinators, of size 0.5 to 2 FTEs[6] per site, who are responsible for carrying out projects, implementing ongoing cybersecurity activities, continuous security improvements, and reporting;
• A central team of 3 to 10 FTEs, to provide overall steering and support local managers – especially in terms of expertise.

[1] IAM i.e. Identity and Access Management.

[2] IDPS i.e. Introduction Detection and Prevention Systems.

[3] SIEM i.e. Security Incident and Event Management.

[4] SOC i.e. Security Operation Center.

[5] CERT i.e. Computer Emergency Response Team.

[6] These figures can vary significantly depending on the size and number of local sites; they are the typical arrangements we observe in the large international organizations that Wavestone supports

Cet article Saga (3/3) – Feedback from the field and good practices for the protection and the security maintenance of industrial ISs est apparu en premier sur RiskInsight.

Administration – the nerve center of network architecture

Good administration of an IS is essential to guaranteeing its availability and security. When carrying out an IS security program, you must be clear about the objectives you want to achieve. The good practices we observe in the field include:

• Creating an administration network isolated from the production network with both central and local scope whose aim is to protect administration flows and avoid integrity losses on flows used to manage sensitive operations;
• Protecting the administrative equipment to prevent an attacker from controlling these critical elements directly;
• Standardizing, as far as possible, practices and equipment to facilitate the deployment of secure, or even centralized, administration architecture, and to maintain security levels over time. This can be achieved by pooling resources within a central, dedicated team.

To note: here, we are discussing only the administration of industrial IS infrastructure. Production PLCs, for example, are administered by the business functions in terms of configuration and will pass through the dedicated configuration and maintenance team, when updates are required.

The first step is to create the structure of the isolated and overarching administration network. This objective can be achieved by putting in place the following measures:

• To optimize and pool resources, and especially to assure the DRP[1], the administration network must be constructed around one or more datacenters.
• In order to reduce the risk of an attack propagating by using an infected site as a springboard, the WAN[2] network placed between the datacenter and the industrial installations can be configured as a hub and spoke[3] network, which ensures the separation of each installation.
• To guarantee the integrity and confidentiality of administrative flows, these must be isolated within a specific VRF[4] or VPN[5] administration network between the datacenter and each site.  Putting in place such a dedicated administration network requires, in particular, the use of telecoms and security equipment, as well as dedicated interfaces on the servers.
• For the most important sites, the risk of intrusion via the user LAN[6] can be reduced by setting up an administration LAN which is only accessible from the datacenter’s administration LAN. However, such architecture must provide a resilient solution in the event that the WAN is cut to allow sites to access it directly and also for equipment that simply cannot be maintained remotely.
• Companies with multiple sites can also use a standardized housing that embeds all the security functions required for the site to be interconnected. This facilitates configuration and security maintenance.

Diagram showing the interconnection of a site with or without a SCADA

The second step consists of connecting the administration tools and equipment to be administered to this network, while protecting it from compromise.

Diagram showing the interconnection of a standalone site

There may also be a variety of reasons to keep part of the IS fully disconnected. A disconnected IS removes the ISS risks, leaving only business risks. Disconnection also lowers the level of exposure and therefore the risk of intrusion. A risk analysis should be carried out to determine how to proceed. The associated infrastructure will need to be modified: moving from simple local administration to dedicated administration – which can be costly. These various network bricks, then, enable administrators to access the industrial equipment. However, they must also be given access to the necessary tools.

Administrator tools: how to meet needs while guaranteeing security

Because corporate and industrial ISs are generally managed separately, they each use their own tools – although these may be based on identical products.  This type of configuration meets several objectives. It:

• Assures access control on the administration interfaces, reducing the likelihood of appropriating a means of attack and the fraudulent use of the tools;
• Tracks administrator activity to reduce the potential impact of an attack, by providing a means of detection and response, and facilitating investigation following an event.

This requires the implementation of an administration chain.

Diagram showing the main functions involved in a chain of administration

To centralize access and maintain close control of authorizations, an administration bastion must be set up. Generic accounts are handled by the bastion and protected in its digital safe. This also ensures the traceability of activity and reduces the risk of theft from generic, privileged accounts. The bastion can also secure administration flows by performing protocol translation (for example, from Telnet[8] to SSH[9]).

Equipment, especially telecom equipment, whose security levels are sufficiently mature (including detailed management of rights, traceability, individual accounts, etc.) can be directly administered without passing through a bastion.

The establishment of a dedicated administration workstation, where the tools needed for corporate management will be housed, requires a process to be put in place for their installation. This will ensure the workstation can remain secure and that the list of tools being deployed on the IS can be documented.

Planning for external maintainers

Lastly, it’s essential that access by third-party maintainers is secure in order to limit the risks that arise from improper or unmanaged access, such as infection of the IS after the installation of an unauthorized tool, data loss triggered by a malicious third party, the unavailability of equipment, etc.

An external access point with strong authentication will be needed to confirm the identity of users. Such an access point allows maintainers to access a rebound server which is controlled and hardened by the customer, while also ensuring the traceability of activity. Here, more sophisticated customers deploy solutions that allow the third-party access to the IS for the duration of the intervention only – and then only once access has been approved internally.

The configuration and maintenance servers that are dedicated to the site and PLCs must be rigorously monitored to keep them up to date and secure, especially in terms of the tools deployed on them.

For more detailed information, note that there is an ANSSI[11]  working group dedicated to the cybersecurity of industrial systems. Its PIMSEC framework[12]  recommends a range of security requirements that can be incorporated into contracts with industrial IS service providers.

We now have knowledge of our equipment and the solutions to secure and manage it. However, cybersecurity issues evolve over time, so it is essential to guarantee a level of security over time and to deploy adequate means of detection. How can this be done? This will be the topic of our next article!

[1] Disaster Recovery Plan.

[2] WAN i.e. Wide Area Network.

[3] Hub and Spoke i.e. A network around the datacenter.

[4] Virtual Routing and Forwarding

[5] VPN i.e. Virtual Private Network.

[6] LAN i.e. Local Area Network.

[7] VLAN i.e. Virtual Local Area Network

[8] Telnet i.e. Terminal Network, Telecommunication Network, or Teletype Network.

[9] SSH i.e. Secure Shell

[10] RDP i.e. Remote Desktop Protocol

[11] ANSSI i.e. The French National Cybersecurity Agency.

[12] PIMSEC i.e. ANSSI’s framework for security requirements for industrial systems integrators and maintenance providers.

Cet article Saga (2/3) – Feedback from the field and good practices for the protection and the security maintenance of industrial ISs est apparu en premier sur RiskInsight.

Opening things up to corporate ISS is now a necessity… but it also carries risks

Historically, industrial ISs were not connected to corporate ISs, either because there was no need or as a way of limiting the risk of exposure. The majority of interventions were local, with work taking place directly on equipment, or remotely, using specific methods. The management of this work and the operations themselves were mostly local too.

Business functions’ changing needs and the optimization of production processes have brought with them new and less localized requirements (such as remote supervision, remote maintenance, the emergence of the IoT1, the standardization and rationalization of technologies and skills, cyber threats, etc.), which are designed to improve performance and facilitate operations. These challenges have led to a need to digitalize and interconnect industrial and corporate ISs.

Although this is now essential for a company’s business functions to operate effectively, our discussions with operational staff highlight the fact that such changes have also led to risks of intrusion and the propagation of threats between these interconnected ISs. These affect:

• Operations and quality – with potential shutdowns and modifications to production lines resulting in financial, reputational, and even people impacts;
• The security of facilities, where production equipment being seriously compromised can have impacts on both people and the environment.

Mitigating these intrusion and propagation risks and their consequences means implementing security measures in several different stages:

• Industrial IS mapping;
• Putting in place secure network architecture;
• The hardening and security maintenance of the various systems over time;
• And, lastly, putting in place the measures to detect incidents and respond to them.

Regulatory authorities have also been considering these risks. For the most sensitive installations, they are now mandating these types of measures and others too.

Interventions (such as patch management, account audits, integrity control, etc.), sometimes done remotely and often frequently, may now need to be carried out by teams more distant from site operations. These quickly come up against a traditional operating model designed to prioritize the continuity and integrity of operations, quality, hygiene and safety – while minimizing disruptions to production.

How can these measures be implemented without losing sight of the industrial IS’s core purpose – to operate a physical process in the way designed?

Mapping, a prerequisite for dealing with cybersecurity risks on industrial ISS

To assess the risks and control the potential impacts of implementing any new measures, the first step is the IS mapping of your industrial installations, which enables you to:

• Know the systems that need to be administered and kept up to date;
• Identify the users (operators, maintainers, etc.), and therefore those who need to be involved when a change takes place, to manage the operational impacts;
• Evaluate the potential impacts of new vulnerabilities and security breaches in terms of safety, operations, and quality.

Once the mapping process is underway, you will also need to develop formal procedures for updating the map. This means defining the update frequency, according to the level of criticality, and then actively managing the risks.

This is a substantial piece of work requiring dialog and close collaboration with automation and other engineers involved with the installation.

Mitigating risks on an industrial IS by putting in place security architecture

Security isn’t a new concept and it makes sense to follow the established principles for corporate IS architecture and security – adapting them to the particularities of industrial ISs:

• Reducing the risks of propagation and intrusion by clearly partitioning the industrial IS and restricting access to it;
• Securing the administration of the IS by putting in place dedicated administration architecture;
• Equipping administrators with appropriate tools that enable them to make interventions across the entirety of the industrial assets;
• Integrating from the start (as far as possible) interventions made by external maintainers.

These four principles form the cornerstones of securing industrial IS architecture.

Partitioning, the first step in reducing exposure

Corporate and industrial ISs have essentially different goals: one is designed to facilitate the operation of a business (by providing messaging, management systems, collaborative tools, etc.), while the other is used to operate physical processes. In theory, these should be separated, and only certain types of information should be allowed to flow between them. However, feedback from the field tells us that this is rarely the case.

As in any work on IS security, the strict necessity principle should be adopted to limit exposure to cyber threats. Any interconnection between an industrial and corporate IS should serve a specific purpose; for example:

• Sending production orders to SCADA[1];
• Transferring CAM[2] files to digitally controlled machines;
• Collecting production data to enable the control of operations.

An industrial IS must also be internally partitioned to reduce the risk of threat propagation. To do this, you can use the principle of zones and conduits described in the IEC 62443 standard.

In practice, this partitioning has to be carried out in several steps:

• The listing of relevant business activities according to their different levels of sensitivity;
• Grouping activities requiring the same security level into zones (with, potentially, a ”legacy” zone and associated sub-zones);
• Putting in place security rules for each zone according to their needs, as described in standard IEC 62443;
• Checking that the interconnections (conduits) between the different zones comply with security rules;
• Migrating the applications. Ensuring applications are compliant can be a long and difficult task, and it’s best to use a risk analysis to prioritize and manage the work, as well as documenting the nonconformities and associated remediation plans. In addition, the migration process itself may be complex, if you are to avoid an impact on operations.

The particularity of safety ISS

Safety ISs are industrial ISs that enable industrial production systems to be put into a safe state. Before the advent of today’s digital systems, such systems had long been used in mechanical, pneumatic, and electrical forms. The particular importance of ensuring their integrity is therefore well understood. A final partitioning step can be considered to achieve this. However, field observations often tell us that existing arrangements act as a brake that complicates the work. When done rigorously, such separation reduces the risks of propagation and enables distinct levels of security to be implemented for the production IS and safety IS according to their risk levels. However, a disadvantage is that doing this requires a dedicated SCADA system, which is both expensive and not operationally friendly.

Diagram of Industrial IS / Safety IS partitioning scheme

After having launched this process of identifying and partitioning industrial IS, it is time to deal with their administration. How to reconcile security, operational gain and availability of the production tool? We will tell you about it very soon.

[1] SCADA i.e. Supervisory Control And Data Acquisition system

[2] CAM i.e. Computer Aided Manufacturing

[3] DMZ i.e. Demilitarized Zone.

Cet article Saga (1/3) – Feedback from the field and good practices for the protection and the security maintenance of industrial ISs est apparu en premier sur RiskInsight.