How to improve operational resilience in a multi-vendor and multi-site world

How to improve operational resilience in a multi-vendor and multi-site world

Introduction to OT/ICS Security

Reviewing the OT cybersecurity threat landscape from 2021 to 2022, we have observed that since the outbreak of the COVID-19 crisis, the world’s reliance on networks and information systems has surged to unprecedented levels, with industries and services becoming increasingly interconnected. The COVID-19 crisis has demonstrated the necessity for the digital transformation of key global industries to be adequately prepared, particularly in enhancing the cyber resilience of critical infrastructure services (such as energy, transportation, chemicals, and critical manufacturing). The increased scope and variety of cyberattacks in recent years highlight the world’s need for a higher level of cyber resilience to protect crucial industries. We can summarize the threat landscape into the following five points:

1. Emerging Threats

In 2022, numerous RaaS with full ecosystems (such as Black Basta, Pandora, LockBit 3.0) emerged, adopting various extortion tactics like destroying data, holding data for ransom, selling data on the dark web, threatening customers or suppliers, and targeting industries like Smart Manufacturing, Energy, Food & Agriculture, and Healthcare & Public Health. Some ransomware used advanced techniques to prevent analysis, for example, requiring a pass parameter to parse the main program (such as Egregor, LockBit 3.0). This made it difficult for researchers to analyze the ransomware, stalling the deployment of countermeasures, thereby deepening the attack’s impact on organizations. They even employed fast encryption methods and hardening tactics to evade detection and prevent counterattacks. Moreover, hackers exploited vulnerabilities (like Log4j) and used legitimate Windows/Microsoft Defender tools to download malicious DLL files and encrypted Cobalt Strike payloads onto their targets’ networks.

Supply chain threats were another focal point. In 2022, records of cyberattacks on suppliers in key industries show that the Energy and Critical Manufacturing industries were among the most affected. Critical manufacturing accounted for 24% of the total. The direct impact of supply chain attacks caused business operation interruptions. For instance, in the first half of 2022, Toyota had to halt production at 14 auto factories due to cyberattacks on its suppliers of plastic parts and electronic components. Shell also suffered a loss in oil production because of a cyberattack on its logistics and storage supplier.

2. Geopolitical Tensions

According to a report by the European Union Agency for Cybersecurity (ENISA), the Russia-Ukraine conflict has led to a surge in radical hacking activities, with 128 government agencies in 42 supporting countries being targeted by state-sponsored hacker groups. Countries such as China, Iran, and North Korea have also increased their espionage efforts, with national hacker organizations setting their sights on nations like Southeast Asia, Japan, and Australia. As geopolitical tensions continue to escalate in Asia, these hacker groups have targeted countries with close ties to Taiwan, including EU member states like the Czech Republic and Poland. These attacks often exploit zero-day vulnerabilities or target OT networks, with a primary focus on critical infrastructure. Social engineering, disinformation, and data threats are also common attack methods used by national hackers.

3. Collateral Damage

In terms of attack surfaces, most OT attacks originate from IT incidents, which we refer to as collateral damage. This leads to operational impacts and data being held hostage, ultimately resulting in financial losses. The inherent vulnerabilities of OT cybersecurity, both internal and external, stem from factors such as supply chain attacks from new assets or insufficient protection within the environment. For instance, ransomware attacks on organizations like Colonial Pipeline and JBS Foods have drawn attention to the dangers IT attacks pose to OT systems. Though these attacks may not initially target OT systems, a compromise on IT systems can force OT teams to manually shut down operations for security reasons, leading to indirect impact on OT. According to our survey, a whopping 94% of the enterprises interviewed acknowledged the likelihood of IT security incidents impacting the OT environment. Considering how ransomware is capable of lateral movement, relying solely on either OT or IT systems is entirely insufficient. These incidents underscore the urgency of incorporating IT-OT fusion defenses in security strategies.

4. Incomplete Cybersecurity Architecture

In the past, the most common defense for OT/ICS was relying on “complete isolation” (air-gapping). As a result, the network architecture of OT/ICS was designed with little consideration for cybersecurity defense capabilities. This prevailing assumption leads to inadequate planning and deployment of cybersecurity countermeasures, such as the absence of security considerations for regional management in the OT/ICS network architecture or even a lack of detailed level segregation. Another concern is internal/supply chain threats. Mobile devices in air-gapped environments, along with lenient management policies, can potentially allow malicious programs to compromise OT/ICS environments or steal sensitive data. For instance, USB flash drives used for data transfer, laptops used for maintenance, and any equipment brought in by suppliers can all serve as perfect carriers for malware propagation.

Additionally, legacy operating systems are common security vulnerabilities. Typically, OT/ICS endpoints are the weakest links in OT/ICS cybersecurity, as many older OT/ICS endpoints perform critical operations or function as decision points in production lines. Key assets running on outdated systems do not receive software and firmware updates, leaving newly discovered vulnerabilities unpatched. For example, every Windows XP or Windows 7 system is an easy target for attacks.

5. Shortage of Cybersecurity Professionals

The last challenge lies in the difficulty of hiring qualified cybersecurity professionals from a limited talent pool. In the IT sector, talent shortage is already a significant issue, and it is even worse in the OT sector. Given the recent trend of digital transformation, coupled with the convergence of IT and OT, cybersecurity is not only a highly complex field but also constantly evolving due to leaps in technological innovation, creating a perpetually changing landscape. Currently, Germany’s manufacturing industry is grappling with the impact of cybersecurity talent shortages. According to our annual survey, 37% of German manufacturers face this issue. This indicates that establishing a dedicated OT/ICS cybersecurity training program or course within the company is a prudent way to prepare for potential future security risks.

The Potential Impact of Cyberattacks on OT Systems

In 2021, many industries fell victim to ransomware attacks, with the most notable case being the Colonial Pipeline ransomware attack. This incident became the largest scale cyber assault against oil infrastructure in US history, leading to the shutdown of fuel transportation pipelines. As a result, many airlines experienced jet fuel shortages, and numerous locations faced gas station fuel scarcity and skyrocketing prices, causing the public to frantically rush to purchase gasoline.

However, in 2022, industries faced a more diverse range of threat attacks, with software attacks, supply chain assaults, and strikes targeting critical infrastructure coming into focus. Coupled with geopolitical issues introducing more state-sponsored APT attacks, these factors have prompted some regions and governments to redouble efforts to implement cybersecurity regulations for critical infrastructure.

1. Future Risks: IT/OT Convergence and Industry 4.0 Associated Risks

The concept of IT/OT convergence aims to integrate physical (OT) equipment and devices into the digital (IT) realm. Although this idea has existed for many years, it didn’t truly take center stage in the industry until after 2020. According to a recent report by IoT Analytics, starting in 2020, approximately 50% of industrial assets in factories were connected to some form of local or remote data collection system. The COVID-19 pandemic, in particular, demonstrated how the Industrial Internet of Things (IIoT) could enhance organizational resilience even in the face of catastrophic events.

In manufacturing, for example, the Industrial Internet of Things, also known as Industry 4.0, is considered key to significantly reducing downtime, enabling new business models, and providing better customer experiences. With the rise of new IIoT architectures, traditional centralized SCADA and MES system communication methods have started to change. For instance, many sensors now employ IoT communication protocols like LoRaWAN, SigFox, or NB-IoT, connecting industrial sensors directly to the cloud. Moreover, industrial computer manufacturers have started to develop edge servers supporting software application platforms that link devices to the cloud, such as Advantech’s ADAM-3600 RTU, which supports Azure cloud connectivity. Some small and medium-sized enterprise factories even prefer using open-source devices and communication protocols, like Linux-based HMI and gateways, or gradually adopting OPC-UA protocol-supporting servers.

However, these trends may increase the attack surface for hackers. For example, according to ICS-CERT data from 2010 to 2021, the number of reported vulnerabilities has been increasing annually since the launch of the ICS-CERT reporting program, with a cumulative total of 4,436 vulnerabilities. Furthermore, the number of reported vulnerabilities in 2021 marked the highest annual increase ever, highlighting the growing arsenal of hidden weapons hackers can utilize in ICS environments.

2. OT Threat Vectors Differ From IT

In the realm of IT, the most common attacks involve social engineering or internal personnel oversights, which grant hackers network access privileges. Hackers then exploit these opportunities for privilege escalation and lateral movement, remaining undetected until they locate critical OT control systems. For example, DoppelPaymer infiltrated Foxconn’s Mexico factory using spear-phishing emails containing malicious links, or by employing attachments disguised as legitimate files to deceive unsuspecting victims into executing malicious code. This code, in turn, downloads even more potent malware (such as Emotet) onto the victim’s computer.

However, the most dangerous scenario in OT occurs when hackers penetrate a device supplier’s network and infect devices with malicious code before they are delivered to customers. Often, OT attacks transpire when hackers infiltrate the software development process of suppliers before the software is compiled, turning the supplier’s products into malware-laden software. In these cases, the story of cybersecurity takes a more sinister turn, as the unsuspecting organizations on the receiving end unwittingly introduce infected devices into their critical systems, leaving them exposed to devastating consequences.

3. OT Security Incidents Come From New Assets

New assets are where the majority of OT security incidents stem from, as they contain vulnerabilities, and they can be carrying malicious files by default. In the US, this is an issue for 54% of the sample size, whereas in Japan this happens 44% of the time. Germany is the outlier here, with 51% of their OT security incidents stemming from IT activities instead of from new assets. These challenges result in both financial losses and badly compromised productivity.

Improving Operational Resilience in a Multi-Vendor and Multi-Site World

Implementing Zero Trust Cybersecurity Defense in OT Through Asset Lifecycle Management

OT managers need to adopt approaches distinct from IT cybersecurity and adhere to the zero trust principle: never trust, always verify. Embracing a zero trust architecture for OT ensures that network defenses never assume trust by default, and continuously assess trustworthiness across the network. By employing automated methods, the zero trust framework can be realized in OT, spanning applications, device controls, and networks to optimize productivity. It is recommended that OT managers utilize an asset lifecycle management approach to deploy and implement a zero trust cybersecurity framework, which encompasses four critical stages of the asset lifecycle: onboarding, configuration, production, and maintenance:

1. Onboarding Stage:

Before assets are introduced to your factory or facility, suppliers should scan each asset and establish an OT/ICS health record, proving that the equipment is free of malicious software. This process is akin to international flight customs checks, where both parties involved in the transaction must independently verify the security of the equipment. Upon arrival at the factory or facility, each device must be considered “hostile” until it undergoes a threat scan and any potential exploitable vulnerabilities are documented, ensuring that the equipment does not contain malicious software or severe vulnerabilities.

TXOne Networks recommends using the Portable Inspector solution, which allows for endpoint security checks without the need for software installation. Through the use of portable scanning tools, automated scanning and system configuration checks can be performed without a network connection. The cybersecurity inspection tool can be used to ensure supply chain security before devices enter the facility.

1) Minimize the impact of antivirus software on machinery or avoid violating machine warranty terms.

2) Suitable for network-free environments, allowing offline virus and configuration checks even in air-gapped environments.

3) Quickly verify whether personnel and supplier electronic devices carry malicious software, then execute cleanup or isolation.

4) Record asset information collected during each scan and send it to a central management console for viewing and archiving.

5) In addition to scanning for malicious software during data transfers, AES-256 hardware encryption is employed to protect files, ensuring data integrity during transfers.

2. Configuration Stage:

Configuration stage encompasses the process of hardening assets to eliminate avenues of attack, which includes addressing cybersecurity vulnerabilities and shutting down non-essential services, such as applications, user privileges, user accounts, network ports, and other unneeded system functions. By hardening assets, technicians can minimize the chances of attackers accessing computers responsible for critical tasks and prevent the execution of malicious software. However, traditional antivirus software is not designed for industrial control environments. It requires constant internet connectivity for updates to its scanning engine and virus signatures. Moreover, file scanning demands significant computational and memory resources, often leading to excessive endpoint load and frequent false positives.

TXOne Networks suggests deploying the following measures for OT endpoint protection:

1) Application Trust List Technology: For older machines, built-in whitelist mechanisms can be used to protect endpoints from malicious software infections or unauthorized changes. A four-in-one lockdown mechanism can be implemented, including USB lockdown, data lockdown, operation lockdown, and configuration lockdown.

2) Next-Generation Antivirus Technology: For modern machines, a dual-engine approach can be adopted. The Advanced Threat Scan Engine technology scans for known attacks that can be addressed. In addition, when offline, a next-generation machine learning engine detects unknown threats, achieving defense against both known and unknown malicious software attacks. With a built-in ICS software database capable of identifying software from over a hundred industrial control vendors, threat scans avoid interfering with recognized ICS software, ensuring zero disruption and minimal false positives for the most lightweight operation.

3) Centralized Management Platform: A client-server architecture central control platform can manage endpoint protection software. Under the same central control, different main functions can be formulated, such as management rules, reporting mechanisms, user management, and scan settings.

3. Production Stage:

In regards to production, any issue that arises can immediately lead to economic losses. At this stage, network security becomes a new variable, requiring meticulous protection to ensure operational integrity and resilience. This means balancing the needs of both modern and legacy assets. Factory owners must be prepared to defend against a variety of cyber threats that hackers are eager to exploit through the network. Zero trust networking can be employed using network segmentation, optimized network access control, and enhanced intrusion detection and analysis to prevent or mitigate the impact of compromised assets from escalating into large-scale disasters. Simultaneously, it simplifies monitoring and makes it more difficult for hackers to gather information or move within the OT network.

TXOne Networks suggests deploying the following measures for OT network defense:

1) Network Segmentation Techniques: Network Segmentation can be further divided into Internal Segmentation and Micro-Segmentation. Internal Segmentation is suitable for large-scale scopes or regions, defined based on available technology, bandwidth, and communication protocols in use. Similarly, Micro-Segmentation refers to technical solutions that allow users to narrow down the protected scope or region to a finer scale, even down to a single asset, achieving OT network visibility and ICS protocol filtering of devices on the network, without altering the existing network architecture or disrupting current configurations.

2) OT-Aware Operational Intelligence: Supporting a variety of industrial control network communication protocols and offering in-depth analysis of L2-L7 network traffic, this measure enables communication protocol instruction editing and endpoint connection allowlist operations, thus creating network rule trustlists. Moreover, the OT network defense solution should uphold the principle of least privilege, allowing businesses to minimize the OT attack surface, restrict OT network attacks, enhance operational performance, and mitigate the impact of human error. By implementing fine-grained access control at different levels, businesses can strike a balance between availability and security, thereby safeguarding critical data and systems.

3) Virtual Patching Technology: This can be implemented through Host-based Intrusion Prevention Systems (IPS) or network IPS. Such devices feature specially designed network policies for packet filtering, specifically designed to defend against attacks exploiting known vulnerabilities, without forcing endpoints to undergo system updates. This means no need to restart the system or halt the production line.

4. Maintenance Stage:

Maintenance encompasses not only the repair of hardware components but also software configuration changes, system upgrades, and security updates. Typically, factories have a scheduled periodic routine for equipment maintenance operations. Currently, technicians need to synchronize assets with state-of-the-art cybersecurity protection to ensure that replaced hardware components, such as computing and storage components, are free from potential malware. Additionally, software changes must comply with the asset owner’s security configuration rules and minimize new software vulnerabilities. Consequently, during maintenance, production equipment managers inevitably need to perform repeated malware and vulnerability scans. It is also recommended to conduct additional malware scans when replacing a component of the equipment or when making software or configuration changes. Furthermore, suppose a device in the production facility requires a software update. In that case, vulnerability scans should be performed on the network-facing components of the production equipment to confirm compliance with the asset owner’s security configuration rules.

 

Since security inspection tasks are usually located on-site at factories, the environment is typically offline. Therefore, maintenance personnel must execute vulnerability scans in a non-disruptive, software-free, and network-independent manner, ensuring that the scanned original equipment software and configurations remain entirely undisturbed while verifying the cleanliness of the equipment.

TXOne Networks suggests deploying the following measures for OT maintenance protection:

1) Utilizing the Portable Inspector tool can verify the cleanliness of components and software within production equipment, providing centralized logs on a single management platform for archiving and reference. By swiftly scanning and recording results, production equipment can be returned to operation without delay.

2) Inspecting all equipment brought into the semiconductor factory by visitors, as well as the common offline asset checks within the production environment.

3) Integrating into industrial control SOPs for routine network security on external maintenance personnel’s computers.

5. Situational Awareness

In order to achieve robust OT cybersecurity, a solid understanding of operations is essential. Consequently, factory security teams require a clear and visible platform to manage the information security of numerous devices simultaneously and in real-time, enabling administrators to promptly detect and address attacks as they occur. Maintaining situational awareness of all assets, software configuration changes, system upgrades, and security updates is crucial.

By continuously monitoring routine schedules and incorporating TXOne’s AI, the complexity of each asset is learned, establishing protection baselines for factory units and deploying appropriate factory defense methods. This ensures smooth operation and ongoing operational continuity within the workplace. TXOne centralizes OT/ICS device-related cybersecurity logs into a single window for comprehensive situational awareness, or archives asset configuration information for managerial analysis and reference. Solutions available include:

1) StellarOne allows management from a single pane of glass with support for Syslog forwarding, indicators of compromise (IoC) integration, and centralized monitoring.

2) EdgeOne manages the policies of networking and endpoint security assets, ensuring operational integrity across distant sites. It allows administrators to modify OT protocol allowlists for asset interoperability and to conduct deep network analysis. It organizes alerts, assets, and incident events, permitting direct monitoring of the enterprise’s industrial control system security, in addition to providing insight into the shadow OT environment.

3) ElementOne’s collected asset information can be converted to the CSV format through the centralized management program as an asset inventory or sent to a SIEM or Rsyslog server for further asset management such as maintaining OT asset inventory or identifying impact levels, known vulnerabilities, and cyber risks.

Incorporating AI/ML technologies into the OT zero trust framework allows for the study of network packets and processes, facilitating the recognition of potential threats and anomalies. This innovative approach enhances the ability to proactively defend against cyberattacks, as well as adapt and respond to evolving threat landscapes. By embracing AI-driven zero trust architectures, organizations can significantly bolster their cybersecurity posture and better protect their critical OT environments.

Read the full article here.