DELIVERABLE D6.3 Integration and validation of Industrie 4.0 …secureiot.eu/D6.3.pdf ·...

This document is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 779899. It is the property of the SecureIoT consortium and shall not be distributed or reproduced without the formal approval of the SecureIoT Management Committee. The content of this report reflects only the authors’ view. The Innovation and Networks Executive Agency (INEA) is not responsible for any use that may be made of the information it contains.

Project Acronym: SecureIoT

Grant Agreement number: 779899 (H2020-IoT03-2017 - RIA)

Project Full Title: Predictive Security for IoT Platforms and Networks of Smart

Objects

DELIVERABLE D6.3 – Integration and validation

of Industrie 4.0 usage scenarios. First version

Deliverable Number D6.3 Deliverable Name Integration and validation of Industrie 4.0

Usage Scenarios. First version Dissemination level Public

Type of Document Demonstrator

Contractual date of delivery M15

Deliverable Leader FUJITSU

Status & version Final – v1.00

WP / Task responsible WP6 / Task 6.2 Multi-Vendor Industrie 4.0 Use Cases

Keywords: Usage scenarios, integration, validation, Industrie 4.0, Industrial

IoT, security, safety, privacy

Abstract (few lines): Prototype implementation of the SecureIoT-enabled Industrie

4.0 use cases based on the task T6.2.

Ref. Ares(2019)2312349 - 01/04/2019

Page | 2

Project Title: SecureIoT Contract No. 779899 Project Coordinator: INTRASOFT International S.A.

D6.3 – Integration and validation of Industrie 4.0 usage scenarios. First version ,

Version: v1.00 – Final, Date 01/04/2019

Deliverable Leader: Jürgen Neises (FUJITSU)

Contributors:

Hendrik Eikerling (ITSOWL), David Schubert (ISOWL), Arjen

Lapidaire (P@SS), Mike Gligoor (P@SS), Peter Rus (P@SS),

Thomas Walloschke (FUJITSU), George Moldowan (SIEMENS)

Reviewers: Sofianna Menesidou (UBI), Stylianos Georgoulas (INTRASOFT)

Approved by: Stylianos Georgoulas (INTRASOFT)

Page | 3




Executive Summary This document describes the actions taken to validate the services developed by the SecureIoT

project within a multi-vendor Industry 4.0 usage scenario. The first part of the document

condenses the usage scenario and its technical architecture. Since it is not feasible to use an

operational production plant as a demonstrator, the validation takes place within a partially

simulated environment. This environment is capable of interfacing with the SecureIoT services

through monitoring probes that deliver application specific and general IT system data to the

services. The probes are described in detail in this document. Additionally, this document

contains a plan for the future activities to be executed until M18 of the project.

Page | 4




Document History

Version Date Contributor(s) Description

0.1 11/03/2019 Arjen Lapidaire,

Jürgen Neises Initial Use case description and ToC

0.11 13/03/2019 Jürgen Neises Update structure of chapter 3 and 4

0.13,0.14,0.15 15/03/2019 Hendrik Eikerling Update Probe Description, Use Case

Description from D6.1

0.16 18/03/2019 Jürgen Neises Update of contents, contributions,

ToC

0.17 20/03/2019 Hendrik Eikerling

David Schubert

(Ab)use-cases

Validation

SeCaas

0.18,0.19 21/03/2019 Hendrik Eikerling Validation Chapter, merging

0.20 - 0.22 21/03/2019 David Schubert

All contributors

Merged and fixed layout

Removed “Validation scenarios”

chapter and renamed corresponding

section in chapter 1

Renamed section 2.1

Aligned the text in 3.2 to fit WP4

0.23 – 0.27 21/03/2019 All contributors Merging of contributions

0.28, 0.29 26/03/2019

Arjen Ladpidaire

David Schubert

Jürgen Neises

Merging of contributions

Consistency checks

0.30 26/03/2019 Jürgen Neises Prefinal draft

031-0.39 28/03/2019

Mike Gligoor

Thomas Walloschke

Jürgen Neises

CMDB, Validation, Document

cléansing, adaptaion to 2nd review

1.00 29/03

Hendrik Eikerling

George Moldovan

Jürgen Neises

Thomas Walloschke

Adaptation to 2nd review

Page | 5




Table of Contents Executive Summary ......................................................................................................................... 3

Definitions, Acronyms and Abbreviations ...................................................................................... 9

1 Introduction ........................................................................................................................... 11

1.1 Overall use case scenario .............................................................................................. 11

1.2 Injection molding .......................................................................................................... 13

1.2.1 Use Case .................................................................................................................... 14

1.2.2 Abuse cases and validation scenarios ....................................................................... 15

2 Scenario implementation ...................................................................................................... 17

2.1 Replica architecture description ................................................................................... 17

2.1.1 Overall Network architecture ................................................................................... 17

2.1.2 Level 0 – Machine Level ............................................................................................ 18

2.1.3 Modelling the Molding Process ................................................................................ 20

2.1.4 Level 1 – Field Level .................................................................................................. 22

2.1.5 Level 2 – Realtime Level / Control Network ............................................................. 22

2.1.6 Level 3 – Site Manufacturing / Process Level ........................................................... 23

2.1.7 DMZ ........................................................................................................................... 24

2.1.8 Level 4 – Site Business Planning ............................................................................... 24

2.1.9 CMDB Data Structure ................................................................................................ 25

2.2 Digital Twin ................................................................................................................... 31

2.2.1 SIEMENS high-overview of a Digital Twin ................................................................. 31

2.2.2 Architecture and Interaction with MindSphere ....................................................... 32

2.2.3 Roadmap for the first release of the Digital Twin MindApp..................................... 34

2.3 IoT-Platform Integration – Mindsphere ........................................................................ 34

2.4 IoT-Platform Integration - Fujitsu Colmina ................................................................... 37

3 Integration of SecureIoT components ................................................................................... 38

3.1 Integration of WP3 Interfacing Data collection and Collaboration .............................. 38

3.1.1 Monitoring the Machine Operations – Filebeat ....................................................... 39

3.1.2 Monitoring the Factory Network - Packetbeat ......................................................... 39

3.1.3 Monitoring the Process Level - Metricbeat .............................................................. 39

3.1.4 Monitoring Access Control - Auditbeat .................................................................... 39

Page | 6




3.2 Integration of WP5 SecureIoT Services Implementation and Integration (SECaaS) .... 40

3.2.1 Risk assessment and mitigation services .................................................................. 40

3.2.2 Compliance auditing service ..................................................................................... 41

3.2.3 Programming support services ................................................................................. 41

3.2.4 IoT security knowledge-base .................................................................................... 42

4 Implementation and Validation Plan until M18 .................................................................... 43

References .................................................................................................................................... 44

Page | 7




Table of Figures FIGURE 1.1.1 ICS - RECOMMENDED NETWORK ARCHITECTURE* .............................................................................................. 12 FIGURE 1.1.2 HIGH LEVEL INDUSTRIAL NETWORK ARCHITECTURE .............................................................................................. 13 FIGURE 1.2.1 GENERIC IMAGE OF INJECTION MOLDING MACHINE ............................................................................................. 14 FIGURE 1.2.2: LOGICAL VIEW - USE CASE 1 ........................................................................................................................... 14 FIGURE 1.2.3 NORMAL INJECTION PRESSURE ........................................................................................................................ 16 FIGURE 1.2.4 ANOMALOUS INJECTION PRESSURE IN THE CASE OF A COLD SLUG ........................................................................... 16 FIGURE 2.1.1 REPLICA OVERALL NETWORK ARCHITECTURE ....................................................................................................... 18 FIGURE 2.1.2 MOULDING MACHINE SCHEMATIC MODEL .......................................................................................................... 19 FIGURE 2.1.3 GENERAL MOULDING PROCESS ........................................................................................................................ 20 FIGURE 2.1.4 MOULDING PROCESS MOULDING STATES ............................................................................................................ 21 FIGURE 2.1.5: MOULDING PROCESS MOULDING RELEASE STATES ............................................................................................... 22 FIGURE 2.1.6 THE IMPLEMENTATION ON THE FUJITSU INTELLIEDGE DEVICE ................................................................................. 23 FIGURE 2.1.7 INVENTORY INTERFACE FOR DISTRIBUTED ASSETS ................................................................................................ 25 FIGURE 2.1.8 INVENTORY INTERFACE FOR AUTONOMOUS ASSETS ............................................................................................. 26 FIGURE 2.2.1 DIGITAL TWIN IN SIEMENS MINDSPHERE ........................................................................................................... 32 FIGURE 2.2.2 MINDSPHERE TWIN MINDAPP ........................................................................................................................ 34 FIGURE 2.3.1 TWIN MINDAPP INTEGRATION INTO THE INDUSTRIE 4.0 SCENARIO ......................................................................... 35 FIGURE 2.4.1 COLMINA ARCHITECTURE ............................................................................................................................. 37 FIGURE 3.1.1 PROBE PLACEMENT ....................................................................................................................................... 38

Page | 8




List of Tables TABLE 1 REST CALL SAMPLES TO THE SECUREIOT TWIN MINDAPP ............................................................................................. 35 TABLE 2 PLANNED IMPLEMENTATION STEPS UNTIL M18 .......................................................................................................... 43

Page | 9




Definitions, Acronyms and Abbreviations Acronym Title

ABAC Attribute Based Access Control

API Application Programming Interface

BT Bluetooth

CAD Computer-Aided Design

CAS Compliance Audit Service

CI Configuration Item

CMDB Configuration Management Database

CO Confidential, only for members of the consortium (including Commission Services)

CPE Common Platform Enumeration

CVE Common Vulnerabilities and Exposures

D Demonstrator

DMZ De-Militarised Zone

DNS Distributed Name Service

DoA Description of Action

Dx Deliverable (where x defines the deliverable identification number e.g. D1.1.1)

HMD Head Mounted Display

HMI Human Machine Interface

HTTP(S) Hypertext Transfer Protocol (Secure)

IAM Identity and Access Management

ICS Industrial Control System

IoT Internet of Things

MES Manufacturing Execution System

MQTT Message Queuing Telemetry Transport

MSx project Milestone (where x defines a project milestone e.g. MS3)

Mx Month (where x defines a project month e.g. M10)

NFC Near Field Communication

OBU Vehicle On-Board Unit

OS Operating System

PLC Programmable Logic Control(ler)

P Prototype

PP Restricted to other programme participants (including the Commission Services)

PU Public

R Report

RE Restricted to a group specified by the consortium (including Commission Services)

REST Representational State Transfer

SECaaS Security as a Service

TCP Transport Control Protocol

TLS Transport Layer Security

WiFi wireless networking; coined from Wireless Fidelity

Page | 10




WP Work Package

YAML Yet Another Markup Language

Page | 11




1 Introduction This document contains an overview of the design and preparation results in preparation of an

action plan for the M18 presentation, unless explicitly stated otherwise.

1.1 Overall use case scenario The multi-vendor use case scenario for Industrie 4.0 has to be implemented in according to the

ISA99 [ISA 99] and later on IEC62443 [IEC 62443] standards as those have turned out to be the

most relevant standards in D2.2 [D2.2]. This defines the environment where the scenario is

hosted into levels as described in Figure 1.1.1.

Page | 12




Figure 1.1.1 ICS - Recommended Network Architecture1*

This view may be simplified collapsing Level 4 and the DMZ and the Cell/Area Zone in single blocks

to a high-level description illustrating the different components in Figure 1.1.2 clearly illustrating

the critical transitions between the network layers handled by so-called conduits.

1https://ics-cert.us-cert.gov/sites/default/files/recommended_practices/NCCIC_ICS-

CERT_Defense_in_Depth_2016_S508C.pdf

https://ics-cert.us-cert.gov/sites/default/files/recommended_practices/NCCIC_ICS-CERT_Defense_in_Depth_2016_S508C.pdf

https://ics-cert.us-cert.gov/sites/default/files/recommended_practices/NCCIC_ICS-CERT_Defense_in_Depth_2016_S508C.pdf

Page | 13




Figure 1.1.2 High Level Industrial Network Architecture

In the use case scenario we focus on level 3 and below. The same scenario will be virtualized by

the P@SSPORT replica and a Digital Twin based on technologies used by Siemens in Mindsphere.

They will be interconnected using Mindsphere. In a later stage the connection to the Fujitsu

industrial IoT platform Colmina will be examined. The logical view of the use case scenario in each

virtual factory integrates three distinct use cases. These use cases represent business processes

that interact with the various levels of the ICS network architecture.

1.2 Injection molding In the injection molding scenario, Real-time machine-, sensor-, and IoT data are used for

Industrial Analytics functions to optimize injection molding production flow on the shop floor and

provide feedback to the machine operator.

Page | 14




Figure 1.2.1 Generic Image of Injection Molding Machine2

1.2.1 Use Case

In this use case, continuous data from machines, sensors and IoT components are pre-processed

on edge-level and subsequently, industrial analytics services are run at edge- or cloud level. The

data is provided by the machine itself in the Euromap63[ EUROMAP 63] format, a standard

format it injection molding and plastics processing. In addition, external sensors (Bosch XDK) can

provide data.

Figure 1.2.2: Logical view - Use case 1

In the initial version of the validation (M15-M18), selected injection molding parameters are

simulated and saved in the Euromap63 format. Then, a Filebeat probe connects to the SecureIoT

services.

2 photograph by Koh i Teck, distributed under a CC-BY 2.0 license.

http://creativecommons.org/licenses/by/2.0/

Page | 15




1.2.2 Abuse cases and validation scenarios

In deliverable D6.1 [D6.1] three general abuse cases for the injection molding scenario are

defined. From these abuse cases, we derive actions to be performed in the validation

environment in order to simulate these abuse cases.

Use case 2.2 (abuse)

id UC2.2

Description/scope

An IoT device is compromised and allows an intruder to monitor network traffic

Validation An intruder with access to the network subscribes to an MQTT topic and reads information (if authentication is not used with the MQTT protocol). Since the Packetbeat monitors the network at a central location and all network interfaces (cf. Figure 2.1.1), an indication of the location, where network traffic is going to ,is included in the data gathered by the Packetbeat. Thus network traffic going to an unusual location is visible in the provided data. This validation scenario will be realized with the implementation of level 3 of the validation environment, until M18.


id UC2.3

Description/scope An intruder can additionally modify data from a single sensor or IoT device and thereby influence machine behavior

Validation A malicious operator manipulates the machine settings in the simulated HMI. Setting the barrel temperature to low leads to the formation of a cold slug in the injector nozzle, causing a cold slug to form. The cold slug is pressed out in the next cycle leading to momentarily maximum pressure (cf. Figure 1.2.3 and Figure 1.2.4). The Filebeat pushes the operation data gathered in Euromap63 compliant files to the SecureIoT data collection. This way among those data the injection molding machine’s injection pressure is monitored and can be analyzed. These activities will be finished until M18.


id UC2.4

Page | 16




Description/scope An intruder can access a large number of IoT components and sensors and use them to attack other network components

Validation Modified abuse case: Intruder can access several (one or two) components (e.g. Raspberry Pi and MQTT broker VM as described in Section 2.1.6). A Denial of service attack is simulated by spamming the network and stopping MQTT broker service to stop diagnostic data flow. The increased network traffic will be reported by the Packetbeat running in network level 3 and the Auditbeat will diagnose the stopping of the MQTT broker service. To be realized in the demonstrator in M24.

Figure 1.2.3 and Figure 1.2.4 illustrate the patterns of UC2.3

Figure 1.2.3 Normal Injection Pressure

Figure 1.2.4 Anomalous Injection Pressure in the Case of a Cold Slug

-200

0

200

400

600

800

1000

1200

1400

-1 1 3 5 7

Injection Pressure

-200

0

200

400

600

800

1000

1200

1400

1600

-1 1 3 5 7

Anomalous Injection Pressure

Page | 17




2 Scenario implementation For the implementation of the use case scenario two separate demonstration environments will

be built. One implementation is built in a custom Replica environment (cf. Section 2.1), while the

other is built in a Mindsphere environment (cf. Section 2.2). As simulations vary on their

abstraction level depending on the concrete simulation goal, the disparity of the two

demonstration environments enables a more thorough validation of the SECaaS. Thus, certain

parts of the demonstration environments are redundant where others are specific to the

realization in question.

The difference between a digital twin and the replica is that the digital twin primarily represents

the virtual production, while a replica together with the associated hardware platform represents

the entire stack. The replica and the associated platform are implemented independently of a

real factory, making them flexible and scalable. It has also been specially developed for use as a

development, test and acceptance environment. Special functions such as snapshots and cloning

functions of the replication platform make it easy to maintain and manage the real systems.

Hence, it will represent a different view to the virtual factory. It is supposed to obtain a valuable

comparison of the SecureIoT services with regard to those approaches.

2.1 Replica architecture description The Replica is used as a cyber to physical virtualization environment to use in “what if” scenarios

and/or testing (new) security controls in the process control environment without the need to

first buy the hardware or test in a production environment. The replica technology is not bound

to a specific ICS vendor. Within the use case the replica will represent a virtual model of the

industrial environment outlined in chapter 1.

The network architecture shown in 2.1.1 will be implemented on various virtual hosting

environments. One environment will host the virtual implementation of levels 0-2, another

virtual hosting environment will host the remaining levels except for the SECaaS cloud

environment. In the following sections the concrete implementation of the use case using the

replica is described for the various layers 0 to 4.

2.1.1 Overall Network architecture The Replica architecture follows the ICS standard as outlined in Figure 1.1.1. This way it reflects

the levels and the network segmentation of a tpical industrial environment. Figure 2.1.1

illustrates the overall network architecture. In the following sections the components of each

level are described.

Page | 18




Figure 2.1.1 Replica overall Network Architecture

2.1.2 Level 0 – Machine Level

The Machine Level consists of the processing devices, i.e. the moulding machine with the

attached external sensors and the HMD In this section the devices and their modelling is

described.

Molding machine:

The molding machine is modelled describing the injection molding process. As industrial security

is in focus, some simplifications of a real piston machine have been applied. Figure 2.1.2 below

illustrates the model.

Page | 19




Figure 2.1.2 Moulding machine schematic model

The molding machine is virtualized in a dynamic model running on a virtual Windows Server 2012

R2 host. For the model to run, another virtual Windows Server 2016 Standard is used as license

server. The molding machine is virtualized in the dynamic model with the exception of the

External Sensor Unit.

External Sensor Unit:

This unit is emitting its sensor data via a wireless network on the shop floor and is then passed

on via platform B to platform C. Integration of the External Sensor Unit is planned for M18.

Head Mounted Display:

The addressed HMD device contains several core building blocks and is built as a ruggedized

device to be used in shopfloor operations and factory facilities:

• Optical output via side display (full color)

• Acoustical output via headset and/or neck loudspeaker

• Acoustical input via microphone (speech recognition, alarm detection etc.)

• Local core processing unit and periphery (memory, storage etc.)

• Local power host with a running time of at least 8h

• Interaction tabloid (e.g. key press recognition, navigation etc.)

• Input device based on a camera module

• Several interface connectivity’s such as BT, WiFi and NFC

• Interface (WiFi, BT, NFC), IoT sensor elements e.g. Beacons etc.

Page | 20




The target usage of such device frame is to ensure workers’ safety and support worker’s

machinery operation by offering guiding and instruction sets. Therefore, configurations of

machinery based on product configuration needs are done with hardly preprocessing time.

Moreover, the information set / guiding instruction preprocessing is part of the backbone which

will be transferred on time towards the HMD device by usage of several modules out of the MES

(Manufacturing Execution System).

As the HMD appliance does have a local processing unit (ARM CPU frame), which involves among

others memory, storage and OS capability (Android 7.x) local installation of microservices and 3rd

party application are possible. Such applications are necessary to adopt local needs and

necessities such as worker identification, pattern recognition, speech recognition, bar code

scanner, etc. just to name a few. As a result the whole HMD frame will operate as a gateway

between corporate and standardized operations (MES based workers guidance) and local

deployed customization (machinery based, factory needs).

In order to have sufficient data connectivity and bandwith, dedicated long range (Wi-Fi,

longrange BT) and short range (NFC) technologies are part of the HMD. This capability ensures

communication from backbones (MES) as well towards local machineries and IoT devices (BT,

NFC).

As a result, the HMD appliance will be used on a daily basis to guide workers on occurrences of

machinery operation (configuration based operation) or react upon unexpected issues needs to

be handled during machinery handlings (safety).

2.1.3 Modelling the Molding Process

In industrial security, anomalies in processes may indicate a security incident or an attack. One

of the most famous examples, Stuxnet, illustrates this and the tremendous impact of a distorted

process. Therefore we illustrate briefly the molding process and the process data.

Figure 2.1.3 General Moulding Process

In general the process may be divided into four parts:

Page | 21




1. Idle, where the machine is not maintained or processing molding parts.

2. Maintenance, when the machine is maintained

3. Molding

4. Releasing a part

Within the virtual Replica, focus is on the cycle of states 1, 3 and 4.

Figure 2.1.4 illustrates the molding cycle in state 3. The filler unit needs to contain sufficient

material, which is heated and will be released into the cavity when a certain temperature is

reached. When the cavity is filled, the material will be expelled into the mold by moving the

piston rapidly.

Figure 2.1.4 Moulding process moulding states

Page | 22




The mold usually is cooled under pressure, so that the piece solidifies soon. Then the piston is

withdrawn, the mold inlet will be closed and the piece will be released. The release process in

shown in Figure 2.1.5.

Figure 2.1.5: Moulding process moulding release states

Taking this process into account a set of synthetic data has been modelled based on real moulding

data representing sensor data in one moulding cycle. For instance, Figure 1.2.3 shows exemplary

data for the injection pressure.

These data may be distorted for training purposes by the HMI in Level 2. Moreover, the data of

the internal sensors are sent to the local process historian server (aka historian) in Level 2. This

local historian shall transport the data via MQTT to the historian in Level 3. For simplicity, the

data of the external sensors are transferred to the historian in Level 3 this way.

2.1.4 Level 1 – Field Level In level 1 the PLC of the injection moulding machine is located. Even if usually a closed system

will be used, the modell shall algin with the various levels as described in Figure 2.1.1. The PLC

will be virtualized as a software agent. The agent will collect the data from the dynamic model in

level 0 and send it to a broker in level 2, thus acting as a publisher. Functionality to transport data

from level 2 to the molding machine in level 0 will be implemented for M18. The intention is to

operate an HMI in level 2 to manipulate the behavior of the dynamic model to enable

presentation of an abuse case (to be defined!).

2.1.5 Level 2 – Realtime Level / Control Network In this level the components of the injection moulding machine use case are implemented. There

are considered:

• Level 2/1 switch

Software defined networking will be used to implement the required switch functionality

in level 2 and between level 1 and 2 and between levels 2 and 3.

Page | 23




• Time server

A software agent will be installed to act as a timeserver for level 2.

• HMI

Data collected from the dynamic model will be near real-time displayed in a simple HMI

(graph/dashboard). The HMI will be built on a web server and can be displayed on a web

browser running in a virtual PC. To present data, data subscriber functionality will run in

the browser to collect data originating from the molding machine. In a second stage, for

M18, the functionality of the HMI is extended to show an abuse case presented in the

dynamic model. This implies adding publisher functionality in the browser to send data to

the molding machine. Interaction to/from the subscriber/publisher will be established

with a broker running on the application server.

• Application server

An application server is used to host a web server (for HMI). It will also store data from

level an communication proxies in a local historian database.

• Data brokers

Three data brokers will be implemented. One for north/south communication between

level 1 and 2 (bi-directional, level 2 to 1 by M18). A second broker for north/south

communication only one-way from level 2 to level 3! And one east-west communication

data broker to communicate between HMI and PLC in level 1 (HMI to PLC for M18).

2.1.6 Level 3 – Site Manufacturing / Process Level The implementation of the virtual validation environment of level 3 is done by deploying virtual

machines on a Fujitsu IntelliEdge Gateway to mirror actual industrial environments. This device

also can provide capabilities of a conduit as illustrated in Figure 1.1.2 and it includes the space

for hosting the virtual machines and containers for the replica. Figure 2.1.6 shows the

implementation environment.

Figure 2.1.6 The implementation on the Fujitsu IntelliEdge Device

Page | 24




The virtual validation environment is implemented in virtual machines VM1 and VM2, described

in here. VM3 depicts the DMZ and its functionalities are described in section 2.1.7, while VM4

depicts the local business planning environment and its functionalities as described in section

1.1.

• Local MES Historian

A Times Series Database will collect the IoT sensor data collected from Control Network

in level 2.

• Database API’s

To access a database (CMDB/Local MES Historian) API libraries will be implemented to

access the respective databases.

• Local MES Control

Production specific applications to manage the production process.

• Proxy

The proxy services will provide an interface between level 2 and 3.

• Beats

Various beats will be implemented to transfer data to the SECaaS platform in this level.

E.g. Filebeats shall provide process data and Packetbeates shall provide network data to

the SecureIoT data collection layer. For details see section 3.

• Dashboards

Dashboards used to monitor the production process and show notifications from SECaaS.

• CMDB

See paragraph 2.1.9.

2.1.7 DMZ

• MindGate server

Communication with MindSphere is only allowed through the centralized gateway – the

MindGate. This is implemented within the DMZ. A similar gateway shall be used for other

IoT platforms. This is a gateway further detailed in paragraph 2.2.2.

• IAM server

An IAM server is used to authenticate and authorize users and other clients accessing the

DMZ level, level 3 and or level 4.

• Other apps

A web server will be implemented to provide remote access from external locations, i.e.

partner access.

2.1.8 Level 4 – Site Business Planning

Functionality to support business processes in level 4, could be a production monitoring

dashboard and/or a production reporting tool. This functionality is out of scope for this project

to be implemented.

Page | 25




2.1.9 CMDB Data Structure

The local CMDB contains local configuration data. Such data are required by the Compliance

Audit Service and are relevant for assessing Trustworthiness. The configuration data necessary

for validation of these services are stored in this CMDB as long as the implementation of a general

SecureIoT CMDB for storing Security relevant configuration data is under construction.

The CMDB for an industrial production site manages a distributed infrastructure, i.e. it "knows"

and recognizes all industrial assets (i.e. the ICS systems such as machines, PLCs, network

components and control systems) of a site's segments and devices that can be reached,

monitored and inventoried. Such a CMDB is implemented as a true local database for the entire

plant inventory. This is illustrated in Figure 2.1.7.

Figure 2.1.7 Inventory Interface for Distributed Assets

The contents of the local CMDB represent the baseline configuration items (CIs) of the assets at

the beginning, i.e. after they have been scanned for the first time. After each further scanning,

only changes are marked as delta configuration items. In practice, the time interval between

scans is usually 24 hours, but can be defined arbitrarily. As a rule, this process is carried out

outside the production times in the shift breaks.

As soon as new baseline or delta CIs are available in the CMDB, they are converted into the

corresponding beat structures and transmitted. If no changes have been made to the assets

between scans, no transfers will take place.

The feature of a CMDB is based on the inventory of, for example, all assets of all systems of a

production site. This principle does not apply to autonomous assets. Therefore, a lightweight

version should be provided for this system, which is implemented directly on the asset as

illustrated in Figure 2.1.8.

This does not mean a complete CMDB implementation as for many systems, but the light

weighted integration of a local inventory service directly on the asset, such as a car’s on-board

unit (OBU) or a social robot, consisting of a baseline and delta management of configuration

items (CIs), analogous to the CMDB description above. The communication principle and the

Page | 26




conversion into the corresponding beats also correspond to the description of a complete CMDB

as in the previous case.

Figure 2.1.8 Inventory Interface for Autonomous Assets

From the perspective of SECaaS Services, a local CMDB or Asset Inventory can be considered as

an intelligent probe inventorying the assets.

The following are examples of data structures from existing network structures that may be

captured using the local Inventory Service.

Page | 27




• CMDB Data Structures – Devices:

Configuration Items (CIs) – Network (Examples)

Page | 28




• CMDB Data Structures – Topology:

Configuration Items (CIs) – (Examples)

Page | 29




Page | 30




• CMDB Data Structures – Services:

Configuration Items (CIs) – Access rights (Examples)

Configuration Items (CIs) – Server Software (Example)

Page | 31




The mapping of these local CDMB configuration items into the central/global CMDB structures is

in coordination with WP3 and WP5.

In order to assign known device-specific vulnerabilities, threats and controls to specific devices

and the above congurations, security connections to the central knowledge database must be

established in the central/global CMDB structures. The security structures are coordinated with

WP3.

• Logical CMDB Security structures - Example:

Confuguration Item: NodeID

Security:

Vulnerability: References ID to Knowledge Base

Threat: References ID to Knowledge Base

Risk: References ID to Knowledge Base

Control: References ID to Knowledge Base

2.2 Digital Twin 2.2.1 SIEMENS high-overview of a Digital Twin

Siemens’ vision of a digital twin represents a virtual representation of a physical system, with a

set of key capabilities:

• A closed feedback loop, ensuring that performance data from production environments

fit into designs

• A permanent connection between physical assets and digital twin, enabling a “common

object memory”,

• Integration with Siemens-specific products, to continuously optimize production digitally.

Figure 2.2.1 depicts these circular process and feedback between the two, digital and physical

environments during the developmental process.

Page | 32




Figure 2.2.1 Digital Twin in Siemens MindSphere

2.2.2 Architecture and Interaction with MindSphere

MindSphere supports the creation and evaluation of digital models representing infrastructure-

specific platforms mainly through its data acquisition and analytic functions. More specifically,

MindSphere supports secure data collections, long-term storage, as well as a plethora of analytics

and machine learning services for analysis of the data, trends, predictions, etc.

In addition, custom models can be executed, allowing for the definition and execution of

scenario-specific models which can further assist in analyzing data available.

A complete digital twin, solution, as required by the Secure IoT project, cannot be supported

through MindSphere alone however, and relies on employing other solutions provided by

Siemens as well, such as Solid Edge, Siemens NX, and other similar CAD, design and modeling

environments.

Alternatively, through the mediation performed as part of a custom solution, such as a MindApp,

other solutions can be employed as well. Figure 2.2.2 details such an approach, which is also

proposed by Siemens as part of the Digital Twin support provided as part of the Industrie 4.0

scenario as well. As depicted, the prepared MindApp is relying on two distinct pillars for

execution its digital twin functionality:

• MindSphere: the MindApp invokes and makes use of typical MindSphere functionality for

persisting, querying, retrieving and analyzing data.

This task relies on a series of MindSphere typical services, such as (i) IoT (Filesystem) for

persisting IoT data in Siemens’ IoT Model format, (ii) Data Exchange, for persisting data in

any unconstrained format and sizes (up to TB), and finally (iii) Model Management and

Job Manager, for defining custom model formats and their execution cycle and data

sources.

Page | 33




Communication with MindSphere is only allowed through the centralized gateway – the

MindGate (to be placed in DMZ), and the access control as well as authorization is

performed by the dedicated Identity and Access manager – IAM. Both are also detailed,

and the proper configuration, registration, tenant setup and tockens required in order to

access MindSphere are enabled and handled by the MindSphere Connector Library

component in the Twin MindApp.

• Dockerized Simulation Environment: the docker environment referred here is executed

OpenModelica, Repast simulations or both on pre-defined setups, as a process simulation

and according to the models persisted at MindApp level alone.

These simulations represent the actual digital twin process – a product level digital twin

– which allows us to analyze how the SecureIoT system can react to various process-

relevant conditions.

The parameters for the process simulation rely on a pre-defined set of curves describing core

parameters and functionality, reliability and inter-dependencies of the components described in

Section 2.1.3. Besides executing the simulation and passing data in/from the MindSphere

persistency and analytics services, the digital twin MindApp is intended to successfully process

and execute a dynamic range of user requests received through an open HTTP REST interface.

More specifically, the MindApp represents, to consortium stakeholders, an invokable service

which can:

• Forward queries either to MindSphere, or to the simulation environment running the

OpenModelica or Repast instances;

• Can store and configure Twin relevant CMDB information through its own, local,

persistence layer. This is especially relevant, since it can store complex configuration of

rules – especially in a project-relevant format.

• Can retrieve sensor-specific information, as well as modify their states within the

Simulation Environment, in order to allow for simulating of events relevant for testing or

replicating certain attack scenarios or system states.

Besides executing the simulation and passing data in/from the MindSphere persistency and

analytics services, the digital twin MindApp is intended to successfully process and execute user

requests.

Finally, the MindApp has one more specific task: data translation functions. More specifically, the

MindSphere Connector Library component is as depicted, has the role of translating any

SecureIoT or use-case-specific model either into the IoT Model as used by MindSphere, or into a

custom format which is persisted in non-typed persistence services such as Data Exchange.

Similarly, on retrieval, especially triggered by external quires, a reverse transformation is

required, as well.

Page | 34




Figure 2.2.2 MindSphere Twin MindApp

2.2.3 Roadmap for the first release of the Digital Twin MindApp The most challenging component in the realization of the MindApp is represented by the correct

modeling of the twin realization. More specifically, the challenge is to correctly represent the

molding process and its data-infusion end-points grammatically within the simulation

environment.

For a complete and correct evaluation and scoping of the core functionality set of the twin,

following activities are currently running:

• Definition and formal specification of the simulation parameters

• Evaluation of both OpenModelica and Repast platforms. To be more specific, the specific

requirements (e.g. whether the process simulation should be extended with various

physical events) will determine the chosen solution.

A second required evaluation parameter refers to the need for alignment and triggers for

external events within the SecureIoT platform which “observes” the replica’s state, traffic

and process executions.

2.3 IoT-Platform Integration – Mindsphere The integration of the Twin MindApp within the IoT platform requires the establishing of a

connection between the on-site log aggregation mechanisms, and the MindSphere environment

– or, in this case, MindApp.

Page | 35




From a protocol point of view, the most straightforward approach is to rely on HTTP REST services

for uploading data within MindSphere. The application detailed in Figure 2.3.1 provides the

required REST endpoints for external clients.

From an interaction point of view, Figure 2.3.1 showcases the interface between the MindApp

and the Industrie 4.0 showcase deployment:

Figure 2.3.1 Twin MindApp integration into the Industrie 4.0 Scenario

Technically, Filebeat supports a series of plug-ins, including APIs configured for pushing data

through a HTTP call to external services. Besides the HTTP API plug-in, a second plug-in is relevant,

namely the HTTP Output Plug-in, which can include the propagation of certain events which can

trigger the MindApp into performing more complex queries to the Filebeat server.

Table 1 details the main API calls currently defined for the SecureIoT Twin MindApp. A 1st

specification, at model, input parameter ranges, error messages and response codes has been

circulated as a YAML specification file for review among the use case participants. It is provided

to the partners in the project’s repository: https://gitlab.atosresearch.eu/SecureIoT/seciot-

industry4.0/blob/268bb851a610bf85ae67e7678d04858914fb0a84/twin_app_yaml_(in%20prog

ress).yaml

Table 1 REST call samples to the SecureIoT Twin MindApp

Id. Operation Type

Endpoint Entities Description

1 PUT /simulation/{tid}/sensors JSON description of the sensor

The endpoint is responsible with registering sensors within the simulation environment.

https://gitlab.atosresearch.eu/SecureIoT/seciot-industry4.0/blob/268bb851a610bf85ae67e7678d04858914fb0a84/twin_app_yaml_(in%20progress).yaml



Page | 36




2 GET /simulation/{tid}/sensors JSON array containing a list of all registered sensors

The endpoint allows the user to list all registered sensors within the system.

3 PATCH /simulation/{tid}/sensors/{id} JSON object representing the change in the resource’s description

Allows a user to update sensor specific information.

4 PUT /simulation/{tid}/sensors/{id}/states JSON description of the sate of a sensor (identified by {id})

The endpoint allows the configuration of a sensor’s state. The last PUT operation defines and sets the current state.

5 GET /simulation/{tid}/sensors/{id}/states JSON array containing a list of all states registered to a specific sensor, ordered descending.

The endpoint allows a user to list all states of one of the registered sensors.

5 GET /simulation/{tid}/sensors/{id}/states/{sid} JSON array, providing the full description of a specifi states

Allows a user to retrieve the complete state description (identified by {sid}) of a specific note (identified by {id}).

6 GET /simulations JSON array providing the user with a list of all

Allows for the listing of all simulations currently being

Page | 37




current twin simulations being executed

executed within the Twin MindApp

7 GET /simulations/{tid} JSON object providing a complete simulation status (identified by {tid})

Allows a user to list the complete status of a simulation running in the MindApp.

2.4 IoT-Platform Integration - Fujitsu Colmina Fujitsu’s platform Colmina is containing major building and feature blocks in order to interface

layers of operation proceedings. Those layers are hosting appliances which contains services in

order to offer end-to-end solutions e.g. pattern recognition, AI based predictive maintenance,

instruction generator.

Figure 2.4.1 COLMINA Architecture

The platform Colmina is going to be launched within CE / EMEIA area by Oct 2019. As for the final

feature specification, it hasn’t been finalized, yet. This chapter will be updated and enriched by

feature description after the final setup has been confirmed.

COLMINA links data on the location of people and products, on factory equipment, and all

systems and expertise throughout the manufacturing process, as well as data among companies

in the supply chain. Underlying this solution, there is the Fujitsu Manufacturing Industry Solution

COLMINA Service, a suite of a variety of operational services for manufacturing, the Fujitsu

Page | 38




Manufacturing Industry Solution COLMINA Edge, which collects and processes sensor data, such

as on equipment operations, the vital signs of people, and the location of products, and the

Fujitsu Manufacturing Industry Solution COLMINA Platform, which brings together the COLMINA

Service and the COLMINA Edge. Standard interfaces between the COLMINA Platform and other

companies' solutions and platforms enable data on everything from design to production to be

linked throughout the entire supply chain, it will facilitate the digital transformation of all

manufacturing work. The manufacturing platform enables integration across ERP, MES and other

manufacturing data sources. Where additional data is required for manufacturing optimization.

Fujitsu is committed to open standards and build digital solutions using Opens Source including

Hadoop, Spark and Elastic.

3 Integration of SecureIoT components 3.1 Integration of WP3 Interfacing Data collection and Collaboration

Figure 3.1.1 Probe Placement

Page | 39




3.1.1 Monitoring the Machine Operations – Filebeat

The Elastic Filebeat is used in all three usage scenarios within the Industry 4.0 cluster, the

following section describes how the filebeat is used in each scenario.

S2: Injection Molding Diagnostics

In injection molding, the Euromap63 protocol is widely used for exchanging configuration and

diagnostic data. The protocol is file-based, with the file format being general purpose csv. The

.csv file contains all values in one row. A schema defines which parameter is represented by a

value in the log file. This data is stored on a historian or application server. On this server, a

Filebeat monitors the location where the logs are saved and when a log is filed or updated, the

Beat reads out the data values and sends them to Logstash. Logstash is then configured using the

Euromap63 schema to assign the values the correct parameter names and push them up the

Elastic Stack towards the SecureIoT services.

Implementation Status: Prototype complete

3.1.2 Monitoring the Factory Network - Packetbeat

The network monitoring is performed by a PacketBeat. The PacketBeat is deployed at a central

location so it can monitor all network data that is of interest. Currently, it is planned to deploy

the Packetbeat on a network level 3 switch, where it monitors all network interfaces. The specific

protocols to be monitored include but are not limited to: HTTP, DNS, TCP, and TLS. For MQTT,

the community mqttbeat (https://github.com/nathan-K-/mqttbeat) may be used.

By default, the packetbeat monitors only the standard ports associated to a protocol. To detect

port abuse, other ports to be monitored must be specified for each protocol.

Implementation Status: Planned for M18

3.1.3 Monitoring the Process Level - Metricbeat

Similar to the Packetbeat, the Metricbeat is independently of the usage scenario. It is deployed

on the application server in network level 2. The Metricbeat is used to detect excessive resource

usage as in denial of service attacks for example.

Implementation Status: Planned for M18

3.1.4 Monitoring Access Control - Auditbeat

Auditbeats allow to monitor user activities and file integrity. It could be used to detect if a user

ends a process that should not be terminated or changes a file that should not be changed. It

could be used to monitor changes to the injection molding machine configuration or the product

configuration file.

Implementation Status: Planned for M18 (Injection molding configuration), M24 (product

configuration)

Page | 40




3.2 Integration of WP5 SecureIoT Services Implementation and

Integration (SECaaS) This section gives an overview on the status of implementation planning. For the sake of

completenes all SECaaS of WP5 are listed.

3.2.1 Risk assessment and mitigation services

No integration with these services is available at M15. Please refer to Chapter 4 for the planned

integration until M18.

The inputs to the “risk assessment and mitigation service” are defined in [D5.1]. For better

readability, we cite the corresponding parts below and explain how the Multivendor Industrie4.0

(MVI4.0) use-case can provide the required information:

• “Business configuration indicators: this is the management-level information compiled

from the employees. It compiles information regarding critical assets, business, type of

services offered, criticality, costs of losing an asset for an unknown period, etc. Usually

the information is compiled using questionnaires or other type of user-level interaction.

This is later transformed to machine-oriented for the assessment of the system.”[D5.1]

Integration: This document and the preceding MVI4.0 deliverables define the main assets and

their interconnection. As soon as the required information is concretized and first questionnaires

are available, we can directly transfer the use-case and add information that might be missing.

• “Vulnerability result indicators: this input is the vulnerability analysis of the target system.

It is a report generated by a tool such as a vulnerability analysis or SIEM. This information

is usually provided machine-ready as it is the output of risk and vulnerability assessment.”

[D5.1]

Integration: In the context of SecureIoT the main tool to provide this information is the IoT

security knowledge base (cf. Section 3.2.4). This will also be leveraged by the MVI4.0 use-case.

• “Network monitoring indicators: this indicator, also machine-oriented, is the

identification and report of the networking of the system and status. The goal with this

indicator is to be aware of the networking of the system and its cyber-status.” [D5.1]

Integration: We will utilize packetbeats (cf. Section 3.1.2) to provide network monitoring

capabilities for the risk assessment and mitigation service.

• “Application monitoring: it provides information about the cybersecurity monitoring of

the applications of the target system. This one, together with the network monitoring and

vulnerability results, aim to provide the cyber-status of the system covering different

layers of the system such as the vulnerabilities of the whole system, network and

application level.” [D5.1]

Page | 41




Integration: The MVI4.0 use-case mainly copes with application level semantics in terms of the

injection molding machine, i.e., information that conforms to the file-based Euromap63 protocol.

Thus, application monitoring will be realized by filebeats (cf. Section 3.1.1).

• Raising the monitoring level as reaction to a detected anomaly is a functionalities the

project targets in case of anomalies that cannot be identified as being attacks yet.

Integration: Raising the monitoring level requires a reconfiguration of the deployed probes. To

realize this, changes of the beats’ configuration is required. To this end, a connector must be

implemented, which can be deployed alongside a probe, which receives instructions from the

reconfiguration service and then changes the beat configuration accordingly.

3.2.2 Compliance auditing service

The Compliance Auditing Service (CAS) provides a small set of features until month M18.

Considering five policies it may evaluate the compliant configuration of assets in the CMDB. The

CMDB shall be aligned with the CAS requirements. We will provide configuration data related to

some assets in Layer3 and examine the application of the CAS for a set of assets in Layer 3 until

M18.

The integration of the CAS depends on several steps considering the CMDB, i.e.

• Composition of an example about the use of a CMDB (Configuration Management Database)

in a real scenario (WP3-4-5).

• Proposal of a CMDB technology to be used within SecureIoT (WP3-4-5).

• Analysis of CMDB potential and fitting with the MVI4.0 scenario (WP6).

• Configuration and deployment of CMDB for MVI4.0 scenario (WP6).

The environment providing CMDB data to the SecureIoT includes the implementation of an

inventory (e.g. a local CMDB or a local inventory). Beats provide a lean way of cyclically providing

data to the SecureIoT data collection (WP3). This environment is planned to be designed and

implemented until M18 providing data to the CAS.

3.2.3 Programming support services

No integration with these services is available in M15. Please refer to Chapter 0 for the planned

integration until M18.

At the moment of writing, the programming support services comprises developer support for

the design-time specification of policies for Attribute-based Access Control (ABAC) and their

enforcement at runtime (cf. [D5.7]). The most natural and intuitive points to utilize ABAC in the

MVI4.0 use-case are

• the Operator HMI in the injection molding usage scenario (cf. Section 1.2).

• the IAM Server in the DMZ (cf. Section 2.1.7)

Page | 42




Integration example: The Operator HMI and the ABAC capabilities provided by the programming

support service can be integrated to prevent malicious usage, e.g, in the context of insider

attacks. To give a simple example: A policy can express that certain parameters of the injection

molding machine must not be changed at unusual time, e.g., at night. If a machine operator

violates this policy his privileges can be revoked.

Integration is not planned to be provided until M18.

3.2.4 IoT security knowledge-base The IoT security knowledge base is mainly meant to give an overview of security information

acquired from different sources (CVE, CPE, etc.). As such, it is supporting developers in choosing

base technologies or staying informed if new vulnerabilities arise in already deployed software

(cf. D5.10). The MVI4.0 use-case will utilize this knowledge base during the development and the

maintenance of the simulated industrial environment described in Chapter 0. Furthermore, the

IoT security knowledge base serves as an input for the risk assessment services that will be

integrated in this use-case (cf. Section 0).

Integration is not planned to be provided until M18.

Page | 43




4 Implementation and Validation Plan until M18 The implementation of the MVI Industry 4.0 use case has been designed according to standards

and along ongoing discussion in Industry 4.0 related initiatives as the Plattform Industrie 4.0.

Hence, during the process some implementation routes needed to be revised.

The major add on in the use case implementation is the approach to utilize different technologies

simulating the industrial environment and interconnecting them by Mindsphere. This way we can

examine, if the same anomaly will cause the same analytical result in analogous environments.

The machine level of this Industry 4.0 use case simulating the injection moulding machine in the

replica is available. Synthetic process data and distortions of it are available to be provided to the

data layer (WP3). Moreover, the other layers of the industrial networks are specified and the

replica as well as the digital twin are prepared for further implementation and connection to

Mindsphere.

The implementation plan listed in Table 2 outlines the implementation until M18.

Table 2 Planned Implementation Steps until M18

M16 • Extension of the implementation of Layers 2 and 3 in terms of the industrial environments, i.e. replica or digital twin

• Initial integration into SecureIoT data collection, i.e. 1st deployment of probes (Metricsbeat, Filebeat, Auditbeat) in the SecureIoT environment, configuration of logstash for transformation into the SecureIoT data model

• Design of the inventory service, CMDB technology and CMDB related data structure

M17 • Extended Integration into SecureIoT data collection, i.e. development of CMDB data related beats, implementation of lean dashboard for validation

• Connection of a simulated industrial environment to Mindsphere

M18 • Testing and Demonstration preparation

Based on this implementation plan WP3 Interfacing, Data Collection and Collaboration will be

validated until M18. Probes will be deployed at the replica in level 3 and registered in the

SecureIoT probe registry. The shipped data will be transformed into the SecureIoT data model

using Logstash and validated in a Kibana based dashboard facilitating training of the analytics

models in WP4. Therefore the KPIs below shall be achieved:

• Number deployed probes (i.e. beats) >=3

• Different levels for which security monitoring probes will be developed >=1

• Visibility of regular data in SecureIoT Data Collection layer >= 90%

• Visibility of distorted data in SecureIoT Data Collection layer >= 70%

Page | 44




References

[D2.2] “D2.2 - Analysis of Stakeholders’ Requirements”, SecureIoT (05.2018)

[D5.1] “D5.1 - IoT Risk Assessment and Mitigation as a Service - First version”,

SecureIoT (12.2018)

[D5.7] “D5.7 - IoT Developers Support as a Service. First version”, SecureIoT (12.2018)

[D6.1] “D6.1 - Detailed Specification of Usage Scenarios and Planning of Validation

Activities - First version”, SecureIoT (09.2018)

[Euromap 63] “EUROMAP 63 Data Exchange Interface”, www.euromap.org/files/eu63.pdf

[IEC 62443] Kobes, P.: “Guideline Industrial Security: IEC 62443 is easy” (2017)

[ISA 99] “ISA99, Industrial Automation and Control Systems Security”,

https://www.isa.org/isa99/

http://www.euromap.org/files/eu63.pdf

DELIVERABLE D6.3 Integration and validation of Industrie 4.0 …secureiot.eu/D6.3.pdf ·...

Documents

Transcript of DELIVERABLE D6.3 Integration and validation of Industrie 4.0 …secureiot.eu/D6.3.pdf ·...