4/30/2008

1

OPC Tunnel for PlantTriage CommunicationCommunication

Benefits of an OPC Tunnel to bypass DCOM issues for secure, efficient PlantTriage communication across firewall / WAN

Chris Friedman

Introduction

Chris FriedmanSABIC Innovative Plastics

2

- SABIC Innovative Plastics- Information Management Organization / IT- Chemical Engineer- Global and local projects- Mt. Vernon, Indiana site- Multi-site PlantTriage implementationMulti site PlantTriage implementation

4/30/2008

2

Implementation BackgroundProject Goals- Implementation at numerous global sites

3

- Consistent and standard implementation strategy- Separate server(s) by sites

• Site size and complexity vary considerably• Number of loops and users vary

- Centralize architecture as much as possible- Minimize hardware footprint and cost- Minimize hardware footprint and cost- Consistent and robust access to process data- No security risk to process network

Implementation Background 4

CentralData Center

LevelWAN

(Wide Area Network)

Site LevelSite 1 Site 2

Site Level

Plant / Process Unit Level

LAN (Local Area

Network)

Process Network

Data Historian

Data Historian

Unit 3Unit 2Unit 1Unit 3Unit 2Unit 1

Network

`

DCS / PLC`

DCS / PLC`

DCS / PLC`

DCS / PLC`

DCS / PLC`

DCS / PLC

4/30/2008

3

Architecture ConcernsCorporate IT- Centralize to data center where possible

E f i t d t

5

• Ease of maintenance and management• Easier hardware replacement

- Consolidate servers• Fewer servers means less hardware cost• Less maintenance cost

- Wide Area Network• Focus on WAN bandwidth and reliabilityocus o ba d dt a d e ab ty• Transparent server location

- Consistent and standard architecture

Architecture ConcernsLocal Site IT / Control- Process network

6

• Security and firewall not impacted for process network• Minimize impact to data traffic load on process network

- Data integrity• Performance of data transfer appropriate (collection frequency)• Accuracy of data collection to PlantTriage

- User accessibility• Users have robust access to appropriate web / engineering UI• Hierarchical data visibility (users can easily find their data)

4/30/2008

4

Architecture AlternativesLocation of PlantTriage server1) Data center (off-site)

7

2) Shared server on site LAN (site computer center)3) Multiple servers on site LAN (per unit / area)4) Server on process network (per unit /area)

Considerations- Data from DCS OPC through firewall or plant historian?Data from DCS OPC through firewall or plant historian?- Performance and accessibility- Corporate IT concerns

Architecture Alternative # 1Data Center (Off-site)

8

CentralData Center

PlantTriageServer

Site 1

Data Center Level WAN

(Wide Area Network)

Site Level

LANLAN (Local Area

Network)

4/30/2008

5

Architecture Alternative # 1Data Center (Off-site)- Considerations

N t k f WAN b d idth d l t

9

• Network performance, WAN bandwidth and latency• Reliability and uptime of WAN• Performance of data collection from site across WAN• Performance of web pages across WAN• Accessibility of PlantTriage engineering interface

- Discussion• High bandwidth and reliability of WAN (comparable to LAN)• Remote desktop access to engineering desktop• Several users sharing data (proper hierarchy of data needed)• Performance unknown, need testing!

Architecture Alternative # 2Shared Server on Site LAN

10

Site 1

Site LevelLAN

(L l A

Plant / Process Unit Level

(Local Area Network)

Process Network

Unit 3Unit 2Unit 1

PlantTriageServer

Data Historian?

`

DCS / PLC`

DCS / PLC`

DCS / PLC

4/30/2008

6

Architecture Alternative # 2Shared Server on Site LAN- Considerations

• Corporate IT architecture goals

11

• Site LAN reliability vs. process network• Hardware management responsibility• Data shared on same server (security, hierarchy)• Loop count limitations per server

- Discussion• Site LAN reliability and performance good (better than WAN)• Site management of server less efficient than centralized• Users accessing same server requires proper data hierarchy for efficient• Users accessing same server requires proper data hierarchy for efficient

visibility• No way to secure data by plant area in engineering interface

Architecture Alternative # 3Multiple Servers on Site LAN

12

Site 1

Site LevelLAN

(L l A

Plant / Process Unit Level

(Local Area Network)

Process Network

Multiple PlantTriage

Servers

Unit 3Unit 2Unit 1

`

DCS / PLC`

DCS / PLC`

DCS / PLC

4/30/2008

7

Architecture Alternative # 3Multiple Servers on Site LAN- Considerations

W ld h i ll t d t b l t ( it )

13

• Would physically separate data by plant areas (security)• High cost of hardware / maintenance for multiple servers• Main users like their “own server”

- Discussion• Performance would be good• Separation of data and users would be good from security

aspect, but Enterprise server needed to see all site data• High cost of multiple servers really eliminates this option without

some consolidation• Eliminate some servers appropriately more like option # 2

Architecture Alternative # 4

Server on Process Network

14

Unit 1

`

PlantTriage

DCS / PLC

PlantTriage Server

4/30/2008

8

Architecture Alternative # 4Server on Process Network- Considerations

• Would physically separate data by plant area / network (security)

15

p y y p y p ( y)• High cost of hardware / maintenance for multiple servers• No server consolidation possible due to network separation• Highest performance, robustness due to network diversity• No need to cross firewall for data collection, but site network users

would not be able to cross firewall- Discussion

• Hardware / maintenance cost makes this poor option• Multiple DCS systems (some with no OPC) eliminates this as a• Multiple DCS systems (some with no OPC) eliminates this as a

consistent option• Support and maintenance would fall to process network group (no IT

support)

Initial Conclusions- Option 4 - Many DCS types, some without OPC, high cost, lack of

access, etc. eliminate this option- Option 3 – Too costly for multiple site servers, server consolidation

necessary

16

- Option 2 – Good option, may require split of servers due to loop counts

- Option 1 – Best option from cost and management standpoint if performance is comparable to on-site location

- Other topic – Data historian as standard data collector?• Some DCS systems without OPC server• OPC / DCOM crossing firewall presents security issue to process network• Mixed environment if data historian not standard collector• Mixed environment if data historian not standard collector• Robust OPC data source required for data historian• Historian data structures, performance, configuration, load, etc. are

issues

4/30/2008

9

Key Decision Points- Number of servers minimized according to loop counts- Areas/units to share data and managed through data

17

organization and process- If server footprint is minimized, key decision is location

on-site or in off-site data center- Main consideration is performance and reliability- Corporate IT focus makes WAN reliability comparable to

LAN- Performance tests over WAN to determine final decision

Performance TestingCriteria- 2 second scan rates for 500, 1000, 1500, 2000 loops- Testing to multiple (4) OPC sources and single source- Test to multiple sites (worst WAN case)

18

p ( )- Data historian OPC server(s) used for testing- Web site response testing (web page update speed)- Compare response of local server to remote server- Performance data such as network load, CPU use, memory use, disk

speed, data processing speed, data accuracy- Stress test to 1 second data (OPC source may be limitation)- Considerations:

• OPC uses DCOM as underlying communication architecture• PlantTriage optimizes traffic through OPC data change “callback”• Tests performed with max stress, all values changing every 2 seconds

4/30/2008

10

Performance Testing 19

Site Network

` `

ExperTune PlantTriage Architecture

Firewall

PlantTriage Server(s)IIS / ASP.Net

OPC-DA from data historian

WAN or LAN ?

Server at Data Center vs. Local

Site Location

Process Network

DCS / PLC OPC Server /Data collector

Performance TestingData Center Server- Data Collection

• 500 loops at 2 seconds from 4 sources same as 500 loops from 1 source• At 500 loops, 2 second data points being lost every 8 to 30 seconds

20

p , p g y• 1000 loops worse, 1500 loops fails to maintain connection, 2000 loops

and PlantTriage fails to start• Performance monitors for network, CPU, disk usage, memory usage

show these are not limiting factors (on client or server side)• Problems due to WAN performance or overloading of OPC server?

- Web site response• Performance reasonable, compare to local response• Latency in WAN will always make remote web server slowerLatency in WAN will always make remote web server slower• Question is how much, is it reasonable?

4/30/2008

11

Performance TestingData Center Server (500 loops every 2 seconds)

PlantTriage Data Historian

21

Performance TestingData Center Server (500 loops every 2 seconds)

22

OPCOPC

4/30/2008

12

Performance TestingLocal Server- Data Collection

• 500, 1000 loops per 2 seconds scan with no data loss

23

• Stress tested to 1000 loops at 1 second with no data loss• 2000 loops per 2 seconds reaches limitations since problems also

experienced on local server (external limitation reached)• Performance monitors for network, CPU, disk usage, memory

usage similar to WAN tests

- Web site responseR f t th WAN i d b it t ffi t• Response faster than over WAN, varied by site, traffic, etc.

• Considered to be reasonable (similar to intranet sites)

Performance TestingData Center Server (1000 loops scanned per 1 second)

PlantTriage Data Historian

24

4/30/2008

13

Test Conclusions- Remote communication over WAN not acceptable

compared to LAN- Data historian OPC server is not the limitation

25

- Issue over WAN not identified as network limitation• Performance monitors show low network, resource usage• Network not limiting data transfer• OPC overhead must be involved• Research indicated that DCOM communication degrades with

network latency• DCOM is cause of issue!!!

DCOM- DCOM is underlying communication protocol for OPC- Difficult to configure, most OPC problems related to DCOM- Web research indicates:

26

• DCOM works best on networks with low latency and high bandwidth

• Networking is not OPC’s strength since it is based on DCOM• OPC was originally developed based on Microsoft COM

(component object model) that runs on single computer• Most users have difficulty on LAN with OPC/DCOM, and wouldn’t

even attempt it over a WANDCOM i hi h i ti h d- DCOM requires high communication overhead• Requires handshaking, security layers, user authentication• Lots of communication sensitive to latency

4/30/2008

14

OPC TunnelPossible solution

with OPC Tunnel- Best described in

27

DCOM

Network

OPC Server OPC Client

depiction- OPC Tunnel

eliminates DCOM, making the communication over network much

OPC is inefficient over a network due to DCOM.It is worse over WAN due to increased latency.

TCP/IPOPC Tunnel OPC Tunnel

more efficient Network

OPC Tunnel eliminates DCOM.OPC communicates via COM locally on computer and efficiently

passes data across network through standard TCP/IP port.

OPC Server OPC Client

COM COM

Verification- Tests from data center repeated using OPC Tunnel from

various vendors- Results proved the theory of DCOM as issue over WAN

S f O C

28

- Showed performance similar to or better than OPC connection on local LAN

- 1500 loops at 2 seconds scanned into PlantTriage across WAN with no data loss

- Server resources (CPU, memory, etc.) showed slight elevation using OPC Tunnel, but no real impactPerformance configuration cost etc vary by vendor- Performance, configuration, cost, etc. vary by vendor –personal preference and testing recommended

4/30/2008

15

Security- OPC Tunnel improves OPC security since DCOM is

eliminated• No user authentication for DCOM, OPC can be locked down

29

• Only connections from clients with OPC Tunnel client can connect to OPC Tunnel server

- Common problems related to DCOM configuration are eliminated

- OPC Tunnel uses a single TCP/IP socket• DCOM uses a range of TCP/IP ports for communication making

OPC diffi lt t t l fi llOPC difficult to control across a firewall• With OPC Tunnel, only the single TCP/IP port needs to be

allowed to pass through a firewall

Caveats- Considerations to note

• OPC Tunnels vary by vendor, so although an OPC Tunnel is a valid technical solution, specific software should be tested and confirmed to meet specific needs by preference

30

p y p• Some settings may need to be investigated for specific OPC

Tunnels by vendor, in particular settings for “data change mechanisms” in OPC Tunnel, which can interfere with the OPC Server callback mechanism used by PlantTriage

• In other words, values that change infrequently still need to scan and pass a timestamp (Forced Update option of PlantTriage may be needed)

• Possible for OPC Tunnels to have difficulty reconnecting• Possible for OPC Tunnels to have difficulty reconnecting automatically on occasion when servers reboot or disconnect

• Still dependent on bandwidth and performance of WAN to be viable

4/30/2008

16

Benefits / Conclusion- Need for DCOM for OPC communication eliminated- Complicated DCOM configuration and common errors

avoided

31

- Increased security and control of communication• DCOM security configuration headaches are gone• Good solution for OPC across firewall

- Performance and reliability of OPC data transfer across WAN greatly improved

- Enables centralized architecture for PlantTriage serversEnables centralized architecture for PlantTriage servers

Benefits / ConclusionProject Goals- Of original goals, following were realized:

• Centralized architecture for improved server support, cost, etc.

32

• Consistent design across sites• Robust data collection across WAN and through firewalls• Reliable access to engineering interface via Terminal Services• Reasonable web access for users on site networks• No impact to security and function of process network• Goals of all involved parties met