New Control Systems at KSTAR Compatible with ITER ... IAEA Technical Meeting on Control, Data...

11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,

08~12 May, 2017, Greifswald, Germany

New Control Systems at KSTAR

Compatible with ITER Standard Technologies

8th

May, 2017

Woongryol.Lee ([email protected]), Taegu.Lee, Giil.Kwon, Taehyun.Tak,

Myungkyu.Kim, Jinseop.Park, Yeon-jung.Kim, Jaesic.Hong, and KSTAR team

National Fusion Research Institute, Daejeon, Korea



Outline

Overview the control system implemented with

CODAC standard technologies

Expansion of KSTAR control system by using

CODAC technologies

Improve network infrastructure

New software components for system I&C

Device standardization

Summary



Evaluation and Demonstration of ITER CODAC Technologies at KSTAR

ITER CODAC technologies successfully demonstrated their capability for

Tokamak control at KSTAR through the discharge experiment

Main missions:

1. Develop the plant system I&C having real-time feedback control function with the CODAC technologies

2. Perform plasma experiments at KSTAR using CODAC compliant plant systems

Evaluate the new features of CODAC technologies according to new releases

HPN interfaces (DAN, SDN, TCN)

New features of device drivers/supports (NI PXI-6259, NI X-series I/O board, NI FlexRIO board)

Investigate and collect technical requirements for ITER operational applications to evaluate the initial release.

Plant Configuration Editor

Supervision System (SUP)

Data Archiving and Visualization tools



ITER standard density feedback control systems in KSTAR

MTCA.4

Milimeter-wave Interferometer

FUELFast

Controller

AIO

GPS PASS

new PCS

miniCODAC (DEV2)(ITER operation

application)

PO

N

DAN

Fast Controller

SDN

TCN

PO

N

SDN

Data archiver

DAN

PO

N

TCN

TCN

PO

N

TCN

PO

N

DAN

KSTAR Timing ITER Timing

CCS

HMI

PO

N

H-alpha

FIRFast

Controller

TCN

PO

N

DAN

SMBIFast

Controller

AIO

SDN

TCN

PO

N

DAN

TCN

AI

AI

AI

IRIG-B

IEEE 1588

SD

N

SDN

SDN

SDN

KO-DA VUV spectrometer

Operation Sequence Inerlock

• Schedule preparation & validation

• Operation Request Gateway(ORG)

• Supervision & Automation

Additional system for fast detection of L-H

transition

Additional function for Unified

Data Access (UDA)

ITER VUV spectrometer

from KO-DA



Experiment result

#14102 • The last shot of CODAC at 2015

CODAC PCS command [x 1e19]

Plasma density on KSTAR PCS [x 1e19]

Plasma current [x 100kA]

Vacuum pressure [x 1e-5 mbar]

NBI-A injected [about 1MW, 0.5~9.0]

NBI Shot: energy 70keV,

power 0.95MW, on time:

0.5s, width: 9.5s

IP min fault: 9.34s

Density value on D1-J4-MMWI



Experiment result

#14102 • Zoom in the PID enabled section

CODAC PCS command [x 1e19]

Plasma density on KSTAR PCS [x 1e19] Target Reference density

NBI-A injected [about 1MW, 0.5~9.0]

Controlled density

converged to target density.

target reference wave form.

PID enable period



Expansion of KSTAR control system by using CODAC technologies

Identified the reliability of CODAC technologies by performing multiple

contractual tasks

Development efforts are naturally concentrated on the system automation and

standardization

Continue the system renovation by adopting new technologies including the ITER

CODAC features

Main applied target:

1. Network Infrastructure

2. Software enhancement

3. Device standardization

Supervisory &

Automation

systems

Integrated

Information System

Database / Server

Data Acquisition Systems

Diagnostics

Local

System I&C

3rd Party

procurements

Plasma Control

System Network

infrastructure

Machine

Protection



Real-time network reconfiguration.

Native real time network

• Using a hardware-driven communication devices, i.e. RFM card. It has benefited from

low communication latency on dissimilar hardware platforms running different

operating systems, and lack of software overhead.

• Higher data rate and fast response time are required. Lack of diagnostic tools drive

us to find an alternate solution.

ITER Synchronous Data Bus Network

• Confirmed a stable operation during an evaluation task with IO. The popularity of

Ethernet product give lots of diagnostic tools.

• Decided to adopt an Ethernet-based high-speed control network, i.e. SDN

• Develop RT-GATE for gradually migrate from RFM to SDN

• Ready to use fast SDN to RFM

gateway system

• The real-time network data archiving

system has been developed, refer to the

Post P2/7/S7_3



10GbE SDN S/W

(Main control room)

Server room

Fueling facility (D2)Real time diagnostics

RFM network

Control device room (5th

)

Local SDN for Heating

NBI ICS

SDN-Gateway

Integrated

Interferometer

FUEL SMBI

RTP#1 RTP#2 RTP#3

Pellet

Injector

LCS#1 LCS#2 LCS#3 H-Alpha

Native KSTAR

Real time control

systems

Current design and progress of real-time network infra

SDN adoption is actively pursued in various control systems.



Complement of time synchronization system

KSTAR native time synchronization system

• Aims to synchronize clocks / triggers with a single clock source and supply them

directly to the controller. It has a star topology connection and provides a high-

precision time protocol.

Up to 5ns resolution, over 50 programmable multi trigger chain, 1~100MHz output clock

Embedded EPICS IOC in ARM® processor

Adopt ITER TCN for system clock synchronization to increase

application design flexibility

Custom protocol

L1, 1.575GHz

GPS Antenna

KSTAR NTPGPS receiver

(Grand Master Clock)

Central Timing Unit(CTU)

Multi-protocolphysical layer switch

Local Timing Unit(LTU)



IRIG-B DCLS 10MHz Ref. Clock


PTP

Layer 3 Precision Time Protocol(PTPv2) switch

ITER PTP

Clock/Trigger supplier

Host time synchronize

Custom

protocol via

Fiber



New software components for system I&C



Basic software architecture of control system I&C

We do not use an integrated solution (eg: sdd-editor) for control system development,

but we use standard libraries, templates, and example code.

KSTAR widely use a fundamental core library. The sfwLib has a device support structure

that provides non-blocking behavior through message queues.

Additional new features are implemented based on the CODAC standards.

• S/W components supporting absolute time based operation

• SDN support

• NI RIO libraries for Fast Interlock System

• Typical IOC based on “sfwLib”

• More than 20 control systems have been implemented using sfwLib

EPICS Standard LayerSystem environments

Template file

Predefined records

KSTAR Network Infrastructure

MDS+ Lib

Thread Manager

Infromation translator

Sequence handler

Custom libraries

State indicator

User function mapper

Operating System

Hardware



Real time core engine for advanced control

Demanding more flexible and stable methodology for implementation of

a real time application

• Inspired by a design philosophy of the ITER RTF, we adopt the function block based

high abstraction layer for implementation of a real time application

• To evaluate, we agilely prototyped an EPICS IOC which is configured with several

libraries include RT-core engine (TAC-engine) from IO

Software design concept, libraries, internal functions are shared with CCS basis

• By developing intuitive HMI system, we will provide a convenient communication and

configuration logging system

• Intel® Xeon® based high end server

• MRG Real-time Linux

• Base s/w package CCS v5.4

• KSTAR standard libraries

• tacLib for RT core

Fundamental of real time core node

Intel® Xeon® based HPC

MRG Real time based on Scientific Linux 6.x

MDS+ lib.

SDN lib. TCN lib. PON lib. LOG lib.

Real time core engine (TAC-engine)

KSTAR customized EPICS libraries

EPICS standard layer



Feasibility check on web based configurator and application

Thread A: 1KHz Thread B: 100Hz

kClose

kPusleChain

SDNPub

kPID1

SDNSub

SDNPub

kLinearWave

kOpen

kChopper

kLevelShift

Offset

Trigger

chain

Fuel commander

PID function

Real time

density

Prefill gas

Target density

Offset

neV command

PCSPCS

FUELFUEL

MMWIMMWI

V command

Rate

param.

level

Conceptual connection

diagram

Web-based dynamic configurator provides drag-

and-drop interface

(will continue to update)

• Measure the processing

time for each block

• Seen the peak noise up to

35us

Organized a single node executor based on current CCS v5.4 environment at KSTAR

MRG-R kernel 3.10.33-rt32.52 (FC based on CCS v5.4)

Intel® Xeon® E5-2620 v3 2.4GHz (Single Socket)

Chelsio - S320E-CT-CR (standard SDN card)

TCN driver with i350 chipset

Representative

measurement value



First step toward advanced control based on TAC

Second NBI is designed to provide off-axis CD by 3 beam injection with

vertical steering on the equatorial plane

Parameters Designed

Beam energy 100 keV

Beam power 6 ㎿ ( 2.0 ㎿ × 3 ea)

No. of Ion source 3 EA

Beam divergence 1°<

Beam target position,

Tangent radius (Rt) 1.6 m

Vertical slant angle 5.5°

Beam size (@outlet) 450 ㎜ × 130 ㎜

Beam input port H port

Max Operation time 300 sec

Focal position (m) 10.2 m

Beam modulation frequency 100 ㎐

Beam line

Ion source Power Supply

Ion source in vertical plane



Infrastructure of Integrated Control System for NBI2

Manipulate each P/S Local Control Systems (LCS) and Beam line LCS

Sequential logic control according to the beam initiation time

Generate predefined/arbitrary reference waveform synchronize to the PCS command

Power Supply Local Control

System

Machine Network(TCP/IP)Real Time Network(RFM)Timming Network(TCN)Interlock Network(Slow)Interlock Network(Fast)

KSTAR MDSPlus

KSTAR PCS KSTAR CCS

RT-Module Server

PCIe

CPU(Xeon)

RFM( PMC5565)

10GbENetwork

Card

1GbE Network

Card

Synchronous Data Network(Fast)

Beam Line Local Control System(PC)

Integrated Control System(PC)ICS-Module

PCIe

CPU(Xeon)

1GbE Network

Card

KSTAR LTU KSTAR LTU KSTAR LTU KSTAR LTU

Local RFM Network

3 * MTCA.4 based LCS

• NBICS : Communicate and

process epics data

Control period : 1KHz

Network : 1Gb based channel

access

• NBRT : Communicate and

process data in real time


Network : RFM(or SDN),TCN

Time Synchronization : TCN and

KSTAR TSS

• RT SIM : Generate simulated

reference waveform




Device standardization for system I&C



Shift to the next generation control device

An extension of MTCA (Micro Telecommunications Computing Architecture) initiated by the

Physics community, MTCA.4, provides high precision clock and trigger through the

backplane, intelligent system management interface, modular structure of high-speed

links, and allows flexible reconfiguration of system functionality.

AMC RTM

Front Back

Analog Digital

Zone 3

AMC backplane

Distribution of clocks and triggers within a shelf

12 slot backplane topology

6 slot chassis for

laboratory or diagnostics

12-Slot fully redundant



Device standardization

CODAC recommend several platforms for a Fast I/O system depend on performance grades.

Refer to “ITER Catalog of I&C products - Fast Controllers”(IDM: 345X28)

For the systematic standardization of a fast

controller at KSTAR, we adopt the MTCA.4 standard. Host side adaptor

Target side uplink

module

MTCA.4 Optical PCI Express 4x

Link to an external CPU dedicated

for μTCA.4 systems (Max. 20GT/s)



Develop a general-purpose FPGA based control device

Diverse local system require different ADC rates and can be categorized several

performance grade. The most demand is 10KSPS ~ 2MSPS sampling speed.

Through practical experience and strategic factors, we are considering

manufacturing a common device.

We developed bespoke KSTAR Multifunction Control Unit (KMCU) with benefit of

modular design

Item KMCU-Z30 (K-Z30) KMCU-Z35 (KZ35)

FPGA XC7Z30T XC7Z35T

Memory 1GB DDR3

Front Interface - 1xLPC FMC site

2x SFP+ cadge 1x SFP+ cadge

Ethernet RJ45

microSD for Linux

LEMO CLK/TRIG

Backplane

interface

PCIe x1(port4) Gen 2 PCIe x4(port 4-7)

Ethernet port-0

TCLK, FCLK

Zone 3 Class D1.0.

http://www.google.co.kr/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&ved=0ahUKEwjv3Kn80NXLAhVoJKYKHVfsAM8QjRwIBw&url=http://www.desy.de/&bvm=bv.117218890,d.dGo&psig=AFQjCNG8MVDMV1m_E4tzwrxue2Igt7n0IA&ust=1458782102891916

http://www.d-tacq.com/



MicroTCA.4: DAQ use case

Several data acquisition systems were renovated or newly developed using

KMCU-Z30

Renovate magnetic diagnostics (unlimited acquisition time), Mirnov coil diagnostics,

and H-alpha monitoring system (higher ADC rate)

New DAQ for a multi chord photo-elastic modulator based MSE system

System Specification Remark

DDS1 (MD &Probe) Max 1MSPS, 576ch, 9*kz30, 16bit PCIe uplink to external host

MSE Max 2MSPS, 16bit, 32ch KSTAR Multichord MSE

Max 2MSPS, 16bit, 32ch PPPL/MIT MSE-MSLP

MC 2MSPS, 16bit, 64ch Front aurora interface

H-Alpha 1MSPS, 16bit, 32ch Front aurora interface

Interferometer 125MSPS, 16bit, 12ch (COTS) Real time density calculation

DDS1

KSTAR MSE

MC

H-Alpha

MSE-MSLP



MicroTCA.4: High end system use case

Use commercial products for a high performance application demanding

lots of FPGA slices

Integrated interferometer system

Resource Available MMWI

Utilization MMWI + FIR Utilization

LUT 254200 63991 (25.17%) 69663 (27.40%)

LUTRAM 90600 13283 (14.66%) 13433 (14.83%)

FF 508400 99256 (19.52%) 109029 (21.45%)

BRAM 795 44.50 (5.60%) 44.50 (5.60%)

DSP 1540 6 (0.39%) 14 (0.91%)

Device : xc7k410tffg900-2

Details available at the

Poster P2/3/05

AMC523

μTCA.4 AMC, Dual DAC 16-Bit @

250 MSPS

MRT523

μTCA.4 RTM For AMC523, 12 CH

ADC 16-Bit @ 125 MSPS

Packet Gen & Control

Period Flag Gen

PCI ExpressDMA

S/W

LPF

Zero CrossPoint Detect

Period Lock Check

Phase Compare

&FrequencyEstimate

Phase Average

Over Sample

Data Gen

LPF


Period Lock Check

Over Sample

Data Gen

Ref Wave

Prob Wave

DegreeConvert

&Remove

Channel 1

Channel 2


Period Lock Check

Phase Compare

&FrequencyEstimate

Phase Average

Over Sample

Data Gen


Period Lock Check

Over Sample

Data Gen

DegreeConvert

&Remove



MicroTCA.4: High end system use case

Compatibility of 3rd party IP logic inside the FlexRIO : provides design flexibility

Measured by KSTAR system Measured by FlexRIO (DAN plot)

• Out put data has 0 ~ -360

phase information

• Makes 1280 data counts per

1msec

3rd-party IP

MMWI Interferometer

PXI Express

MTCA.4 based MMWI

D1-J4-MMWISDN

DAN

Bus ext.

PON

Bus ext.

TCN

FlexRIOPXIe-7966R / 5734



Integrated information system



KSTAR Data Integration System (KDIS)

The data integration system of KSTAR supporting a wide range of KSTAR

pulse operation automation with off-line data process.

• Scheduled process of stream and batch data according to KSTAR events with user

interface service, including H/W and S/W infrastructure, applications, and libraries.

KSTAR

Systems

KSTAR

Pulse Automation

System

On-line Operation Process

KSTAR

Main Storage

(MDSPlus,

Channel Archiver

, ETC)

CCS

ICS

PCS

NBI

ECH

MPS

PASS

Data Stream

(EPICS, SDN,

Logs, RestAPI,

ETC)

User Interface (OPI, Web Service, ETC)

SIS

KDIS

System Infrastructure

Distributed Clustering Platform

Clustered Servers

(Resource Manager)

KSTAR Integrated Stream

Processing Framework and

Applications

Schedule Algorithm

and Stream

Processing Logic

KSTAR Integrated Batch

Processing Frameworks and

Applications

KSTAR Integrated User Interface

Web Application Service

Optional Back-End Storage

(HDFS, NoSQL, RDB, ETC…)

Data Processing and

Analysis Algorithm

Integrated web view

Supporting Data

interface



KDIS Data Cycle Orchestration

Design of data cycle orchestration with developed applications

• KSTAR operation related data cycle ecosystem synchronized with KSTAR event

KSTAR

Stream Processing

Framework

Applications for

KSTAR Event

Stream

KSTAR

Data Processing

Framework

Applications for

KSTAR Data

Processing

Clients

KDIS Main

Web Service

Data Source and Storage

MDSPlus

MemoryDB

MessageQ HDFS

ETC Retrieving

Task Launch

Stream Processing,

Data Archiving

EPICS

RDB

Data

Exchange

NoSQL

• KSTAR Shot Summary

• Shot Fault Analysis Application

• EPICS PV Snapshot for

Operation Parameter

• TSS Parameter Snapshot

• TSS Template Automation

• Processing KSTAR

Pulse Automation

System Event

• Supports PV snapshot View

• Supports Summary View

• Statistical Information Summary view

• Supports Logbook and Integrated view

Details available at the Poster

P1/2/S2_1



Summary and Future work

ITER CODAC technologies were successfully evaluated in the real-life

environment of an operational tokamak.

KSTAR has been investigating the next generation control platform and

new standards both hardware and software infrastructure.

By adopting TCN and SDN, we decided to increase the performance of

control infrastructure and increase the flexibility of system design.

We will use the core engine to reflect the real-time framework design

concept and use it for integrated control of the second NBI system.

KSTAR adopt the MTCA.4 standard for the systematic standardization of

a fast controller and real time diagnostics.

A KSTAR data integration and information system based on the big-data

platform is under development and will be serviced in 2017.



Thank you for your attention. Q & A

New Control Systems at KSTAR Compatible with ITER ... IAEA Technical Meeting on Control, Data...

Documents

Transcript of New Control Systems at KSTAR Compatible with ITER ... IAEA Technical Meeting on Control, Data...