Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing &...
Transcript of Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing &...
![Page 1: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/1.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Holistic system integration and validation procedure developed in ERIGrid
Cyndi MoyoCenter for Energy – Electric Energy SystemsAIT Austrian Institute of Technology, Vienna, Austria
Workshop “Holistic System Validation, Methods and Tools, Free Access to Best Smart Grid Laboratories”
October 5, 2017, Amsterdam, The Netherlands
![Page 2: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/2.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Coordinated Voltage Control (CVC)
05.10.2017Cyndi Moyo, AIT Energy 2
Validate this!
![Page 3: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/3.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Holistic Testing & Validation
05.10.2017Cyndi Moyo, AIT Energy 3
"Holistic testing is the process and methodology for the evaluation of a concretefunction, system or component (object under investigation) within its relevantoperational context (system under test), corresponding to the purpose of investigation”
Holistic Validation Procedure(an interactive process)
![Page 4: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/4.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Holistic Test vs. Component Test
Component Test Example: inverter MPPT test, anti-
islanding
No interactions with the system
Usually open loop test (predefined voltage, frequency; setpoints are applied to the hardware under test)
Holistic/System Test Combining several tests
(testing process)
Using simulations
Testing a system rather than just component
05.10.2017Cyndi Moyo, AIT Energy 5
DC AC
PV simulator AC gid simulatorHardware inverter
under test
RTDS Simulated Network
PV inverterPV simulator
Central Controller(optimization)
PHIL
![Page 5: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/5.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
HOLISTIC TEST DESCRIPTIONThe basics
05.10.2017Cyndi Moyo, AIT Energy 6
![Page 6: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/6.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Holistic test description
05.10.2017Cyndi Moyo, AIT Energy 7
THRE
E le
vels
ofsp
ecifi
catio
n
Map/split
Test Case (TC)
Use CasesTest Objective(s)
ExperimentSpecification
Map/split
ExperimentSpecification
ExperimentSpecification (ES)
Test specificationTest specificationTest Specification (TS)
Generic
Specific
Lab
![Page 7: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/7.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Holistic testing procedure –different mapping steps
05.10.2017Cyndi Moyo, AIT Energy 8
5
6
4
3
2
1
RI 1 RI 2 RI n
ExperimentSpecification
Exp. Spec.
ExperimentSpecification
Exchange of data and results
Holistic Test Case
Sub-test specification
Sub-test
spec. 2
Sub-test specification n
Division into individual (sub-)tests
1
3
5
6
7
Experiment in RI n
Exp. in RI 1
…
…
…
Exp.in RI 2
Use Case(s)Use Case(s)
holistic test
evaluation
test refinement
rese
arch
infr
astr
uctu
re (R
I) ca
pabi
lity
prof
iles Scenario & Generic
System ConfigurationUse Cases
2
4
Test ObjectiveMapping from SC, UC,
TO to holistic test case
Mapping of holistic test case to (sub-) test specifications, i.e. specific (sub-) test
system(s)
Mapping of (sub-)tests to RI and specify
experiments
Mapping between tests resp. experiments
Mapping (preliminary) results to test adjustments
Mapping of results of experiments to
represent holistic result
x – mapping step
532x Process steps
![Page 8: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/8.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Validate this!WHY TO TEST?WHAT TO TEST?WHAT TO TEST FOR?HOW TO TEST?
Key Questions to be answered for test specification:
05.10.2017Cyndi Moyo, AIT Energy 9
![Page 9: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/9.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Validate this!WHY TO TEST?WHAT TO TEST?WHAT TO TEST FOR?HOW TO TEST?
Key Questions to be answered for test specification:
05.10.2017Cyndi Moyo, AIT Energy 10
![Page 10: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/10.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Test System & Domain
05.10.2017Cyndi Moyo, AIT Energy 11
System under Test (SuT): is a system configuration that includes all relevant properties, interactions and behaviors (closed loop I/O and electrical coupling), that are required for evaluating an OuI as specified by the test criteria.
Object under Investigation (OuI): the component(s) (1..n) that are subject to the test objective(s).
Remark: OuI is a subset of the SuT.
Domain under Investigation (DuI): Identifies the domains of test parameters and connectivity relevant to the test objectives.
OuI
![Page 11: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/11.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
SuT
measurements
Test System Functions
05.10.2017Cyndi Moyo, AIT Energy 12
Functions under Test (FuT): the functions relevant to the operation of the system under test, as referenced by use cases. Function(s) under Investigation (FuI): the referenced specification of a function realized (operation-alized) by the object under investigation.
Remark: the FuI are a subset of the FuT.
OuI
DER P,Q control
optimization in the controller
state estimation
OLTC tap controlmeasurementsmeasurements
Use Cases define FunctionsOf Systems (IEC 62559 approach)
![Page 12: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/12.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
WHY TO TEST?WHAT TO TEST?WHAT TO TEST FOR?HOW TO TEST?
Key Questions to be answered for test specification:
05.10.2017Cyndi Moyo, AIT Energy 13
![Page 13: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/13.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Purpose of Investigation (PoI)
Verification
Validation
Characterization
05.10.2017Cyndi Moyo, AIT Energy 14
Scoring / Performance
Modeling /Understanding
Test objectives/PoI: Characterization and validation of the DMS controller1. Convergence of the optimization
(validation)2. Performance of the optimization
under realistic conditions (characterization)
3. Accuracy of the state estimation (characterization)
![Page 14: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/14.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Designing Test CriteriaDetailing Sequence
Test objective PoI Test Crit.
Test criteria: How to break down the PoIs?
– Target metrics (criteria): list of metrics to quantify each PoI
– Variability attributes:controllable or uncontrollable parameters to “disturb” SuT
– Quality attributes (thresholds): test result level or quality of the TM required to pass or conclude the testing.
05.10.2017Cyndi Moyo, AIT Energy 15
Target metrics: 1. 1.1 convergence (when/how
often?), 1.2. How fast?, 1.3. solution quality
2. 2.1 Voltage deviation 2.2 number of tap changes, 2.3 network losses
3. Voltage, P, Q estimation errorsVariability attributes: Load patterns (realistic, annual variation; applies to criteria 1-3); Communication attributes (packet loss, delays)Quality attributes (thresholds):
“1.2: convergence within 2 sec” (validation)“3.* estimation quality characterized with confidence 95%” …
![Page 15: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/15.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
WHY TO TEST?WHAT TO TEST?WHAT TO TEST FOR?HOW TO TEST?
Key Questions to be answered for test specification:
05.10.2017Cyndi Moyo, AIT Energy 16
![Page 16: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/16.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Test Specification & Design Given:
Purpose of Investigation (PoI) & Test Criteria
System & Domain categories and relations
To Specify:
Precise system (specific system configuration)
Which variables to manipulate & which to measure
How to quantify the test metrics (based on test data)
Sampling of the input spaces (design of experiments methodology)
Combination and interpretation of the outputs
The test design / procedure.
Mapping to actual lab setup (experiement setup)
05.10.2017Cyndi Moyo, AIT Energy 17
Research Infrastructure (RI)
Experiment Specification
Test CaseSuT, FuT, PoI, TC
Test SpecificationTest Design, I/O
1
6
7
Experiment in RI
test evaluation
test refinement
TO
rese
arch
infr
astr
uctu
re (R
I) ca
pabi
lity
prof
iles
SC UC
2
4 map
3
5
Test SpecificationTest Design, I/O
![Page 17: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/17.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Scoping & specification of
test system.
Separate specification
of lab implementation
Detailing test setup &Mapping to the Lab
05.10.2017Cyndi Moyo, AIT Energy 18
![Page 18: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/18.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
FIRST APPLICATION EXPERIENCE
05.10.2017Cyndi Moyo, AIT Energy 19
Can I just say, that it's very nice to get these questions sorted out now, rather than when you're sitting down and have to implement something. You would then usually go "Oh shit, how does this work again?“ – ERIGrid participant
![Page 19: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/19.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
ERIGrid Transnational Access: Preparation & Documentation
External Lab users apply description procedure
E.g. DiNODR – distribution network oriented application of demand response – currently ongoing in SYSLAB
“The preparation work helped us a lot. Except minor changes in the plan and configurations due to a number of device, communication and control unavailabilities, we are following our test and experiment specifications. The template is also useful for our user team to exchange ideas in an organized and effective way. “ - Alparslan Zehir (DiNODR)
05.10.2017 Cyndi Moyo, AIT Energy 20
![Page 20: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/20.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Collaboration with other Projects
ELECTRA IRP – Web-of-Cells (WoC) concept
– Large set of use cases (distributed control)
– Concurrent development and lab implementation
– Specification of test cases andexperiments
SmartNet – TSO/DSO interaction
– Different interaction schemes
– Proof-of-concept lab validationand field testing
– Specification of test cases andexperiments
05.10.2017Cyndi Moyo, AIT Energy 21
![Page 21: Holistic system integration and validation procedure ......Oct 05, 2017 · Holistic Testing & Validation Cyndi Moyo, AIT Energy 05.10.2017 3 " Holistic testing is the process and](https://reader034.fdocuments.us/reader034/viewer/2022051923/6011a9d5019aab14cd1b7da2/html5/thumbnails/21.jpg)
© The ERIGrid ConsortiumEU H2020 Programme GA No. 654113
Summary and future work
A clear vision for “holistic validation”
Initial results
– 3-level – test description template & guidelines
– Multi-Domain System Configuration description (CIM compatible)
– Several successful applications & encouraging feedback
Future work
– Further exemplify, simplify & detail description method
– Develop & apply full holistic validation procedure
05.10.2017Cyndi Moyo, AIT Energy 22