Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered...
Transcript of Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered...
![Page 1: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/1.jpg)
Qualitätssicherung und Testen im IoT
![Page 2: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/2.jpg)
Outline
• Our Context
• CPIoT Fundamentals– Test objectives, test levels, risk analysis– Test automation, architectures, techniques– Security, interoperabity, performance, certification
• IoT Testware– Eclipse project– ETSI standardization
![Page 3: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/3.jpg)
© M
ichael Zalewski/ Fraunhofer FO
KUS
We connect everythingsecure, reliable, trustworthy
Fraunhofer FOKUSInstitute for Open Communication Systems
![Page 4: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/4.jpg)
4
BERLIN CENTER FOR DIGITAL TRANSFORMATION
![Page 5: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/5.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 5
Risk analysisTestlevels
Test-architectures
Test techniques
SummaryCertification
Test automation
Security,Interop., Performance
Test-objectives
![Page 6: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/6.jpg)
Quality attributes –Specific priorities for IoT systems
Reason Prioritized quality attributes for IoT
Specific (distributed) architecture
InteroperabilityPerformance and capability Adaptability Robustness and resilience
Interrelated lifecycles and interdisciplinary nature of IoT
CompatibilityMaintenancePortability
Interrelated and wide ranging business processes which can be represented in IoT systems
Functional security (safety)IT-SecurityPrivacyUsabilityEthical aspects
CPIoT V1.1 - Copyright ASQF e.V. 2019 6
![Page 7: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/7.jpg)
Prioritization of test objectives
The priority of test objectives… • relates to the priority of the quality attributes to be tested• … must be continuously evaluated over the lifecycle of the IoT system • … and where necessary adjusted or extended
Useful subdivisions of IoT test requirements and test objectives are• Process• System/component• Communication protocols
CPIoT V1.1 - Copyright ASQF e.V. 2019 7
![Page 8: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/8.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 8
Risk analysis
Test-architectures
Test techniques
SummaryCertification
Test automation
Security,Interop., Performance
Test-objectives
Testlevels
![Page 9: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/9.jpg)
Long operational life
• After the usual system and acceptance tests could follow a very long operational life. new test level „operation“
• Parts of the IoT solutions may be included Updates + Tests during „operation“
CPIoT V1.1 - Copyright ASQF e.V. 2019 9
![Page 10: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/10.jpg)
General test levels
Operational / Diagnosis test
IoT specific
CPIoT V1.1 - Copyright ASQF e.V. 2019 10
Requirements specification
Functional system design
Technical system design
Module design
Module coding
Acceptance test
System test
Integration test
Module test
V-Model
VerificationValidationConstruction & integration
![Page 11: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/11.jpg)
How do we deal with certificates after a security gap has been revealed?
IoT Security ?CRACK
CPIoT V1.1 - Copyright ASQF e.V. 2019 11
Earlier certificates issued at product release lose their meaning New liability questions after security incidents
![Page 12: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/12.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 12
Test-architectures
Test techniques
SummaryCertification
Test automation
Security,Interop., Performance
Test-objectives Risk analysisTest
levels
![Page 13: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/13.jpg)
Inclusion of a risk analysis
Security-criticalIoT-Systems
Special consideration of
criticality
Risk analysis for the whole IoT system
Risk analysis in the test development process: e.g., ETSI EG 203 251 Risk-based Security Assessment and
Testing MethodologiesDerivation and priorization of test objectives
CPIoT V1.1 - Copyright ASQF e.V. 2019 13
![Page 14: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/14.jpg)
• Provide justification for the absence of potential vulnerabilities.• Provide justification for the functional correctness of treatment scenarios
and countermeasures.• Discover unknown risk factors (i.e. vulnerabilities)• Provide feedback for reassessing risk values
Risk Identification
Test Planning & Test Design
Test Execution
Test Results
Risk-based testing (Example: Security)Inclusion of test experiments
CPIoT V1.1 - Copyright ASQF e.V. 2019 14
![Page 15: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/15.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 15
Test-architectures
Test techniques
SummaryCertificationSecurity,Interop., Performance
Test-objectives
Test levels Risk analysis
Test automation
![Page 16: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/16.jpg)
Need for test automation
Assuring the quality in the lifecycle is
linked with a high degree of regression
The factor time to
market has a fundamental
and continuoussignificance
High complexity
and dynamicof the system
context for the IoT product
High risk of mistakes in a
manual procedure
CPIoT V1.1 - Copyright ASQF e.V. 2019 16
time
costs
Automated tests
![Page 17: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/17.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 17
Test techniques
SummaryCertificationSecurity,Interop., Performance
Test-objectives
Test levels Risk analysis
Test automation
Test-architectures
![Page 18: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/18.jpg)
Which IoT test architectures are there?
CPIoT V1.1 - Copyright ASQF e.V. 2019 18
Device-basedIoT test architectures
E.g., for testing Retroboxes or Gateways
Service-based IoT test architectures
E.g., for the data-oriented testing of Dashboards in the Cloud
Infrastructure-based IoT test architectures
E.g., for testing oneM2M functional elements
Since IoT systems are distributed, many distributed test architectures and corresponding process strategies can be applied.
Example: Efficiency improvements by virtualization of the entire test system
![Page 19: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/19.jpg)
Example: various test systems
CPIoT V1.1 - Copyright ASQF e.V. 2019 19
![Page 20: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/20.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 20
SummaryCertification
Test-objectives
Test levels Risk analysis
Test automation
Test-architectures
Security,Interop., Performance
Test techniques
![Page 21: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/21.jpg)
Integration of existing testing techniques
IoTTesting
Software Testing
System Testing
Security Testing
Test Automation
Protocol Testing
Testing methods and testing tools are NOTfundamentally new
they are a special selection of established approaches• considering the IoT-specific characteristics of the
SUT• considering the specifics from the
requirements analysis
Main focus for practical applications• Security• Interoperability• Performance
CPIoT V1.1 - Copyright ASQF e.V. 2019 21
![Page 22: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/22.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 22
SummaryCertification
Test-objectives
Test levels Risk analysis
Test automation
Test-architectures
Test techniques
Security,Interop., Performance
![Page 23: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/23.jpg)
Combination of techniques for security testing
(Model-Based)
Security Testing
Risk analysis
Functional Testing(security functions)
TestAutomation
(e.g., TTCN-3)
Fuzz Testing(use of invalid data)
CPIoT V1.1 - Copyright ASQF e.V. 2019 23
Ina Schieferdecker, Model Based Security Testing: Selected Considerations (Keynote) Sectest 2011, Workshop on the 4th IEEE International Conference on Software Testing, Verification and Validation Berlin, Germany
![Page 24: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/24.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 24
SummaryCertification
Test-objectives
Test levels Risk analysis
Test automation
Test-architectures
Test techniques
Security,Interop., Performance
![Page 25: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/25.jpg)
What role does interoperability play in IoT ?
• IoT-Systems and their components can differ strongly different and they can originate from different manufacturers.
• Example: – Mass-produced sensors and actuators – Specially configured gateways as well as other devices with dedicated
protocols / versions– Wide variety of end devices for display of analyzed data.
• Example Smart Home:LED-Lamps, Amazon Echo, iPhone, switchable intermediate plugs, Telekom-Starter Package, window sensors, Smart Home heating package, …
Are the various systems and components able to interact with each other?
25CPIoT V1.1 - Copyright ASQF e.V. 2019
![Page 26: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/26.jpg)
Overview interoperability test
Test object Technique What is tested?Technicalinteroperability
Basic tests of connectivity and communications protocols
Coupling of hardware / software components in order to ensure basic communication
Syntacticinteroperability
Targeted checks of messages and the syntax of abstract data formats.Use of encoders and decoders
Correct use of syntax for e.g., HTML, XML or ASN.1 data structures
Semanticinteroperability
Execution of sample scenarios and user scenarios, possibly with support from standardized Use Case catalogs
Checks on whether the implementation of the interconnected components/systems follows a common interpretation.
CPIoT V1.1 - Copyright ASQF e.V. 2019 26
![Page 27: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/27.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 27
SummaryCertification
Test-objectives
Test levels Risk analysis
Test automation
Test-architectures
Test techniques
Security,Interop., Performance
![Page 28: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/28.jpg)
Time behavior Resource utilization
Capacity Scalability
Several types of performance to test
under normal andexceptional conditions
Reminder about performance tests
CPIoT V1.1 - Copyright ASQF e.V. 2019 28
![Page 29: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/29.jpg)
What needs to be considered for performance tests?
CPIoT V1.1 - Copyright ASQF e.V. 2019 29
The configuration of the test system:
Specialties for IoT:
• Many interfaces
• Distribution and synchronization of the scenario control across all test components
• Manufacturer-independent Use Cases for test (standardization!)
• Simulation and reaction from devices/sensors instead of users
• Irregular/ uncommon status changes (caused by environment)
• Unsteady connectivity
• Interaction of variouscommunications protocols
• Consideration of specific hardware characteristics
• Delays in Cloud transfers (Test access points)
![Page 30: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/30.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 30
Summary
Test-objectives
Test levels Risk analysis
Test automation
Test-architectures
Security,Interop., Performance
Certification
Test techniques
![Page 31: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/31.jpg)
Fundamental challenges (1)
Basis for the certification:guiding principles, norms and standards
Collection of high-level norms that are still too immature
Currently there is no „IoT Standard“
Main focus for IoT:checking IT-Security
Checking products, processes and services
Functional security requirements, stability
Conformity and vulnerability to errors in communication protocols typically used in IoT
CPIoT V1.1 - Copyright ASQF e.V. 2019 31
Product certification =
Checking conformity to the requirements stated in standards and additional normative documents (DIN EN ISO/IEC 17065)
![Page 32: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/32.jpg)
Fundamental challenges (2)
CPIoT V1.1 - Copyright ASQF e.V. 2019 32
Fundamental questions in a certification
Which criteria shall be applied? Test, evaluation and checking approaches? Which part of an IoT product or an IoT solution shall be included?How shall the checks be performed and what must the certification authority achieve?General validity / level of detail of requirements: check/test(pass) criteria
Practical questions
What shall be the certificate‘s period of validity?
What happens if a security incident occurs?
What significance do updates and patcheshave for the certificate?
The goal is to establish confidence in a part of the IoT
![Page 33: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/33.jpg)
Fundamentals of IoT Testing
CPIoT V1.1 - Copyright ASQF e.V. 2019 33
Test-objectives
Test levels Risk analysis
Test automation
Test-architectures
Security,Interop., Performance
Test techniques
Certification Summary
![Page 34: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/34.jpg)
Summary
Test Objectives
For all layers of the architecture
Inclusion of risk analysis
Priorization!
Test Levels
Test and Monitoring „in operation“ due to longer operational lives and updates
Automation
Thorough test
Regression
Distributed architecture
High data volumes
IT-Security
Overall consideration
For all layers of the architecture
Many techniques
Certification
Missing standards
Lack of generally accepted check criteria
CPIoT V1.1 - Copyright ASQF e.V. 2019 34
![Page 35: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/35.jpg)
THE IOT-TESTWARE PROJECT
The Testware
![Page 36: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/36.jpg)
36
TRENDS IN IOT
![Page 37: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/37.jpg)
37
• Supplement to running and active Eclipse projects− Paho, OM2M, Titan…
• Running project at Eclipse Foundation:https://projects.eclipse.org/projects/technology.iottestware− TTCN-3 test suites for CoAP, MQTT, OPC-UA, LoRa?
• Assured licenses for users
• Currently in cooperation withrelayr GmbH, Ericsson, LAAS/CNRS, itemis AG, Spirent Communications, Easy Global Market, Iskratel/Sintesio, …
THE ECLIPSE PROJECT
![Page 38: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/38.jpg)
38
Take available software and tools …
… and adding public testuites as a result of insights from IoT testing:
IOT-TESTWARE
…
https://projects.eclipse.org/projects/technology.iottestware
![Page 39: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/39.jpg)
39
IOT TEST LANGUAGE
Testing and Test Control Notation
• Specification and implementation of all kinds of black-box tests
• Component-based approach
![Page 40: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/40.jpg)
40
MQTT EVALUATION
Broker PASS FAIL INCONC
Name Version# % # % # %
Mosquitto 1.5.5 90 85,71% 11 10,48% 4 3,81%
HiveMQ CE 2019.1 86 81,90% 15 14,29% 4 3,81%
lannister v0.9.8 68 64,76% 33 31,43% 4 3,81%
Apache ActiveMQ 5.15.9 58 55,24% 43 40,95% 4 3,81%
Aedes 0.38.0 58 55,24% 43 40,95% 4 3,81%
RSMB 1.3.0.2 50 47,62% 51 48,57% 4 3,81%
Mosca 2.8.3 43 40,95% 58 55,24% 4 3,81%
Apache Apollo 1.7.1 34 32,38% 70 66,67% 1 0,95%
April 2019 – 105 Test Cases
![Page 41: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/41.jpg)
THE IOT-TESTWARE PROJECT
Standardization
![Page 42: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/42.jpg)
42
• Working Group TST develops IoT test catalogues and specifications (not covered elsewhere)
• The types of testing includeconformance, interoperability, security and performance testing
• The initial technical focus will be:− IoT network layer
(communication protocols, node connectivity, edge computing etc.),− Basic security of IoT devices
ETSI TC MTS
![Page 43: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/43.jpg)
43
MTS TST WORK PROGRAMME
IEC 62443-4-2
CoAP
MQTT
LoRaWAN
Vul. database
https://portal.etsi.org/tb.aspx?tbid=860&SubTB=860
![Page 44: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/44.jpg)
44
IOT-TESTWARE - BIG PICTURE
…
ETSReporting
Logging
TSS: Test Suite StructureTP: Test Purpose
ATS: Abstract Test SuiteETS: Executable Test Suite
TSS TP ATS
![Page 45: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/45.jpg)
45
Advanced testing technology:
Open source IoT-Testware (code):
External (open source) SW
Standardized IoT test purposes:
SUMMARY
![Page 46: Qualitätssicherung und Testen im IoT - ASQFIoT test catalogues and specifications (not covered elsewhere) • The types of testing include conformance, interoperability, security](https://reader033.fdocuments.us/reader033/viewer/2022042413/5f2ce3bbafa2b22393436726/html5/thumbnails/46.jpg)
46
CONTACTS
Thank you for your [email protected]@fokus.fraunhofer.de
https://www.fokus.fraunhofer.de/en/sqc