Interoperability in the DNA of standards: Ensuring interoperability between building blocks
description
Transcript of Interoperability in the DNA of standards: Ensuring interoperability between building blocks
![Page 1: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/1.jpg)
1
Anthony Wiles, ETSI Protocol and Testing Competence Centre (PTCC)
@metis Kick-off meeting
30-31 March 2005
Interoperability in the DNA of standards: Ensuring interoperability between building blocks
![Page 2: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/2.jpg)
2
In Pursuit of Interoperability
Ultimate aim of ICT standardisation is interoperability
Likelihood of interoperability is increased with Well-defined, accurate and unambiguous standards Systematic testing of products based on those standards
ETSI produces Base Standards and Test Specifications
Development Testing
Test Specs
Specification
![Page 3: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/3.jpg)
3
Unique Resources Available to ETSI Technical Bodies
TC MTS (Methods for Testing and Specification) Develops specification methodologies and techniques http://portal.etsi.org
ETSI PTCC (Protocol and Testing Competence Centre) Supports ETSI Technical Bodies with the development of:
• technology standards: protocols, services, APIs etc.• test specifications
http://www.etsi.org/ptcc
ETSI Plugtests Service Organises interoperability events http://www.etsi.plugtests
![Page 4: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/4.jpg)
4
How Does the PTCC Help?
Assist ETSI Technical Bodies on the use of state of the art techniques for Specification, validation and testing Good working practices (Standards Engineering)
Pragmatic and flexible approach Based on experience
Help to develop usable methodologies For ETSI’s current and emerging needs
Knowledge transfer Quality through Continuity
![Page 5: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/5.jpg)
5
Who PTCC Supports
Technical Bodies (TB) Technical Committees ETSI Projects Partnership Projects etc.
Chairmen, Rapporteurs, Individuals Working Groups (WG) STFs (Specialist Task Forces)
PTCC budget for test specifications (15-20 STFs per yr)
ETSI Secretariat
![Page 6: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/6.jpg)
6
PTCC Support in Standards Development
Conformance Test Specifications
Interoperability Test Specifications
(Unit) Conformance Testing
Interoperability Testing
Specification, validation and simulation
Products mature from prototypes to commercial products
ETSI TB: Development of base standards
ETSI PTCC: Support on Protocol Specification and Testing
Good practice, pragmatic application of methodology, text complemented by the use of specification languages: UML, SDL MSC, ASN.1, IDL, XML etc.
Planning, methodology, test specifications, TTCN, ISO-9646, TS 102 237
Certification
Industry
![Page 7: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/7.jpg)
7
Typical PTCC Areas of Activity
GSM and 3G (UMTS) terminals Wi-Fi: HiperLAN/2, HiperMAN, HiperACCESS Cordless phones: DECT Access terminals: FSK, SMS DSL Splitters ISDN, Broadband ISDN OSA (API, IDL) – web services NGN VoIP: H.225, H.248, H.245 (ITU), SIP (IETF) SIGTRAN Smartcards DECT IPv6: Core, IPSEC, Mobility, v4 -> v6 DSRC (ITS) DMR ... future: More telematics/ITS, more security?
![Page 8: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/8.jpg)
8
Design for Interoperability
Interoperability takes place on external interfaces Normative (mandatory) features Options
• Consequences must be clear
Abnormal behaviour should be well-defined Robustness
Never specify how product is to be implemented internally Different levels of abstraction e.g., 3-stage approach
Methods like SDL, ASN.1, XML, MSC and UML used to specify these interfaces Data transferred on protocol interfaces
• syntax and encodings (ASN.1, XML)
Behaviour of modules as seen from these interfaces
TTCN and test methods used for black box testing No need to know how product is implemented External interface is tested according to specification
![Page 9: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/9.jpg)
9
Software Interfaces: APIs
APIs are external interfaces to software specification of external interface:
• Protocol used at interface• Data exchanged on interface• Behaviour displayed by interface
API can be specified in high-level language like Java, C++ etc.
No specification of software system as a whole Interoperability is difficult without well-defined
external interfaces or connectors Standardised test procedures required to
demonstrate compliance with interface definition Use of TTCN (TTCN-3)
![Page 10: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/10.jpg)
10
Profiling and Interworking
Many standards are ‘open’ Include many options
Interoperability requires profiling e.g. ETSI TISPAN NGN Options are screwed down (dropped) Works with ‘similar’ technologies Implement and test the profile
Significantly different technologies Interworking functions e.g., SIP – H.323
General reluctance to specify ‘profiles’ Indications that situation is changing
![Page 11: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/11.jpg)
11
Different Kinds of ETSI Test Specifications
Conformance Robustness
PerformanceInteroperability
Network Integration
RF/EMC
![Page 12: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/12.jpg)
12
Conformance Testing vs Interoperability Testing
A
TestSystem
latigid
Conformance testing(of a network element)
1 2 3
4 5 6
7 8 9
* 8 #
Conformance testing(of terminal equipment)
latigid
TestSystem
B
Interoperability testing(of terninal equipment)
1 2 3
4 5 6
7 8 9
* 8 #
1 2 3
4 5 6
7 8 9
* 8 #
![Page 13: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/13.jpg)
13
Characteristics of Conformance Testing
Is unit testing Tests a single ‘part’ of a product (e.g., a protocol layer)
for conformity to the requirements specified in the base standard (specification)
Tests over standardised interfaces May not be available to ‘normal’ user
Works at a 'low' level Tests at the protocol (message/behaviour) level (bits)
High control and observability Means we can explicitly test error behaviour and other
'difficult' scenarios Repeatable
Requires a test system (and executable test cases) Can be expensive (e.g., radio-based test system)
Conformance Testing is DEEP and NARROW
![Page 14: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/14.jpg)
14
Limitations of Conformance Testing
Does not prove end-to-end functionality (interoperability) between communicating systems Conformance tested implementations may still not
interoperate• this is often a specification problem rather than a testing
problem!• need minimum requirements or profiles
Does not test a complete product Tests the parts, not the whole
• often a product is more than the sum of its parts
Does not test proprietary features, functions etc. • but this may well be done by a manufacturer with own
conformance tests for proprietary item
![Page 15: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/15.jpg)
15
Characteristics of Interoperability Testing
Do not confuse interoperability testing with interoperability demos
Shows that two products interoperate Within what may be a limited scenario
Tests Functionality (of an ‘entire’ product) Tests the ‘whole’, not the parts
• e.g, protocol stacks ...• or protocol stacks + applications
High-level (as perceived by users) Shows function is accomplished (but not how)
Does not necessarily require a test system Uses existing interfaces (standard/proprietary) Usually lower-cost than conformance But can (should be) automated => test drivers
Interoperability Testing is BROAD and SHALLOW
![Page 16: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/16.jpg)
16
Limitations of Interoperability Testing
Does not prove interoperability with other implementations with which no testing has been done
Does not prove that a product conforms Interoperable products may still be non-conformant
Cannot explicitly test error behaviour or unusual scenarios Or other conditions that may need to be forced (lack of
controllability) Has limited coverage
May not be repeatable
![Page 17: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/17.jpg)
17
Conformance Testing and Interoperability Testing complement each other
Not always a case of either ... or ... Though sometimes it has to be for practical or
economic reasons
But for the best probability of interoperability between products – do both!
Interop Testing
Conformance Testing
Interoperability
![Page 18: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/18.jpg)
18
Summary
Standards can be designed for interoperability
Standards should be engineered
Plan for testing (early)
Do the right kind of testing and test in parallel
![Page 19: Interoperability in the DNA of standards: Ensuring interoperability between building blocks](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681436b550346895dafea65/html5/thumbnails/19.jpg)
19
The End
Thank you for your attention
Interoperability in the DNA of standards: Ensuring interoperability between building blocks