Verification & Testing for Conformity Assessment / EC ...€¦ · for Conformity Assessment / EC...

Co-ordination group of Notified Bodies for railway products and systems.

Verification & Testingfor

Conformity Assessment / EC Verification

NB-Rail(ERTMS Group)


Overview

1. Obligations2. Requirements3. Test Specifications4. Use of Test Results5. Acceptance of Safety Cases5. Acceptance of Safety Cases

6. Inspection - General Criteria7. Inspection - Verification & Validation Methods (V&V)8. Conclusions


ObligationsRef. TSI CCS 2006/860/EC, 2006/679/EC

Fulfilment of ‘essential requirements’ by the applicant!

Ensured by compliance with the applicable ‘mandatory’

specifications, as demonstrated by a positive result of the

assessment of conformity and/or verification of the subsystem.

The mandate of NoBo’s is defined by specific procedures*

Ensured by the assessment of conformity of interoperability

constituents and/or the EC verification of subsystems.

(*ref. TSI CCS Annex E; based on modules as defined in Directive 93/465/EEC)


ObligationsRef. TSI CCS 2006/860/EC, 2006/679/EC

The Notified Body shall verify:

- that all mandatory (M) functions are implemented;

- which optional (O) functions are implemented;

- that additional functions do not conflict with M or O functions.- that additional functions do not conflict with M or O functions.

Note:

EC verification of the assembly includes the verification of the

integration* of the ICs that are ‘notified’ as part of the assembly!

(*ref. TSI CCS, Section 6.2.1 / 6.2.1.1)


ObligationsRef. TSI CCS 2008/386/EC

Questions:

• Where in the TSI are tests specified that must be

carried out in lab and/or real configuration?

• What tests are at minimum required by the TSI to

be performed in lab and/or real configuration?be performed in lab and/or real configuration?

• What is meant by “final testing of the subsystem”?

• What provisions are to be made by the applicant

for tests in lab and/or full operational conditions?

• Does the NoBo have permanent access for testing

and verification activities? – What, if not?


RequirementsRef. TSI CCS 2006/860/EC, 2006/679/EC

The applicable requirements originate from requirements:

- imposed by the applicable procedure/module;

- imposed by the TSI CCS independent of the module;

- imposed by reference to chapter 4 of the TSI CCS that - imposed by reference to chapter 4 of the TSI CCS that

characterize the IC and/or assembly.

Note:

EC verification of the assembly includes the verification of the

integration* of the ICs that are ‘notified’ as part of the assembly!


Characteristics TSI

section

Index Standard Version

Safety 4.2.127 Subset-091 2.5.0

28 reserved s. B1

47 reserved s. B43

ETCS on-board functionality(excl. radio in-fill, Euroloop, train integrity, level 3, data recording for regulatory purposes)

4.2.21 ERA/ERTMS/

003204

5.0

2 99E 5362 2.0.0

4 Subset-026 2.3.0

6 Subset-033 2.0.0

7 Subset-034 2.0.0

8 Subset-035 2.1.1

9 Subset-036 2.4.1

10 Subset-037 2.3.0

11 reserved

05E 537

s. B6

12 Subset-039 2.3.0

Characteristics TSI

section

Index Standard Version

EIRENE functions(data communication only)

4.2.422 intent. deleted

32 EIRENE FRS 7

33 EIRENE SRS 15

48 reserved

ETCS and EIRENE air gap interfaces

4.2.59 Subset-036 2.4.1

10 Subset-037 2.3.0

35 ECC/DC(02)05

39 Subset-092-1 2.3.0

40 Subset-092-2 2.3.0

43 Subset-085 2.2.2

Interfaces- STM(excl. Interface K)

4.2.6.1 4 Subset-026 2.3.08 Subset-035 2.1.115 [H] Subset-108 1.2.0

RequirementsRef. TSI CCS 2006/860/EC, 2006/679/EC

13 Subset-040 2.3.0

14 Subset-041 2.1.0

15 [H] Subset-108 1.2.0

23 Subset-054 2.0.0

24 [C] intent. deleted --

25 Subset-056 2.2.0

26 Subset-057 2.2.0

29 [H] Subset-102 1.0.0

31 Subset-094 2.0.2

36 Subset-074-2 1.0.0

37 Subset-076-5-2

Subset-076-6-3

Subset-076-7

2.3.1

2.3.1

1.0.2

39 Subset-092-1 2.3.0

40 Subset-092-2 2.3.0

43 Subset-085 2.2.2

49 [H] Subset-059 2.1.1

51 reserved s. B34

52 Subset-058 2.1.1

53 reserved --

15 [H] Subset-108 1.2.025 Subset-056 2.2.026 Subset-057 2.2.046 [H] Subset-100 1.0.149 [H] Subset-059 2.1.1

- GSM-R 4.2.6.2 4 Subset-026 2.3.0

7Subset-034 2.0.0

15 [H]Subset-108 1.2.0

22 intent. deleted34 A11T6001 12 12

- Key management system

4.2.8 11reserved

56 [H] reserved05E538

s. B35

- Key Management 4.3.1.7 [C] -- intent, deleted- ETCS-ID Management 4.2.9 23 Subset-054 2.0.0

53 reserved- ETCS Driver Machine Interf.

4.2.13 51 reserved s. Bxx

- Data Interface 4.3.2.8 7 Subset-034 2.0.033 EIRENE SRS 15

- Environmental conditions

4.3.2.5 A4 EN50125-1 1999

- EMC 4.3.2.6 A6 EN50121-3-2 20009 Subset-036 2.4.116 [H] Subset-044 2.3.0

Sample listing of identified mandatory specificationapplicable to a Group of Interoperability Constituents (GoICs): ERTMS/ETCS on-board; Safety Platform on-board; Odometry; ERTMS/GSM-R on-board


Test SpecificationsRef. Annex A 2010/79/EC

TITLE SUBSET

Scope of the test specifications (OBU testing) Subset-076-7 v102 (scope)

Test Cases related to features Subset-076-5-2 v231

Test Sequences Subset-076-6-3 v231

Functional Requirements for an On-board Reference Test Facility Subset-094 v202 (test facilities)

Test specification for Interface “k” Subset-102 v100

FFFIS STM Test Cases document Subset-074-2 v100

ETCS test specifications listed in Annex A

TITLE SUBSET

Test specification for mobile equipment GSM-R Index 48 / reserved

ERTMS GSM-R QoS Test Specification Index B5 / O2475 informative

GSM-R tests specifications listed in Annex A

ERTMS EuroRadio Test Cases Safety Layer Subset-092-2 v230

JRU Test Specification Subset-028 (reserved)

Test Specification for Eurobalise FFFIS Subset-085 v222

Test specification for EUROLOOP Subset-103 v100

RBC-RBC Test specification for Safe Communication Interface Subset-099 (reserved)


Use of Test Results..Ref. RFU-STR-022 (Modules B, SB, F, H2, SF, SG, SH2)

USE OF TEST RESULTS FROM TESTING BODIES USE OF TEST RESULTS FROM TESTING BODIES

OTHER THAN NOTIFIED BODIES OTHER THAN NOTIFIED BODIES

a. Accreditation of testing body- accredited against EN 17025- accredited against EN 17025

or (at least)

- independent, competent and procedures/facilities subject to a quality

system for product inspection and testing.

b. Competence - knowledge, training and practical experience.

c. Acceptance of test results- documented tests and documented testing method


CRITERIA FOR CROSS ACCEPTANCE OF SAFETY CRITERIA FOR CROSS ACCEPTANCE OF SAFETY

CASE ASSESSMENTS CASE ASSESSMENTS

a. Acceptance of the ISA competency competence with regard to the product or subsystem and the standards

Acceptance of Safety CasesRef. RFU 2-000-16 (CCS SUBSYSTEM CERTIFICATION )

competence with regard to the product or subsystem and the standards

and methods

b. Acceptance of the ISA entityNotified Body, other accreditation or notification by member state

c. Acceptance of the safety assessment results (Cross -) Acceptance criteria as stipulated guideline in the RFU


Inspection - General CriteriaRef. DIN EN ISO/IEC 17020

10 10 InspectionInspection methodsmethods andand proceduresprocedures10.1 The 10.1 The InspectionInspection bodybody shall use the methods and procedures for shall use the methods and procedures for inspection which are defined in the requirements, against which inspection which are defined in the requirements, against which conformity is to be determined.conformity is to be determined.

Observation:Observation:Observation:Observation:

(1)(1) The basic notified procedures for conformity assessments are The basic notified procedures for conformity assessments are

documented in the TSI CCS, Annex E (Modules). documented in the TSI CCS, Annex E (Modules).

(2) More detailed procedures may be found in notified test specifications (2) More detailed procedures may be found in notified test specifications

Ref. TSI CCS, Annex A, e.g. Subset Ref. TSI CCS, Annex A, e.g. Subset --085, 085, --094, 094, --076, 076, --102, 102, --074, 074, --092. 092.


10 10 InspectionInspection methodsmethods andand proceduresprocedures10.3 When the inspection body has to use inspection methods or 10.3 When the inspection body has to use inspection methods or procedures which are nonprocedures which are non--standard, such methods and procedures standard, such methods and procedures shall be appropriate and fully documented.shall be appropriate and fully documented.

Observation:Observation:


Observation:Observation:

(3) Application of non(3) Application of non--standard methods or procedures are neither considered standard methods or procedures are neither considered

nor advised by the TSIs.nor advised by the TSIs.


ConclusionConclusion::Measures, methods and procedures to be applied for conformity to be applied for conformity assessments assessments byby NotifiedNotified BodiesBodies are:are:

-- not defined for all mandatory technical specificationsnot defined for all mandatory technical specifications

-- not defined to an adequate extent not defined to an adequate extent


-- not defined to an adequate extent not defined to an adequate extent

in in the mandatory requirements baseline, against which conformity is to the mandatory requirements baseline, against which conformity is to be assessed. be assessed.

Whenever Whenever methods or procedures are not specified, the Notified methods or procedures are not specified, the Notified Body may use inspection methods or procedures which are Body may use inspection methods or procedures which are ‘non‘non--standard’ !standard’ !


Inspection – V&V MethodsRef. DIN EN ISO/IEC 17025

5.4.15.4.1 General General

…… Deviation from test and calibration methods shall occur only if the Deviation from test and calibration methods shall occur only if the

deviation has been documented, technically justified, deviation has been documented, technically justified, authorizedauthorized, ,

and accepted by the client.and accepted by the client.

NOTE: NOTE: International, regional or national standards or other recognized specifications International, regional or national standards or other recognized specifications

that contain sufficient and concise information on how to perform the tests that contain sufficient and concise information on how to perform the tests

and/or calibrations do not need to be supplemented or rewritten as internal and/or calibrations do not need to be supplemented or rewritten as internal

procedure if these standards are written in a way that they can be used as procedure if these standards are written in a way that they can be used as

published by the operating staff in the laboratory. published by the operating staff in the laboratory.

(* V&V := Verification and Validation)


Inspection – V&V Methods Ref. DIN EN ISO/IEC 17025

5.4.25.4.2 SelectionSelection ofof methodsmethods

…… The laboratory shall ensure that it uses the latest valid edition of a The laboratory shall ensure that it uses the latest valid edition of a

standard unless it is not appropriate or possible to do so. …standard unless it is not appropriate or possible to do so. …

… … LaboratoryLaboratory--developed methods or methods adopted by the developed methods or methods adopted by the … … LaboratoryLaboratory--developed methods or methods adopted by the developed methods or methods adopted by the

laboratory may also be used if they are appropriate for the intended laboratory may also be used if they are appropriate for the intended

use and if they are validated. ...use and if they are validated. ...

…… The laboratory shall inform the client when the method proposed by The laboratory shall inform the client when the method proposed by

the client is considered to be inappropriate or out of date. …the client is considered to be inappropriate or out of date. …



5.4.45.4.4 NonNon--standardstandard methodsmethods

…… The method developed shall have been validated appropriately The method developed shall have been validated appropriately

before use. …before use. …

NOTE: NOTE: For new test and calibration methods, procedures should be developed prior to For new test and calibration methods, procedures should be developed prior to NOTE: NOTE: For new test and calibration methods, procedures should be developed prior to For new test and calibration methods, procedures should be developed prior to

the tests and/or calibrations being performed and should at least contain the the tests and/or calibrations being performed and should at least contain the

following information: …following information: …

h) h) description of the proceduredescription of the procedure, including …, including …

-- identification, handling, transport, storing and preparationidentification, handling, transport, storing and preparation

-- checks to be made (for preparation, execution, calibration, adjustment etc.) checks to be made (for preparation, execution, calibration, adjustment etc.)

-- methods of recording observations and resultsmethods of recording observations and results

-- any safety measures to be observedany safety measures to be observed

ii) ) criteria and/or requirements for approval/rejectioncriteria and/or requirements for approval/rejection;;



5.4.5.25.4.5.2 NonNon--standardstandard methodsmethods

…… The laboratory shall validate nonThe laboratory shall validate non--standard methods,… standard methods,…

The validation shall be as extensive as necessary to meet the needs The validation shall be as extensive as necessary to meet the needs

of the given application or field of application. ….of the given application or field of application. ….

NOTE 2:NOTE 2: The techniques used for the determination of the performance of a method The techniques used for the determination of the performance of a method

should be one of, or a combination of, the following:should be one of, or a combination of, the following:

-- CalibrationCalibration using using reference standardsreference standards or or reference materialsreference materials;;

-- Comparison of resultsComparison of results achieved with other methods;achieved with other methods;

-- InterlaboratoryInterlaboratory comparisonscomparisons;;

-- Systematic assessmentSystematic assessment of the factors influencing the results; of the factors influencing the results;

-- Assessment of the uncertaintyAssessment of the uncertainty of the results based on scientific understanding of the results based on scientific understanding

of the theoretical principles of the method and practical experience.of the theoretical principles of the method and practical experience.


Conclusion:Conclusion:Verification methods to be applied for conformity assessments by to be applied for conformity assessments by Notified Bodies are for numerous mandatory specifications:Notified Bodies are for numerous mandatory specifications:

-- not harmonised or validated for their adequacy/applicability not harmonised or validated for their adequacy/applicability (before use)(before use)

-- neither agreed, authorized and/or technically justifiedneither agreed, authorized and/or technically justified


-- neither agreed, authorized and/or technically justifiedneither agreed, authorized and/or technically justified

--do do not enclose information on how to perform verification/tests not enclose information on how to perform verification/tests

Criteria or requirements for the approval/rejection of Criteria or requirements for the approval/rejection of verification, verification, e.g. by test results, are e.g. by test results, are often missing. Besides, findings concerning often missing. Besides, findings concerning the methods are not communicated (extensive as necessary).the methods are not communicated (extensive as necessary).


Conclusions

Most Most of the ‘mandatory technical specifications‘ leave of the ‘mandatory technical specifications‘ leave

room for interpretations and room for interpretations and may thus lead to divergent may thus lead to divergent

verification verification results. Therefore they results. Therefore they are are considered considered

immature for inspection and certification !immature for inspection and certification !

The techniques used for determining the performance of The techniques used for determining the performance of

measures and methods are not applied (extensive as measures and methods are not applied (extensive as

necessary) on the standards against which conformity necessary) on the standards against which conformity

is to be determined. is to be determined.


Questions?

NB-Rail - Subgroup ERTMS

http://circa.europa.eu/irc/nbg/nbrail/info/data/en/information/nbrail/00nb rail homepage.htm

Verification & Testing for Conformity Assessment / EC ...€¦ · for Conformity Assessment / EC...

Documents

Transcript of Verification & Testing for Conformity Assessment / EC ...€¦ · for Conformity Assessment / EC...