Introduction and Overview of 3.5 Years of PEGASUS · 2019-05-20 · Introduction and Overview of...

Introduction and Overview of 3.5 Years of PEGASUS

Daniel Lipinski| 14th May 2019

2

The PEGASUS-history

Mid 2014: First Discussion about a Common Proving Center for AD

Autumn 2014: Focusing on a Method for Safeguarding

January 2015: Project Outline

September 2015: Project Description

3

The PEGASUS-history January 2016: Project start with 17 partners OEM: Audi, BMW, Daimler, Opel, Volkswagen Tier 1: ADC, Bosch, Continental Test Lab: TÜV SÜD SMB: fka, iMAR, IPG, QTronic, TraceTronic, VIRES Scientific institutes: DLR, TU Darmstadt

Subcontracts: IFR, ika, OFFIS

Key-facts: 42 Months Term 1.791 man-month or 149 man-years 34,5 Mio. EUR Budget 4 Sub-Projects 13 Workpackages 38 Sub-Workpackages

Mid 2016: Convention of an Advisory Board • Federal Ministry for Economic Affairs and Energy • Federal Ministry of Transport and Digital Infrastructure • Federal Ministry of Justice and Consumer Protection • German Association of the Automotive Industry (VDA) • German Road Safety Council (DVR) • ADAC

Associated Partner: Federal Highway Research Institute (BASt)

dSPACE

4

The PEGASUS-history November 2017: PEGASUS-Half-Time-Event in Aachen

For the first time: Presentation of the PEGASUS-Approach

Presentation of Intermediate Results

PEGASUS becomes International

The PEGASUS-history Germany: BMVI, BMWi, BMJV, KBA, BASt, DVR, ADAC, Ethics commission (Prof. Hilgendorf)

Europe: OICA UN-ECE Horizontale Initiative; EU-Comission, EU Strategic Transport and Innovation Agenda

Japan: METI, JAMA, Toyota, Honda, Nissan

China: CATARC

Symposium Aachen

Symposium Wien

Symposium San Francisco

Symposium Tokyo

Singapur: CETRAN

US: NHTSA, US DOT, AutoAlliance, RAND

World wide: ISO: ISO/TC 22/SC 33/WG 9 und WG16 sowie ISO TR21959 Part 2 und SOTIF DIN SAE: Spec Project Terms and definitions

Addtl. Cooperation Requests & bilateral Exchange: FP Nouvelle France Industrielle, AutoAlliance, Jaguar LandRover, Hyundai, Volvo, RDW, etc.

Korea: Hyundai.

Symposium Wolfsburg

© PEGASUS 5

The statement that we all need source: daijiworld.com 6

Zentrale Fragestellungen in PEGASUS

How safe is safe enough and how can we verify that it achieves the

desired performance consistently?

…by introducing a

Scenario Based Method to Assess Highly Automated Driving Functions

With PEGASUS,

we contribute to answer the question…

7 © PEGASUS

Database

Data processing Database

Assessment of Highly Automated Driving Function

Requirementsdefintion

Test HAD-F:

SimulationProving Ground

Real World Drive

Evidence

Co

ntrib

utio

n to

Safe

tyState

me

nt

Data inPEGASUS-

Format

Application of Metrics + Mapping to Logical

Scenarios

Logical Scenarios +

Parameter Space

Preparation for Test Concept

IntegrationPass Criteria

Application of Test Concept incl. Variation

Method

PEGASUS Method for Assessment of Highly Automated Driving Function (HAD-F)

Risk Assessment

Release Test Evaluation Test Execution Test Case Derivation

Source of Information Evaluation & Conversion Scenarios

Argumentation

Contribution to Safety

Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing

Preprocessing / Reconstruction

Systematic Identification of Scenarios

21

10 11

875

4

Daten:

Test DriveSimulationSimulatorFOT/NDSAccident

2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018

Process Guidelines + Metrics for HAD

Assessment

6

cen

tral

dec

entr

al

20 19 15 13

Space of Logical Test Cases

12Test C

ases

14

Test D

ata

16

Test R

esu

lts

18

Evaluation and

Classification

17

9Knowledge:

Laws,Standards,

Guidelines, ...

1Requirements Analysis

3

8 © PEGASUS

Database




Test HAD-F:


Real World Drive

Evidence

Co

ntrib

utio

n to

Safe

tyState

me

nt

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space

Preparation for Test Concept



Method


Risk Assessment



Argumentation


Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing



21

10 11

875

4

Daten:


2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018


Assessment

6

cen

tral

dec

entr

al

20 19 15 13


12Test C

ases

14

Test D

ata

16

Test R

esu

lts

18

Evaluation and

Classification

17

9Knowledge:

Laws,Standards,

Guidelines, ...


3

9

Start: Use-Case

Goal: Safety

argument

© PEGASUS

Database




Evidence


Argumentation

V1.4 Status21.09.2018

10

Start: Use-Case

Goal: Safety

argument

© PEGASUS

Database




Evidence


Argumentation

Use Case,Knowledge,

Data

V1.4 Status21.09.2018

11 © PEGASUS

Use Case

© PEGASUS 12

Safeguarding of Level 3 (Highly Automated Driving) function

Based on an application-oriented example, highway chauffeur

Basic function:

Highways or highway-like roads incl. road markings

Speed 0 - 130 km/h

Automated following in stop & go traffic jams

Automated lane changing

Automated emergency braking and collision avoidance

Construction sites

Automated exiting off the highway

Extreme weather conditions

source: VW

Database




Evidence


Argumentation

Data / Content

Procedure

Workflow

Process Instruction

5

4

Use Case,Knowledge,

Data

V1.5 Status14.02.2019

cen

tral

dec

entr

al

Knowledge:Laws,

Standards,

Guidelines, ...


3 Process Guidelines + Metrics for HAD

Assessment

6

13 © PEGASUS

Application of different risk acceptance principles

© PEGASUS 14

10 -10

10 -9

10 -8

10 -7

10 -6

10 -5

10 -4

10 -3

10 -2

acceptable

Major technology

Involuntary

Job-related

Voluntary

inacceptable

Individual mortality risk per person per year

MEM/20 MEM

ALARP lower limit ALARP upper limit

GAMAB

quoted by: U. Steininger, L. Wech; „Wie sicher ist sicher genug? - Sicherheit und Risiko zwischen Wunsch und Wirklichkeit“, VDI Berichte 2204, p 221-228, VDI Verlag Düsseldorf 2013, ISSN 0083-5560) Original by A.F.Fritsche: Wie sicher leben wir? Risikobeurteilung und –bewältigung in unserer Gesellschaft; TÜV Rheinland, Köln 1986, ISBN 3-88585-278-0

ALARP = As Low As Reasonably Practicable MEM = Minimum Endogenous Mortality GAMAB = Globalement Au Moins Aussi Bon

Method Gap How do we use test results with regard to an approval recommendation?

The challenge is to map single test results to global acceptance criteria.

© PEGASUS 15

Database




Evidence


Source of Information

Argumentation

Data / Content

Procedure

Workflow

Process Instruction

5

4

Data:


2

Use Case,Knowledge,

Data

V1.5 Status14.02.2019

cen

tral

dec

entr

al

Knowledge:Laws,

Standards,

Guidelines, ...


3 Process Guidelines + Metrics for HAD

Assessment

6

16 © PEGASUS

Input data

© PEGASUS 17

NDS / FOT

Simulator

crash

Real world

Simulation

source: UDRIVE, IPG, Audi, DLR

http://www.google.de/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwjk6cKot6PaAhVC_qQKHZ7KAaIQjRx6BAgAEAU&url=http://www.zeit.de/mobilitaet/2015-04/autonomes-fahren-audi&psig=AOvVaw3EWli0Blw1yZP6DiUp-U3i&ust=1523027909795447

http://www.google.de/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwjt4YmBtqPaAhUxsaQKHY8PDOQQjRx6BAgAEAU&url=http://www.traffictechnologytoday.com/news.php?NewsID%3D72233&psig=AOvVaw0YpXlPuN3Uk7Ql5PgodotM&ust=1523027564065435

https://www.google.de/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwiQzpLjtqPaAhVB_aQKHXttB2YQjRx6BAgAEAU&url=https://mhh-unfallforschung.de/de_DE/publikationen/&psig=AOvVaw3hag7IG3pfLGU3MxmaC62O&ust=1523027784576979

Database




Evidence

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space



Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing



875

4

Data:


2

Use Case,Knowledge,

Data

V1.5 Status14.02.2019

cen

tral

dec

entr

al


11

9Knowledge:

Laws,Standards,

Guidelines, ...


3


10


Assessment

6

18 © PEGASUS

Layer 2: Traffic Infrastructure Boundaries (structural) Traffic signs, elevated barriers

Layer 1: Road-Level Geometry, topology Quality, boundaries

(surface)

Layer 3: Temporary manipulation of Layer 1 and 2 Geometry, topology (overlaid) Time frame > 1 day

Layer 4: Objects Static, dynamic, movable Interactions, maneuvers

Layer 5: Environment • Weather, lighting and other

surrounding conditions

Layer 6: Digital Information (e.g. )V2X information,

digital map

Scenarios and possibilities for description – Layer model

© PEGASUS 19

© PEGASUS 20

Functional scenarios

Base road network:

Three-lane motorway in a curve, 100 km/h speed limit indicated by traffic signs

Stationary objects:

-

Moveable objects:

Ego vehicle, Traffic jam; Interaction: Ego in maneuver „approaching“ on the middle lane, traffic jam moves slowly

Environment:

Summer, rain

Logical scenarios

Base road network:

Lane width [2..4] m Curve radius [0,6..0,9] km Position traffic sign [0..200] m

Stationary objects:

-

Moveable objects:

End of traffic jam [10..200] m Traffic jam speed [0..30] km/h Ego distance [50..300] m Ego speed [80..130] km/h

Environment :

Temperature [10..40] °C Droplet size [20..100] µm rainfall [0,1..10] mm/h

Level of abstraction

Concrete scenarios

Base road network:

Lane width 3 Curve radius 0,7 km Position traffic sign 150 m

Stationary objects:

-

Moveable objects :

End of traffic jam 40 m Traffic jam speed 30 km/h Ego distance 200 m Ego speed 100 km/h

Environment :

Temperature 20 °C Droplet size 30 µm rainfall 2 mm/h

Number of scenarios

Scenarios and possibilities for description – Level of abstraction

Database




Test HAD-F:


Real World Drive

Evidence

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space


Method


Test Evaluation Test Execution Test Case Derivation


Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing



875

4

Data:


2

Use Case,Knowledge,

Data

V1.5 Status14.02.2019

cen

tral

dec

entr

al

14 12


11Test C

ases

13

Test D

ata

15

Test R

esu

lts

17

Evaluation and

Classification

16

9Knowledge:

Laws,Standards,

Guidelines, ...


3


10


Assessment

6

21 © PEGASUS

Test Objectives

© PEGASUS 23 Source: QTronic

Search for safety violations / worst case(s)

Characterize the regions with safety violations, e.g. find their borders

Deliver coverage reports for one or for a suite of experiments

Assessment result for concrete sample Scenario depending on multiple parameters. Color range form green (not critical) to red

(critical)

Test concept

© PEGASUS 24

Space of logical test

cases

Allocation of test cases to

test platform

Automatized variation

of parameter

Manual selection of scenarios/ parameter (e.g. tests as per ECE R79, rating

tests)

Specific scenarios

Test execution looking for

critical cases (e.g.

accidents)

Evaluated concrete scenarios (Pass/Fail)

Pass-/Fail-Criteria

Test object: Functional implementation of highway chauffeur

Test level: Functional Test Test platform : Simulation

Test object: Overall system highway chaffeur including vehicle behavior Test level: Vehicle test Test platform : Test ground/ field

Test

execution Driving on

TG +

validation simulation

Test execution of real world drive with

guidelines: • Route • Weather • Time • Specific

features

“all logical scenarios”

„selected scenarios“

no direct link to space of logical test cases

Comparison

Test platform

Simulation

Feedback

Replay 2Sim

Logical scenario

+ parameter space

+ exposure of parameter

Test platform

Test ground

Test platform

Field

Concrete scenarios

„Surprises“

Measured data for

database

Measured data for Replay 2Sim

Critical cases

Database




Test HAD-F:


Real World Drive

Evidence

Co

ntrib

utio

n to

Safe

tyState

me

nt

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space


Method


Risk Assessment



Argumentation


Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing



20

875

4

Data:


2

Use Case,Knowledge,

Data

V1.5 Status14.02.2019

cen

tral

dec

entr

al

19 18 14 12


11Test C

ases

13

Test D

ata

15

Test R

esu

lts

17

Evaluation and

Classification

16

9Knowledge:

Laws,Standards,

Guidelines, ...


3


10


Assessment

6

26 © PEGASUS

The overall rating of a test-case is currently derived by aggregating the time-discrete results of the multiple stages.

The contribution of the different stages to the overall test-case result differs depending on their character.

Further knowledge about exposure and significance will improve strength of argument

Safety Statement - Assessment of a single test-case

27

Picture, example of overall test-case rating based on the 4 proposed stages. 0 and 1

are indicating if a stage is failed or passed, respectively. Picture, application the different safety criteria over time. The result is PASS with stage 1 fail, stage

2 fail, stage 3 pass und stage 4 pass.

© PEGASUS

Layers of the Safety Argumentation

Automated Driving Systems (ADS) are widely accepted in the public.

There is an understanding of what factors foster acceptance of ADS.

Top level goals are set to be met in order to achieve acceptance of ADS.

Logical structure of the Safety Argumentation links top level goals with methods & tools and their results.

The Safety Argumentation is implemented using methods & tools.

Results become evident when they can be traced back to the achievement of a goal.

Motivation

Context

Argumentation / Approval

Recommondation

1

2

3

4

0

Ico

ns

ma

de

by

Fre

ep

ik f

rom

ww

w.f

lati

con

.co

m

© PEGASUS 28

Many thanks for your attention!

Introduction and Overview of 3.5 Years of PEGASUS · 2019-05-20 · Introduction and Overview of...

Documents

Transcript of Introduction and Overview of 3.5 Years of PEGASUS · 2019-05-20 · Introduction and Overview of...