INRETS, Villeneuve d’Ascq, December 15 th -16 th 2005 ETISEO Annotation rules Data structure...

INRETS, Villeneuve d’Ascq, December 15th-16th 2005

ETISEO Annotation ETISEO Annotation rulesrules

•Data structure•Annotation tool and format•Ground truth creation rules•Reference data generation


Data structure - Data structure - contextcontext

• Id• Type (ex: zone)• Name (ex: exit_zone)• Polygon

Context


Data structure - objectData structure - object

• Id• Start time• Type (ex: person,

vehicle…)• Subtype (ex: car,

van…)• Info2D• Info3D

• Links• Speed• GT Nature• GT Status• GT Occluded (partially or

completely)• GT Connected with

Physical object

GT : specific to annotation purpose


Data structure - eventData structure - event

• Id• Name• Start and end time• Status• Involved objects (physical object of interest

and contextual object)

Event



•Data structure•Annotation tool and format

•Ground truth creation rules•Reference data generation


Annotation tool and Annotation tool and formatformat

• Database - Excel file• Context annotation - Viper-GT tool and format

Annotation delivered with video:

• Ground truth - Viper-GT tool and format• Reference data (computed from GT) – Specific

tool and Viper-GT format

Annotation delivered at the end of Etiseo:



Viper-GT format example<?xml version="1.0" encoding="UTF-8"?><viper xmlns="http://lamp.cfar.umd.edu/viper#" xmlns:data="http://lamp.cfar.umd.edu/viperdata#">

<config> <…></config><data>

<sourcefile filename="file:ETI-VS1-CR-4-C1.avi"><object framespan="1:100" id="4" name="physical_object">

<attribute name="nature"><data:svalue value="physical_object_of_interest"/>

</attribute><attribute name="type">

<data:svalue value="person"/></attribute><attribute name="info2D">

<data:bbox framespan="1:1" height="182" width="86" x="138" y="59"/><data:bbox framespan="2:2" height="183" width="86" x="139" y="59"/>

<data:bbox framespan="3:3" height="185" width="86" x="141" y="60"/> <data:bbox framespan="4:17" height="187" width="87" x="143" y="60"/> <…>

<data:bbox framespan="40:100" height="212" width="10" x="3" y="10"/></attribute><…>

</sourcefile></data>

</viper>



•Data structure•Annotation tool and format•Ground truth creation rules

•Reference data generation


Ground truth creation Ground truth creation rulesrules

• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation

• Rules for events


Ground truth creation – Ground truth creation – Rules for objectsRules for objects

Contextual object

• Annotated during its period of movement



Static physical object of interest

• Any physical object of interest is annotated on the whole video clip (ex: a parked vehicle)

First frame and last frame of a video clip



Occlusions – general rules

• Apparition of an object: the BBox increase with object apparition including only its visible part,

• All BBox are limited by the image boundary,• Partial and complete occlusion will be

differentiate in the evaluation (attribute “occluded”).



Static occlusions (by a contextual object)

• Partial occlusion: full BBox is drawn including hidden part,



Static occlusions (by a contextual object)

• Complete occlusion - two cases: all “escapes” of the contextual object are

visible: full BBox is drawn all the time,



all “escapes” of the contextual object are not visible: the object is no more annotated when completely occluded



• Partial or complete occlusion: in both cases, the full BBox is drawn including hidden part.

Dynamic occlusion (by another object)



Split object due to occlusion

• A unique BBox including all sub part is drawn.



Artefacts• Only the part constituting the objects forms the

BBox, artefacts (shadow, reflection…) are ignored for instant.

• Annotated in video data set 2



Sudden illumination changes (tagged frames)• Static object

• Object in movement: position interpolation



Held object

Example: bag-person, person-bike, door-vehicle

• The holder and the held object are both annotated (if visible),

• The carried object is connected to the holder.



Specific case

• Person getting in/out a vehicle: annotated only when visible



Group - Crowd• Group: multiple bounding boxes (persons +

group)

• Crowd : unique Bbox when individual annotation is impossible.



Crowded video clip• Only object involved in event of interest are

annotated. No false detections evaluation



Multi-view annotation


Ground truth creation – Ground truth creation – Rules for eventsRules for events

• Id• Name• Start and end time• Status• Involved objects:

Id of the physical object of interest in GT files

Id of the contextual object in context information files

Event – data structure



• Choice of event for each video clip in the event ontology => defined in the database xls file (provided with video)

Sequence Name ETI-VS1-RD-15

… … …

Content

Physical objects type Person, vehicle

Events of interest gets_in, gets_out, parked



• Automatic annotation of events with quantitative constraints, example:

Related to kinematics properties: stopped, running…

Related to spatial properties: close_to, inside_zone…

• Manual annotation of other events: holding, pushing, waiting, gets_in/out,…



Example



•Data structure•Annotation tool and format•Ground truth creation rules•Reference data generation


Reference data Reference data generationgeneration

• Filtering on object size• Object speed computation• Group generation• Multi-view and 3D reconstruction• Events with quantitative constraints

generation



Documentation:• ETI-Data structure and output format• ETI-Video Annotation Rules



Questions …

INRETS, Villeneuve d’Ascq, December 15 th -16 th 2005 ETISEO Annotation rules Data structure...

Documents

Transcript of INRETS, Villeneuve d’Ascq, December 15 th -16 th 2005 ETISEO Annotation rules Data structure...