Valuing Diversity - Human Dimensions - IT’S CHRISTMAS TIME!!!!!!!
Human Dimensions of Navigation
Transcript of Human Dimensions of Navigation
![Page 1: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/1.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Dr. Valérie Renaudin
15 November 2018, Toulouse, France
Integrating human dimension in the development of
pedestrian navigation
![Page 2: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/2.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
25 - 50 billion smart
devices worldwide
2
![Page 3: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/3.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Promote multimodal transport with the following priority rules : at first pedestrians, then cyclists, then motorists
Use crowdsourced data collected by a wide range of stakeholders to produce accessibility maps
Expand the prescription sports-health program to integrate the practice of walking and cycling in the treatment of patients
Development driven by market needs
3
![Page 4: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/4.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
GNSS, lowerconsumption, multi-freq
Inertial Sensors, Atomic, 3D glass shell
Sounds, radiowave, …
Camera, mono, stereo, depth, …
Infrastructure basedmotion detection
Light Fidelity
Technology Push & Sensors’ Locationon HumanBody
4
![Page 5: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/5.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Pedestrian Dead Reckoning
t: time index, 𝐏𝑡:Position estimate, 𝐋𝑡: Step length, 𝜃: Walking direction
5
Gyro
Gait Analysis
Mag
Acc
Rotation
IMU misalignment
Step Length Model
Attitude
Walking Direction
Position
sin
cos
𝐏𝑡 = 𝐏𝑡−1 + 𝐋𝑡−1. Ԧ𝜃𝐏𝑡
𝐏𝑡−1
𝐏𝑡+1
𝐋𝑡
![Page 6: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/6.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Pedestrian Dead Reckoning
6
Gyro
Gait Analysis
Mag
Acc
Rotation
IMU misalignment
Step Length Model
Attitude
Walking Direction
Position
6
𝐏𝑡
𝐏𝑡−1
𝐏𝑡+1
𝐋𝑡
![Page 7: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/7.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Monocular Visual
OdometryAssisted by Step Length
7
![Page 8: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/8.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Assisted Monocular Visual Odometry
ContextMonocular vision
Tracking Method Visual odometry
ProblemScale ambiguity
SolutionsState-of-the-art
Use of perfectlyknown 3D objects (GIS)Calibration on known motions
PropositionUsing PDR step length estimates to solve for the ambiguity
8
![Page 9: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/9.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Monocular Visual Odometry
Vision (VO)SURF features extraction
Matching with SSD between 𝒙𝒊 𝑡𝑖−2 ,𝒙𝒊 𝑡𝑖−1 and 𝒙𝒊 𝑡𝑖Outliers filtering with MSAC
Triangulation
Relative pose estimation
Inertial (PDR)PDR processing
Fusion Vision/InertialScale Determination
𝑠𝑡𝑖𝑚 =𝐷𝑃𝐷𝑅 𝑡𝑖𝑚𝐷𝑉𝑂 𝑡𝑖𝑚
Review of visual odometry: types, approaches, challenges, and applications, M. O.
A. Aqel et al, SpringerPlus vol. 5, 2016,
9
![Page 10: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/10.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Step Length Model
Activity classification
Step frequency estimation (STFT, Energy in sub-bands): 𝑓𝑘
Step detection: 𝑡𝑠𝑡𝑒𝑝𝑘
Step length modeling: 𝑠𝑡𝑒𝑝𝑘 = ℎ𝑝𝑒𝑑 ∗ 𝑎 ∗ 𝑓𝑘 + 𝑏 + 𝑐
Mode Recognition and Step Detection Algorithms for Mobile Phone Users, M. Susi et al.
Sensors, 2013.
Step Length Estimation Using Handheld Inertial Sensors, V. Renaudin et al., Sensors, 2012.
Synchronization between step events and visual odometry is important
Height changes extracted from Digital Terrain Model considering a fixed height of the hand
10
![Page 11: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/11.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Experimentation
Scenarios800 m walk in urban environmentStops in front of 2 reference objects3 pedestrians
HardwareHandheld IMMU: ULISS (Ubiquitous Localization with Inertial Sensors and Satellites) at 200 Hz1920*1080 visual measurements at 10 Hz with a Garmin monocular Camera
ReferenceDifferential GNSS (handheld helicoidalantenna + Septentrio)Foot mounted PERSY (PEdestrianReference SYstem) : positionning error0,3% traveled distance
A new PDR navigation device for challeng. urban environments, M. Ortiz et al., Sensors, 2017
Foot-mounted pedestrian navigation reference with tightly coupled GNSS carrier phases,
inertial and magnetic data, J. Le Scornec et al., IEEE Proceedings IPIN 201711
![Page 12: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/12.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Pedestrian path in urban environment
12
![Page 13: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/13.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Accurate Vision/PDR PVT Estimates
13
![Page 14: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/14.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Pedestrian path in urban environment
Specular reflectionintroduces
orientation error
Very distant feature points
Dynamicinitialization
Mean HPE: 8,3 m for the 3 pedestriansHPE <10 m
14
![Page 15: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/15.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Monocular Visual
OdometryAssisted by Step Length
Linking Human Motion and
Map Locations
15
![Page 16: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/16.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Position updates by means of human motion signal patterns recognition
ContextPedestrian navigation with GIS content
Tracking Method Particle Filter combined with topologicalgraph
ProblemsGraph mis-matchingPDR error mitigation
SolutionsState-of-the-art
Heading correction with principal bulding directions/cornersPosition correction with lift locationsAdjust dynamic model to motion context (staircase, lift, …)
PropositionAssociate recognized humanmotions with map regions(Point of Interest)
Entrance Doors
Up/Bottom of stairs
Elevators
Building cornersId E N Type
16
![Page 17: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/17.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Collect dataLearn signal
patternsLink POIs to
the map
Graph Mis-Matching Mitigation
• 4 women & 4 men
• Different motion speeds
• Six Buildings
• Feature Selection
• Different dataset for test and validation
• SVM, Random Forest, Neural Network
• 5 POI types
• Online signal segmentation & classification
• Online POI detection
Supervised Learning Positioning
17
![Page 18: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/18.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
PDR error mitigation
Particle Filter DesignGraph design
Correction of step length estimates
Correction of over/mis-stepdetection
Correction of heading misalignment
MechanizationPDR
UpdatesProximity distance betweenPOI/GPS positions and graph
Likelihood between PDR headingand graph
18
Pedestrian dead reckoning navigation with the help of A* -based routing graphs in
large unconstrained spaces, F. T. Alaoui et al, Wirel. Commun. Mob. Comput., 2017
![Page 19: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/19.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Signal patterns: indoor/outdoor transition (1/2)
IO transition event is marked by a parabolic form of the acc. norm upper envelope: deceleration - quasi-static phase - acc.
The duration of pattern depends on doors’ type: shorter for sliding doors
Sliding door (10-12s)Swinging door (6-9s)
19
![Page 20: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/20.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Signal patterns: indoor/outdoor transition (2/2)
In a texting mode, the vertical component of the angular rate has a greater variance duringthe indoor/outdoortransition
Swinging doors (4-6 s)
20
![Page 21: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/21.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Finding the Right POI Using Signal Pattern
Search for POIs’candidates
Normal walking + Descending stairsMounting stairs + Normal walking
Normal walking + Mounting stairsDescending stairs + Normal walking
Motion classes = Normal walking, Descending/Mounting stairs, Taking an elevatordown/up, Corridor change, Entering/Exiting a building
Consideringmotion sequence
Top of stairs
Bottom of stairs
Consideringmotion class
CornerElevatorBuilding Gate
POI types
Selection of the nearest POI
Nav. Filter
21
![Page 22: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/22.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Supervised Learning Perf. Assessment
Features selected using Mutual Information
Best performance obtained with Random Forest, especiallyfor the indoor/outdoor transition case
Precision: True positive / (True Positive + False Positive)
Sensitivity: True positive / (True Positive + False Negative)
Overall Detection/CLassification Accuracy: 98.41M
Corner Entry / Exit Down StairsElevator
Down / UpNormal Walk. Up Stairs
91.67 100 96.84 100 98.71 98.13
100 96.36 97.87 100 98.71 97.22
22
![Page 23: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/23.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Experimental Assessment
Distance Correction
Detection of Indoor/Outdoor
Transition
Mis-MatchingCorrection
Intra-class variation affects the performance of classification
Intra-class variation is significantly related to cross-individualdispersion (Speed, Gait patterns, etc.)
23
![Page 24: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/24.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Monocular Visual
OdometryAssisted by Step Length
Building on Modeling Personal Dynamic Profile
Linking Human Motion and
Map Locations
24
![Page 25: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/25.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Embedding Personal Dynamics Pattern in the Navigation Filters
ContextImproving personal mobility (transport) with mass market devices
Tracking Method Kalman Filter, Complementary Filter, Machine Learning aiding
ProblemsVarying physiological featuresImpact of environmental changes (groundslopes)
SolutionsState-of-the-art
Calibration by fusing multiple signalsCalibration on known paths
PropositionModeling personal way of holding the sensor in hand to construct navigation filters
25
![Page 26: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/26.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Attitude Estimation
MAGYQ: Attitude estimation EKF filter based on quaternions (inertial and magnetometer data)
WAISS: Walking direction estimation based on statistical modeling of human gait features with handheld MIMU
26
Magnetic, acceleration fields and gyroscope quaternion (MAGYQ)-based attitude estimation
with smartphone sensors for indoor pedestrian navigation, V. Renaudin et al., Sensors, 2014
![Page 27: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/27.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Distribution of horizontal accelerations depends on the walking direction
Distribution modeling with a Gaussian Mixture
Straight-line data set with 0°walking direction
Expectation Maximization algorithm
WAISS Method: Building Individual Models
27
Walking direction estimation based on statistical modeling of human gait features with
handheld MIMU, C. Combettes et al., IEEE/ASME Trans. Mechatronics, vol. 22, 2017
![Page 28: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/28.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Experimental AssessmentUse of PERSY as angular reference
Test different types of carrying modes: texting / swinging
Create individual models appropriate to the person's walking style
Maximization of model likelihood with observations corresponding to one stride
WAISS Method: Walking Direction Estimation
28
![Page 29: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/29.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Choosing the right GMM
Different distributions for different persons
Need to adapt the number of GMM modes for each individual
Use of criterions
29
Building individual inertial signals models to estimate PDR walking direction with
smartphone sensors, J. Perul and V. Renaudin, IPIN 2018 Proceedings
![Page 30: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/30.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Impact of the Carrying Mode on Modeling
Different distributions for the same person
Need to adapt the number of modes for each individual and for each carrying mode
30
Texting Swinging Texting Swinging
![Page 31: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/31.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Assessment: Texting Mode with Misalignment
31
Man – 321 m
Error MAGYQ WAISS
µ 34.1° 13.7°
σ 14.6° 17.3°
Error MAGYQ WAISS
µ 19.7° 11.2°
σ 11.1° 12.4°
Woman – 291 m
WAISS
MAGYQ
PERSY
WAISS
MAGYQ
PERSY
![Page 32: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/32.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
Assessment: Swinging Arm
32
Man – 324 m
Error MAGYQ WAISS
µ 16.1° 8.8°
σ 13.1° 10.3°
Error MAGYQ WAISS
µ 20.9° 8.6°
σ 16.7° 9.9°
Woman - 290 m
WAISS
MAGYQ
PERSY
WAISS
MAGYQ
PERSY
![Page 33: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/33.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018
What scientific approach can solve it? Modeling, artificial intelligence, …?
Human diversity challenges personal navigation technology
Is the hypothesis of an average human being possible?
33
CONCLUSIONS
![Page 34: Human Dimensions of Navigation](https://reader036.fdocuments.us/reader036/viewer/2022071612/615702e3a097e25c76501170/html5/thumbnails/34.jpg)
Valérie Renaudin©- ITSNT18 – Toulouse, 15 November 2018 34
http://www.geoloc.ifsttar.fr/Valérie Renaudin©
Inventing navigation for the new
forms of mobility?