Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion...
Transcript of Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion...
![Page 1: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/1.jpg)
Probabilistic modelling for sensor fusion(using inertial sensors and magnetometers)
Manon Kok
Delft University of Technology, The Netherlands
September 5th, 2018
![Page 2: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/2.jpg)
Brief background
• MSc in Philosophy of Science, Technology and Society at the Universityof Twente, the Netherlands.
• MSc in Applied Physics at the University of Twente, the Netherlands.
• Research Engineer at Xsens Technologies (February 2009 – July 2011).
• PhD in Automatic Control from Linkoping University (August 2011 –February 2017).
• Postdoc in the Machine Learning Group at the University of Cambridge(March 2017 – February 2018).
• Assistant Professor in the Delft Center for Systems and Control at theDelft University of Technology (April 2018 – Now).
2 / 29
![Page 3: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/3.jpg)
Probabilistic modelling for sensor fusion(using inertial sensors and magnetometers)
3 / 29
![Page 4: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/4.jpg)
About my researchSensor fusion: Using disparate sensors in order to infer more informationabout the quantity of interest than would be possible using each sensorindividually.
Quantity of interest: e.g. the position and orientation of the sensor.
• Inertial sensors• Accelerometers• Gyroscopes
• Magnetometers
4 / 29
![Page 5: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/5.jpg)
Accelerometer measurements
5 10 15 20 25 30−20
0
20
Time [s]
y a,t
[m/s
2 ]
Stationary data ⇒ Earth gravityRotating sensor ⇒ Earth gravity
Quickly moving sensor ⇒ Earth gravity + sensor’s accelerationThe combination of earth gravity and sensor acceleration is called the
external specific force.
5 / 29
![Page 6: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/6.jpg)
Accelerometer measurements
5 10 15 20 25 30−20
0
20
Time [s]
y a,t
[m/s
2 ]
Stationary data ⇒ Earth gravity
Rotating sensor ⇒ Earth gravityQuickly moving sensor ⇒ Earth gravity + sensor’s acceleration
The combination of earth gravity and sensor acceleration is called theexternal specific force.
5 / 29
![Page 7: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/7.jpg)
Accelerometer measurements
5 10 15 20 25 30−20
0
20
Time [s]
y a,t
[m/s
2 ]
Stationary data ⇒ Earth gravityRotating sensor ⇒ Earth gravity
Quickly moving sensor ⇒ Earth gravity + sensor’s accelerationThe combination of earth gravity and sensor acceleration is called the
external specific force.
5 / 29
![Page 8: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/8.jpg)
Accelerometer measurements
5 10 15 20 25 30−20
0
20
Time [s]
y a,t
[m/s
2 ]
Stationary data ⇒ Earth gravityRotating sensor ⇒ Earth gravity
Quickly moving sensor ⇒ Earth gravity + sensor’s accelerationThe combination of earth gravity and sensor acceleration is called the
external specific force.
5 / 29
![Page 9: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/9.jpg)
Gyroscope measurements
2 4 6 8 10 12 14 16 18 20 22
−5
0
Time [s]
y ω,t
[rad
/s]
Stationary data ⇒ Zero angular velocityRotating the sensor around the y -axisRotating the sensor around the x-axisRotating the sensor around the z-axis
6 / 29
![Page 10: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/10.jpg)
Gyroscope measurements
2 4 6 8 10 12 14 16 18 20 22
−5
0
Time [s]
y ω,t
[rad
/s]
Stationary data ⇒ Zero angular velocity
Rotating the sensor around the y -axisRotating the sensor around the x-axisRotating the sensor around the z-axis
6 / 29
![Page 11: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/11.jpg)
Gyroscope measurements
2 4 6 8 10 12 14 16 18 20 22
−5
0
Time [s]
y ω,t
[rad
/s]
Stationary data ⇒ Zero angular velocityRotating the sensor around the y -axis
Rotating the sensor around the x-axisRotating the sensor around the z-axis
6 / 29
![Page 12: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/12.jpg)
Gyroscope measurements
2 4 6 8 10 12 14 16 18 20 22
−5
0
Time [s]
y ω,t
[rad
/s]
Stationary data ⇒ Zero angular velocityRotating the sensor around the y -axisRotating the sensor around the x-axis
Rotating the sensor around the z-axis
6 / 29
![Page 13: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/13.jpg)
Gyroscope measurements
2 4 6 8 10 12 14 16 18 20 22
−5
0
Time [s]
y ω,t
[rad
/s]
Stationary data ⇒ Zero angular velocityRotating the sensor around the y -axisRotating the sensor around the x-axisRotating the sensor around the z-axis
6 / 29
![Page 14: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/14.jpg)
Inertial sensors for pose estimation
∫
rotateremove
gravity
x
angular
velocity orientation
external spe-
cific force acceleration position
Accelerometers and gyroscopes (inertial sensors) can be used forposition and orientation (pose) estimation ⋯
7 / 29
![Page 15: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/15.jpg)
Integration drift
however, the position and orientation estimates obtained using inertialsensors suffer from integration drift.
0 2 4 6 8 100
1
2
3
Time [s]
Orie
ntat
ion
[◦ ]
0 2 4 6 8 10
0
5
Time [s]
Posit
ion
[m]
⇒ Combine inertial sensors with other sensors and / or additionalmodels to obtain accurate estimates.
8 / 29
![Page 16: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/16.jpg)
Probabilistic modelling for sensor fusion(using inertial sensors and magnetometers)
9 / 29
![Page 17: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/17.jpg)
Probabilistic modelling
Probabilistic modelling to
• take into account that all models are approximations,
• take into account that all sensor measurements are not perfect,
• represent the best estimate and its uncertainty.
10 / 29
![Page 18: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/18.jpg)
Probabilistic modelling for sensor fusion(using inertial sensors and magnetometers)
11 / 29
![Page 19: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/19.jpg)
Magnetometers
Magnetometers measure the local(earth) magnetic field
12 / 29
![Page 20: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/20.jpg)
Magnetometer measurements
2 4 6 8 10 12 14 16
−200
0
200
400
Time [s]
y m,t
[µT]
Stationary magnetometer data ⇒ Local (earth) magnetic fieldMagnetic disturbance close to sensor ⇒ Local (earth) magnetic field +
magnetic disturbance
Inertial sensors and magnetometers can be combined to estimateorientation.
But how to model the magnetometer measurements?
13 / 29
![Page 21: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/21.jpg)
Magnetometer measurements
2 4 6 8 10 12 14 16
−200
0
200
400
Time [s]
y m,t
[µT]
Stationary magnetometer data ⇒ Local (earth) magnetic field
Magnetic disturbance close to sensor ⇒ Local (earth) magnetic field +magnetic disturbance
Inertial sensors and magnetometers can be combined to estimateorientation.
But how to model the magnetometer measurements?
13 / 29
![Page 22: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/22.jpg)
Magnetometer measurements
2 4 6 8 10 12 14 16
−200
0
200
400
Time [s]
y m,t
[µT]
Stationary magnetometer data ⇒ Local (earth) magnetic fieldMagnetic disturbance close to sensor ⇒ Local (earth) magnetic field +
magnetic disturbance
Inertial sensors and magnetometers can be combined to estimateorientation.
But how to model the magnetometer measurements?
13 / 29
![Page 23: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/23.jpg)
Magnetometer measurements
2 4 6 8 10 12 14 16
−200
0
200
400
Time [s]
y m,t
[µT]
Stationary magnetometer data ⇒ Local (earth) magnetic fieldMagnetic disturbance close to sensor ⇒ Local (earth) magnetic field +
magnetic disturbance
Inertial sensors and magnetometers can be combined to estimateorientation.
But how to model the magnetometer measurements?
13 / 29
![Page 24: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/24.jpg)
My research
Goal:
Push the boundaries of accuracy that can be obtained from sensormeasurements by using
• advanced models,
• advanced algorithms.
Motivation:
• Sensors are becoming smaller, cheaper and more widely available.
• More and more computational power is available.
• More and more application areas are opening up.
14 / 29
![Page 25: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/25.jpg)
Applications
Analysis of motion of:
• Cars
• Airplanes
• Quadcopters
• Trains⋮
Figures courtesy of Xsens Technologies
15 / 29
![Page 26: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/26.jpg)
Applications
Figures courtesy of Xsens Technologies
Analysis of human motion for:
• Sports
• Rehabilitation
• Movies / games
• Robotics⋮
15 / 29
![Page 27: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/27.jpg)
Two examples
• Human motion capture• Take as much information about the situation at hand into account.
• Indoor localisation using measurements of the magnetic field.• Extract as much information from sensor measurements as possible.
16 / 29
![Page 28: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/28.jpg)
Example: Human motion capture
Estimate the relative position andorientation of body segments.
17 sensors placed on the body
Figures courtesy of Xsens Technologies
17 / 29
![Page 29: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/29.jpg)
Example: Human motion capture
Estimate the relative position andorientation of body segments.
17 sensors placed on the bodyFigures courtesy of Xsens Technologies
17 / 29
![Page 30: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/30.jpg)
Example: Human motion capture
Estimate the relative position andorientation of body segments.
17 sensors placed on the bodyFigures courtesy of Xsens Technologies
17 / 29
![Page 31: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/31.jpg)
Model
Inertial sensors provide information aboutthe change in position and orientation.
The position and orientation of the sensorson the body is approximately constant.
The body segments are connected at thejoints. ⇒ constraint
Additional information...
18 / 29
![Page 32: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/32.jpg)
Model
Inertial sensors provide information aboutthe change in position and orientation.
The position and orientation of the sensorson the body is approximately constant.
The body segments are connected at thejoints. ⇒ constraint
Additional information...
18 / 29
![Page 33: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/33.jpg)
Model
Inertial sensors provide information aboutthe change in position and orientation.
The position and orientation of the sensorson the body is approximately constant.
The body segments are connected at thejoints.
⇒ constraint
Additional information...
18 / 29
![Page 34: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/34.jpg)
Model
Inertial sensors provide information aboutthe change in position and orientation.
The position and orientation of the sensorson the body is approximately constant.
The body segments are connected at thejoints. ⇒ constraint
Additional information...
18 / 29
![Page 35: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/35.jpg)
Model
Inertial sensors provide information aboutthe change in position and orientation.
The position and orientation of the sensorson the body is approximately constant.
The body segments are connected at thejoints. ⇒ constraint
Additional information...
18 / 29
![Page 36: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/36.jpg)
Example: human body motion capture
arg minx1∶NT
− NT∑t=2
NS∑i=1
log p(xSit ∣ xSit−1, ySit )´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
dynamics of the state xSit
− NS∑i=1
log p(xSi1 ∣ ySi1 )´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶prior
− NT∑t=1
NS∑i=1
log p(xBj
t ∣ xSit )´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
placement of sensor Si on body segment Bj
subject to c(x) = 0
19 / 29
![Page 37: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/37.jpg)
Human motion capture
Use only sensor data
Courtesy of Xsens Technologies
20 / 29
![Page 38: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/38.jpg)
Human motion capture
Use sensor data + a biomechanical model
Courtesy of Xsens Technologies
21 / 29
![Page 39: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/39.jpg)
Human motion capture
Use sensor data + a biomechanical model + a world model
Courtesy of Xsens Technologies
22 / 29
![Page 40: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/40.jpg)
Two examples
• Human motion capture• Take as much information about the situation at hand into account.
• Indoor localisation using measurements of the magnetic field.• Extract as much information from sensor measurements as possible.
23 / 29
![Page 41: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/41.jpg)
Example: Indoor localisation
Indoor localisation
• to find your way in largeshopping malls, train stations,etc,
• for first responders toemergencies,
• ⋮
0 1 2 3 4 5 10 m
30
40
50
60
Mag
nitu
de(µ
T)
Which sensors to use?
• GPS does not work (well) indoors.
• Use sensors that are present in any modern smartphone.
⇒ Use measurements of the magnetic field as a source of positioninformation!
24 / 29
![Page 42: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/42.jpg)
Example: Indoor localisation
Indoor localisation
• to find your way in largeshopping malls, train stations,etc,
• for first responders toemergencies,
• ⋮0 1 2 3 4 5 10 m
30
40
50
60
Mag
nitu
de(µ
T)
Which sensors to use?
• GPS does not work (well) indoors.
• Use sensors that are present in any modern smartphone.
⇒ Use measurements of the magnetic field as a source of positioninformation!
24 / 29
![Page 43: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/43.jpg)
Modelling the magnetic field
• Use machine learning to learn the magnetic field from data.
• Gaussian process models: Learn a map of the magnetic field. Alsolearn where you are certain or uncertain about this map.
25 / 29
![Page 44: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/44.jpg)
Building magnetic field maps
Build a map of the indoor magnetic field to use it for localisation.
https://www.youtube.com/watch?v=enlMiUqPVJo
26 / 29
![Page 45: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/45.jpg)
Magnetic field SLAM
Simultaneous localization and mapping
https:
//www.youtube.com/watch?v=enlMiUqPVJo
27 / 29
![Page 46: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/46.jpg)
Two examples
• Human motion capture• Take as much information about the situation at hand into account.
• Indoor localisation using measurements of the magnetic field.• Extract as much information from sensor measurements as possible.
28 / 29
![Page 47: Probabilistic modelling for sensor fusion (using …...Probabilistic modelling for sensor fusion (using inertial sensors and magnetometers) Manon Kok Delft University of Technology,](https://reader033.fdocuments.us/reader033/viewer/2022042913/5f496a35e08f6905ee57ffdc/html5/thumbnails/47.jpg)
Thank you for your attention!
Questions?
https://www.tudelft.nl/staff/m.kok-1/
29 / 29