ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain
description
Transcript of ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain
![Page 1: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/1.jpg)
1
ForeSight: Mapping Vehicles in Visual Domain and Electronic
Domain
Dong Li, Zhixue Lu, Tarun Bansal, Erik Schilling and Prasun SinhaDepartment of Computer Science and Engineering
The Ohio State University
![Page 2: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/2.jpg)
2
Need for Targeted Communication
OK, but who are you?
What’s in front?
Are you talking to
me?
Hey you at the back -- Your lights are off!
I am overtaking you, don’t change
lane!
![Page 3: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/3.jpg)
3
Today’s Solutions
Unicast: Hand Gestures, Eye ContactRequires parties to see each other
Broadcast: Honk, ShoutDisturbs othersAgitates/annoys both parties
![Page 4: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/4.jpg)
4
Tomorrow’s Technology: One Possibility
Broadcast using Smartphone/DSRCHonking/shouting in the electronic domainWould cause sensory overload for drivers
![Page 5: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/5.jpg)
5
Fundamental Problem in Targeted V2V Communication
Who is the sender/receiver?
Sender: What is the receiver’s unique address?Receiver: Which vehicle sent message to me?
![Page 6: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/6.jpg)
6
To match vehicles in visual and electronic domains.
Objective
At the same time• Decrease matching time• Increase accuracy• Generate less network traffic
VID: Visual ID assigned by camera (e.g., red/yellow/blue box)
EID: Electronic ID of the vehicle (e.g., IP/MAC address)
![Page 7: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/7.jpg)
7
Available Features
Features Accuracy & UniquenessVehicle Color Not always uniqueGPS Not accurate enoughVehicle Image Not unique, environment dependentPlate number Unique, but hard to readRelative Speed May not be accurate…. ….
A unique set of features known to both vehicles is desired.
![Page 8: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/8.jpg)
8
Main Idea
If single feature is unreliable, can we use multiple features to do matching?
System requirement: Camera, GPS Receiver and Radio
Radio: communicationSmartphone, DSRC
Camera: identify vehiclesSmartphone, Vehicle Security Driving Recorder Camera
![Page 9: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/9.jpg)
9
Challenges
Feature InaccuracyE.g., A blue vehicle might be observed as black.
Heterogeneous CapabilityVehicles may not have smartphone, camera, radio, or may not be running our solution.
Distributed in NatureEach vehicle only knows limited information
![Page 10: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/10.jpg)
10
Vehicle Matching Process
Matching vehicles based on similarity
Estimate similarity between vehiclesWeight Features Cluster {VIDs, EIDs}
Obtain VIDs & EIDsGet VIDs from camera Get EIDs from radio
![Page 11: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/11.jpg)
11
Visual Matrix(from Video-Analysis)
VID N Features
V1 f11 f12 … f1N
V2 f21 f22 … f2N
... … … …. …VM fM1 fM2 … fMN
Vehicles Observed through Camera
VID : Visual ID (camera assigns visual IDs to the observed vehicles)
VID only has local meaning (cannot be used by neighbors)
![Page 12: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/12.jpg)
12
Electronic Matrix(from Electronic Messages)
EID N Features
E1 f11 f12 … f1N
E2 f21 f22 … f2N
... … … …. …EK fK1 fK2 … fKN
IDs received through WiFi/DSRC
EID: Electronic ID (IP address, MAC address, etc.)
![Page 13: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/13.jpg)
13
Create Similarity Matrix
E1 E2 ... EK
V1 … 0.84 ... …
V2 … … ... ...
... ... ... ... ...
VM ... ... ... ...
VID Features
V1 f11 f12 … f1N
V2 f21 f22 … f2N
... … … …. …VM fM1 fM2 … fMN
EID Features
E1 f11 f12 … f1N
E2 f21 f22 … f2N
... … … …. …EK fK1 fK2 … fKN
Electronic Matrix E Visual Matrix V
Similarity Matrix S
S = V ET
![Page 14: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/14.jpg)
14
Adaptive Weight (AW) Algorithm
The Problem:How to combine different features to get the similarity value between two cars?
The Intuition: Features with diversity values are important.
E.g., color provides no information if the cars have the same color
The Solution:Define Feature Distinguishability: the probability that any two observed vehicles are different based on this featureSimilarity of two vehicles: weighted mean of the feature distinguishability values.
![Page 15: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/15.jpg)
15
different lane, different color
Eb
Ea
Matching with Similarity Matrix
V1
me
V2
Visual Domain Electronic Domain
0.99
0.50.5
0.01
Steps• Assign VIDs• Receive EIDs• Calc. Similarity• Remove low
similarity linksdifferent lane, similar color
same lane, different color
same lane, similar color
![Page 16: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/16.jpg)
16
Matching with Similarity Matrix
Greedy Matching Maximal Matching
Weighted Bipartite Graph Matching Problem
Ea
Eb0.9
0.50.5
VIDs EIDs
V2 Ea
Eb0.9
0.50.5
VIDs EIDs
V2
Greedy matching is preferred.
V1 V1
![Page 17: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/17.jpg)
17
Clustering Vehicles
Global distinguishability not requiredNearby cars need to be distinguished
Cluster the cars into smaller groups based on feature distance.
Apply the AW algorithm within clusters
Clustering
![Page 18: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/18.jpg)
18
Experiment
Driving in freeway & local drive with 3 carsUsing smartphone to collect GPS, videoExperiment result
Vehicles with same color leads to low precision
![Page 19: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/19.jpg)
19
SimulationUsing SUMO + NS3Modeled the visibility of neighboring carsModeled car detection prob., color detection accuracy, etc.
ForeSight significantly improves the matching performance!
0.230.18
![Page 20: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/20.jpg)
20
Case Studies: Improve GPS
Each vehicle estimates its location withIts own GPS measurementNeighbors’ estimation of its location (assistance from Nbrs.) Interesting Observations:
• When a car’s GPS error low, it is more likely to be matched by more neighbors.
• The match error increases as the number of neighbors increases: dense traffic makes matching more unreliable.
High vehicle density
![Page 21: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/21.jpg)
21
Case Studies: Reduce Disturbance
Application: Send message to vehicles that are in front but has a slower speedCompare Broadcast, GPS and ForeSight
GroundTruth
Broadcas
tGPS
ForeSig
ht0
50010001500200025003000
# of
Car
s Noti
fied
29× Schemes Notified Vehicles
Ground Truth 1021Broadcast 29 x 1021GPS 2706 (95% recall)ForeSight 1141 (95% recall)
![Page 22: ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain](https://reader035.fdocuments.us/reader035/viewer/2022062811/56815f79550346895dce818b/html5/thumbnails/22.jpg)
22
Eb
E1 E3Ea
E1 E2
Future Work: Conflict Resolving
Conflicts may AppearMatching result computed by different vehiclesMatching result at different time
Possible SolutionCollaboration between neighbors
Eb
E1 E3
E1
Ea
E2
EID
VID