Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting...
Transcript of Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting...
![Page 1: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/1.jpg)
1
SelectingUser-GeneratedVideosforAugmentedRealityApplications
IEEEBigMM 2016
HienTo,Hyerim Park,Seon HoKim,CyrusShahabi.
IntegratedMediaSystemsCenterUniversityofSouthernCalifornia
![Page 2: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/2.jpg)
2
Motivationü Popularityofaugmented-reality(AR)
ü Scarcity ofARcontent
ü Lackofusercontextscausesimpreciseregistration
DifficulttocreatecontentinARbrowsersAdoptmultimediacontentfromsocialmedia
ARbrowsers,e.g.,Layar,Wikitude,andJunaio
Needcontentsourcewithrichcontextinformation
![Page 3: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/3.jpg)
3
Motivation(Cont…)Mobilecamerasareeverywhere!
AnywaytousemobilevideosasARcontent?
![Page 4: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/4.jpg)
4
Motivation(Cont…)
• Currentmostimportantkeywordsinvideomarketareasfollows:(byKleiner PerkinsCaufield &Byers (KPCB), 2015)–User-generated– Tagged– Curated– Indexed&Searchable
![Page 5: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/5.jpg)
5
Challenges
Knownissuesofmobilevideos
1.OutofControl 2.HardtoSearch 3.HardlySystematic
HardtosearchforinterestingvideosforARcontent
![Page 6: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/6.jpg)
6
RelatedWorkü Content-basedretrieval• Extensivecomputationrequired,stillinaccurate
ü Spatialmetadata(cameralocation,orientation)• Geo-taggingrequired,sensorsonmobiledevice
𝑁𝑑
pR𝜃
p:cameralocationd:cameradirection𝜃:viewableangleR:viewabledistancet:timestamp
FieldOfView(FOV)
MediaQ:mobilemediamanagementsystemusingspatialmetadatafromsensors(http://mediaq.usc.edu)
![Page 7: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/7.jpg)
7
Geo-taggedMobileVideos
üSelectingvideosforacertainareaisnowfast!
![Page 8: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/8.jpg)
8
ProblemStatementüSimplegeospatialrelevanceisnotenough.Forexample,selecting“interesting”videosforARapplicationsfrommanyatalocation?
üIdentifyinteresting(orsignificant)videostoARusersasasequenceofFOVsthatfollowaparticularcamerashootingpattern usedinfilmingsuchastracking,zooming,panning,andarchingscenes
![Page 9: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/9.jpg)
9
ModelingSignificantVideoSegmentsSingle Multiple
SingleZoom Pan
MultipleTrack Arch
PositionDirection
1d2d
3d
4d5d
1d2d 3d 4d
5dθ
1d
2d3d
4d
5d1d
2d3d
4d 5d
3θ4θ 5θ
0
< 𝑡'(), 𝑑'+, >
< 𝑡'(), 𝑑'+, , 𝑟'+, > < 𝑡'(), 𝑑'(), 𝑟'+, >
< 𝑡'(), 𝑑'+, >
![Page 10: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/10.jpg)
10
Statistics ValueNumberofvideoswithgeo-metadata 2,397Averagelengthper videowithcontent(sec) 72.14Averagecameramovingspeed(km/h) 4.5Averagecamerarotationspeed(degrees/sec) 10Numberofusers 289Numberofvideos byeachuser 8.29NumberofFOVs 208,978NumberofFOVpersecond 1.03NumberofFOVpervideo 74.16
[ACMMMSys 2016DatasetPaper,USCMediaQMobileVideoData]
ExperimentUsingRealDatasets
![Page 11: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/11.jpg)
11
SignificantVideoSegments
Numberofscenesfound Searchingtime(onevideo)
0
500
1000
1500
2000
Track Arch Zoom Pan
Num
bero
fscenes
051015202530
Track Arch Zoom Pan
Runtime(m
s)
𝑡'() = 15, 𝑑'+, = 15, 𝑑'() = 120
Trackingisthemostpopularwayofcapturingmobilevideos
Overallsearchtimeisfast
![Page 12: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/12.jpg)
12
ZoomingScene
TrackingScene
PanningScene
ArchingScene
![Page 13: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/13.jpg)
13
DetectingHotSpots
Hotspotscoveredbymanyvideoframes
0
200
400
600
800
1000
100 90 80 70 60 50 40 30 20 10Totalnum
berofhotspots
MinimumFOVsperhotspot
![Page 14: Selecting User-Generated Videos for Augmented Reality ...hto/resources/BigMM2016.pdf · 1 Selecting User-Generated Videos for Augmented Reality Applications IEEE BigMM 2016 Hien To,](https://reader034.fdocuments.us/reader034/viewer/2022050415/5f8b84bd2a2e834405681760/html5/thumbnails/14.jpg)
14
ConclusionandFutureWork
üProposedmodelsandalgorithmstoidentifyinterestingvideosegmentsandhotspots
üExperimentsonrealgeo-taggedvideodatashowthatthealgorithmsarefastandabletoidentifyinterestingscenes
üWillconsiderusermobilityandrankingofvideosegments