Photographer Paths: Sequence Alignment of Geotagged Photos for Exploration-based Route Planning
-
Upload
abdo-el-ali -
Category
Technology
-
view
993 -
download
1
description
Transcript of Photographer Paths: Sequence Alignment of Geotagged Photos for Exploration-based Route Planning
Photographer Paths: Sequence Alignment of Geotagged Photos for Exploration-based Route Planning!
Abdallah ‘Abdo’ El Ali Sicco van Sas
Frank Nack
Feb. 26, 2013
h6p://staff.science.uva.nl/~elali/
Outline!
I. Introduc3on
II. Photographer Paths
III. User Evalua3on
IV. Results
V. Discussion & Future Work
2
3
Introduction
4
5
6
Pffttt…
We don’t always want to supply user preferences
7
Social and local interpreta3on of city places and routes 8
Off-‐the-‐beaten track, social trails, ≠ Lonely Planet!
9
Assump3on:
Loca3ons of photographs are poten3ally interes3ng
10
But sequen3al property needs to be captured!
11
Sequence Alignment methods
12
Analysis of mobility behavior of city photographers:
where photographers have been
in what order they have been there
how closely their movements parallel those of other photographers
13 By Keiichi Matsuda via supercolossal
Research Questions!
How can walkable route plans be automaCcally generated for residents (and tourists) that would like to explore a city?
And are these route plans desirable?
14
Three factors:
1) Which data sources? 2) Which methods to generate routes?
3) User percep3ons compared to fastest and popular routes?
15
Photographer Paths!
Approach!
1) Crawl Flickr geotags, 3mestamps
2) Map each geotag/loca3on in a sequence to a cell in a par33oned grid map
3) Mul3ple Sequence Alignment on photographer routes to find aligned loca3on sequences
These alignments are Photographer Route Segments (PRSs)
16
Dataset!
Flickr geotagged photos within Amsterdam, The Netherlands
Area: 17.3 km N-‐S and 24.7 km E-‐W (center)
5-‐year period (Jan. 2006 -‐ Dec. 2010)
Aeributes:
owner ID
photo ID
date and 3me-‐stamp
la3tude and longitude (street level accuracy)
Database: 426,372 photos
17
Preprocessing!
Sequence inference with following constraints:
photo taken within 4 hours from previous photo and in same order
minimum 2 or more different loca3ons (or nodes)
early experiments determined 125 x 125m cells in center of Amsterdam grid suitable
1691 routes (average length of 9.92 loca3ons)
1130 unique photographers
18
!"#$%
##&%
!'(%
)'%
'!&%
!*!%
!*%
!"#$
+'% !%*%
+**%
"**%
#**%
$**%
!***%
!+**%
!"**%
!#**%
+% )% "% '%
%&'()*+$,*-.*)/*0$
1)'-.*$&2/342)0$
Photographer Route Segments (PRSs)!
+%,-./.012,-314%
)%,-./.012,-314%
"%,-./.012,-314%
Sequence Alignment!
To find photographer paths from Photographer Route Segments (PRSs), constraints set:
PRS has minimum 4 photographers with minimum 2 aligned nodes/loca3ons
231 PRSs
(average length of 2.61 nodes)
19
PRSs in Amsterdam!
20
PRS Aggregation!
Modified Dijkstra’s shortest path algorithm
21
Start
P1N2
EndP2N2
P1N1 P2N1
!
"
#$%
&
!
P1N3
&
#
!"#
PRS Aggregation to Crude Routes!
22
WW Photographer Route
CM Photographer RoutePRSs
23
User Evaluation
Laboratory Study Design!
~45 min. Quan3ta3ve/Qualita3ve lab-‐based study
15 par3cipants (10 m, 5 f) aged between 21-‐35 (M = 29.2; SD = 3.3)
Interac3ve web-‐based prototype route planner
Expert route evalua3on by ‘city residents’ (lived in Amsterdam > 1 year)
Plain routes to avoid informa3on type bias
24
Laboratory Study Design!
Two scenarios:
Route 1: Central Sta3on to Museumplein anernoon scenario favoring explora3on
Route 2: Waterlooplein to Westerkerk evening scenario favoring efficiency
Baseline comparisons:
Photo Density (PD) route: highest density of photos (over 5 year period) in grid cells along route
Google Maps (GM) route: shortest route between two loca3ons
Counterbalanced within-‐subject design
Route Varia3on (IV): Photographer Paths vs. Photo Density vs. Google Maps
25
Central Station to Museumplein (CM)!
26
Photographer Paths route (5.36 km)
Photo Density route (3.83 km)
Google Maps route (3.35 km)
Waterlooplein to Westerkerk (CM)!
27
Photographer Paths route (2.28 km)
Photo Density route (2.60 km)
Google Maps route (1.59 km)
Laboratory Study Design!
Data collected:
1. AerakDiff2 (Hassenzahl, 2003) UX ques3onnaire responses [7-‐point seman3c differen3al scale]: Usability, Hedonic Quali3es (Iden3ty, S3mula3on), Aerack3veness
2. Two-‐part semi-‐structured interviews
Part 1: Route preferences, feedback on Photographer Paths
Part 2: Inves3ga3on of visualized informa3on types (visualized info type handouts): a) Google maps b) Color coded PRSs (PP route)
c) Density geopoints (PD route) d) Thumbnail photo geopoints
e) Foursquare POIs
28
29
a)
b)
c)
d)
e)
Web Survey Study!
Short web-‐based survey for CM and WW routes and varia3ons
Basic demographics collected: age, gender, years in Amsterdam
Sta3c route images, no counterbalancing
82 par3cipants (55 m, 27 f) aged between 17-‐62 (M= 27.6; SD= 6.1)
Most lived in Amsterdam for more than 3 years (44/82)
Some between 1-‐3 years (15/82)
Less than a year (11/82) Only visited before (12/82)
30
31
Results
AttrakDiff2 !
32
Central Station to Museumplein (CM) Route
!"#
!$#
!%#
&#
%#
$#
"#
'()*+),-#./)0123#4'.5# 6789:1-#./)0123#!#;87:,23#46.!;5#
6789:1-#./)0123#!#<,+/0),9:#46.!<5#
=>()-,?7:7@@#4=AA5#
!"#$%
&'("$)'*$%
'B929*()CB7(#')2B@# 'B929#D7:@123# E99*07#F)C@#** !"#"$%$&
******
******
**
* !"#"$%$'
AttrakDiff2 !
33
!"#
!$#
!%#
&#
%#
$#
"#
'()*+),-#./)0123#4'.5# 6789:1-#./)0123#!#;87:,23#46.!;5#
6789:1-#./)0123#!#<,+/0),9:#46.!<5#
=>()-,?7:7@@#4=AA5#
!"#$%
&'("$)'*$%
'B929*()CB7(#')2B@# 'B929#D7:@123# E99*07#F)C@#** !"#"$%$&
****
****
Waterlooplein to Westerkerk (WW) Route
Route Preference!
Lab Study
CM route: most chose to follow the PP route (9/15), PD route (4/15), GM route (2/15)
“One of the routes [PP] was long and took many detours, and I thought that was a very aFracHve route!”
WW route: most chose to follow GM route (10/15), PD route (4/15), no route (1/15)
“You are going for coffee so you just want to get there, unlike in the first [CM] scenario where it is a nice day and you have Hme.”
34
Web Survey
• CM route: GM (40/82), PD (23/82), PP route (10/82), neither (9/82) • No experimenter steering; many Amsterdam residents know the city already quite well! • “I would not easily walk these routes... who in Amsterdam walks? ;)”
• WW route: GM (67/82), PD (6/82), PP route (3/82), neither (6/82)
Digital Information Aids!Lab Study
Interview: Part 1 “How many persons (focus on city photographers) took a given route segment over a certain 3me
period (e.g., 1 year)?”
Useful (8/15) for exploring a city one already knows
Not sure (4/15)
Depends on which photographers (2/15)
Not for me (1/15)
Interview: Part 2 Found PP info type aerac3ve (10/15), but combine with Photo thumbnails (3/10) and POIs (3/10)
35
Digital Information Aids!
Web Survey
POIs along a route (51x)
Route distance (51x)
Comments along a route (ranked by highest ra3ngs or recency) (24x)
Expert travel guides (22x)
Photos of route segments (17x)
No digital aids (13x)
Number of photographers that took a given path over a Hme period (9x)
Number of photos along a route over a 3me period (9x)
36
37
Discussion
Discussion!
Discrepancy between lab-‐study and web survey
Quick web survey insufficient? Visualiza3on/explana3on of digital aids important?
Proof-‐of-‐concept approach requires real-‐world ‘outdoor’ evalua3on
Different street grid network
Scalability to larger ci3es
More context-‐awareness
38
Take Home Message!
Going towards data-‐driven explora3on-‐based route planners…
Some3mes it’s the journey, not the des3na3on
A quan3ta3ve approach may oversimplify human needs for explora3on
But some3mes we want an automa3c solu3on, so as not to be bothered with supplying user preferences and encounter serendipity
39
40
Questions
h6p://staff.science.uva.nl/~elali/
References! 1. Cheng, A.-‐J., Chen, Y.-‐Y., Huang, Y.-‐T., Hsu, W. H., and Liao, H.-‐Y. M. Personalized travel recommenda3on by mining people aeributes from community-‐contributed photos. In Proc. MM ’11, ACM (2011), 83–92. 2. M. Clements, P. Serdyukov, A. P. de Vries, and M. J. Reinders. Using flickr geotags to predict user travel behaviour. In Proc. SIGIR ’10, pages 851–852. ACM Press, 2010. 3. M. De Choudhury, M. Feldman, S. Amer-‐Yahia, N. Golbandi, R. Lempel, and C. Yu. Automa3c construc3on of travel i3neraries using social breadcrumbs. In Proc. HT ’10, pages 35–44. ACM Press, 2010. 4. F. Girardin, F. Calabrese, F. D. Fiore, C. Ra|, and J. Blat. Digital footprin3ng: Uncovering tourists with user-‐generated content. IEEE Pervasive Compu3ng, 7:36–43, October 2008. 5. N. Shoval and M. Isaacson. Sequence alignment as a method for human ac3vity analysis in space and 3me. Annals of the Associa3on of American Geographers, 97(2):282–297, 2007. 6. A. Vaccari, F. Calabrese, B. Liu, and C. Ra|. Towards the socioscope: an informa3on system for the study of social dynamics through digital traces. In Proc. GIS ’09, pages 52–61. ACM Press, 2009. 7. Hassenzahl, M., Burmester, M., and Koller, F. AerakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragma3scher Qualit¨at. Mensch & Computer 2003. Interak3on in Bewegung (2003), 187–196. 8. Lu, X., Wang, C., Yang, J.-‐M., Pang, Y., and Zhang, L. Photo2trip: genera3ng travel routes from geo-‐tagged photos for trip planning. In MM ’10, ACM (2010), 143–152. 9. Wilson, C. Ac3vity paeerns in space and 3me: calcula3ng representa3ve hagerstrand trajectories. TransportaHon 35 (2008), 485–499.
41
Related Work! Sequence Alignment (SA) methods:
Borrowed from bioinforma3cs and later 3me geography
Time geography systema3cally analyzes and explores the sequen3al dimension of human spa3al and temporal ac3vity (Shoval & Isaacson, 2007).
Visualize human movement on 2-‐D plane: x-‐ & y-‐ axis longitude and la3tude; z-‐axis 3me
useful for analyzing sequences of human ac3vity (in this case, photo-‐taking behavior of photographers)
Photo-‐based City Modeling: Understand tourist site aerac3veness based on geotagged photos (Girardin et al., 2008) Construct inter-‐city travel i3neraries (De Choudhury et al., 2010) Generate personalized Point-‐of-‐Interest (POI) recommenda3ons of where to go in a city based on
the user's travel history in other ci3es (Clements et al., 2010)
Approaches focus on describing loca3ons, not on fine-‐grained within-‐city routes that connect them
Non-‐efficiency Driven Route Planners
Automa3c genera3on of travel plans based on millions of photos (Lu et al., 2010) Personalized data-‐driven travel route recommenda3ons (Cheng et al. 2011)
Systems geared towards recommending hotspots and popular routes, not off-‐beat explora3on routes 42
Sequence Alignment Overview!Input: two sequences over the same alphabet Output: an alignment of the two sequences
Example: Source: GCGCATGGATTGAGCGA
Target: TGCGCCATTGATGACCA
A possible alignment: -‐GCGC-‐ATGGATTGAGCGA TGCGCCATTGAT-‐GACC-‐A
Three opera3ons (each with cost):
Perfect matches (MATCH)
Mismatches (DEL)
Inser3ons & dele3ons (INDEL)
The less distance cost, the higher the similarity between two sequences
43 (Shoval & Isaacson, 2007)
Multiple Sequence Alignment Overview! Used ClustalTXY sonware (Wilson et al., 2008) for photo alignment:
makes full use of mul3ple pairwise sequence alignments, where alignments are computed for similarity in parallel
uses a progressive heuris3c to apply mul3ple sequence alignment (MSA)
allows elements to be represented with up to 12-‐character words, which allows unique representa3on of small map regions, used for represen3ng the geotagged photos
to deal with differences in sequence length, ClustalTXY adds gap openings and extensions to sequences.
MSA in 3 stages:
1) Pairwise alignments are computed for all sequences
2) Aligned sequences are grouped together in a dendogram based on similarity
3) Dendogram used as a guide for mul3ple alignment
44