Fly See Buy
Barry KolleeSelvi RatnasingamSylvia van SchieWouter StuifmeelRobert Jan Prick
INTELLIGENT INTERACTIVE SYSTEMS | OCTOBER 8TH, 2013
Introduction: Topics
- ambient intelligence
- mobile technology
Introduction: Topics
ambient intelligence (en.wikipedia, 7/9/2013):
“ambient intelligence (AmI) refers to electronic environments that are sensitive and responsive
to the presence of people”
Introduction: Topics
Mobile technology (en.wikipedia, 7/9/2013):
“a standard mobile device has gone from being no more than a simple two-way pager to being
a mobile phone, GPS navigation device, an embedded web browser and instant messaging
client, and a handheld game console”.
Introduction: Vision
Weiser, M. The Computer for the 21st Century, Scientific American (1991)
Introduction: Ambient Intelligence
Weiser, M. The Computer for the 21st Century, Scientific American (1991)
“Specialized elements of hardware and software, connected by wires, radio waves and
infrared, will beso ubiquitous that no one will notice their
presence”
Introduction: Mobile technology
“Little is more basic to human perception than physical juxtaposition, and so ubiquitous
computers must know where they are. If a computer knows merely what room it is in, it can adapt its behavior in significant ways….”
Weiser, M. The Computer for the 21st Century, Scientific American (1991)
Introduction: Vision
Weiser, M. The Computer for the 21st Century, Scientific American (1991)
Introduction: Goal
“When things disappear ….. we are freed to use them without thinking and so to focus on
new goals”
Weiser, M. The Computer for the 21st Century, Scientific American (1991)
The Big Data Challenge
● Lausanne Data Collection Campaign
● A large-scale mobile data resource
● ‘Privacy by design’
● Image logfiles
● Monitor entire smartphone (N95)
Laurila, J. K., Gatica-Perez, D., Aad, I., Blom, J., & Bornet, O.The mobile data challenge: Big data for mobile computing research. , http://privacybydesign.ca/
The Big Data Challenge
Laurila, J. K., et al.The mobile data challenge: Big data for mobile computing research.
The Big Data Challenge
● Semantic place prediction
● Next place prediction
● Demographic attribute prediction
Laurila, J. K., et al.The mobile data challenge: Big data for mobile computing research.
Applying to Fly See Buy system
“Each data type corresponds to a table in which each row represents a record such as a phone call or an observation of a WLAN access point. User IDs and
timestamps are the basic information for each record.“
Laurila, J. K., et al.The mobile data challenge: Big data for mobile computing research.
How long?
● Investigate properties of learning
● Predicting social and individual models
Altshuler, Y., (2012). Incremental learning with accuracy prediction of social and individual properties from mobile-phone data.
‘Reality mining’
A. Pentland, in The Global Information Technology Report 2008-2009 (World Economic Forum, Geneva, 2009)
Methodology
● Classifiers○ Personal properties (first level)○ Social links (life-partner?)
● Correlation amount of time vs. accuracy
Altshuler, Y., Aharony, N., Fire, M., Elovici, Y., & Pentland, A. S. (2012). Incremental learning with accuracy prediction of social and individual properties from mobile-phone data.
Methodology
● Android Application○ GPS○ Accelerometer○ Third Party application○ Cell tower ID’s○ WIFI LAN ID’s (proximity)
Altshuler, Y., Aharony, N., Fire, M., Elovici, Y., & Pentland, A. S. (2012). Incremental learning with accuracy prediction of social and individual properties from mobile-phone data.
Methodology
● Feature vector○ Location○ Sms-pattern○ Internet usage○ Call-pattern○ Phone applications○ Alarms
● Friends and family dataset (140 people)
Altshuler, Y., Aharony, N., Fire, M., Elovici, Y., & Pentland, A. S. (2012). Incremental learning with accuracy prediction of social and individual properties from mobile-phone data.
Conclusion
● Ethnicity 60%
● Is student
● Significant other 65 %
Altshuler, Y., Aharony, N., Fire, M., Elovici, Y., & Pentland, A. S. (2012). Incremental learning with accuracy prediction of social and individual properties from mobile-phone data.
Conclusion
Modeled using ‘gompertz function’
“At a moment I can say with an amount of certainty who you are.”
Laurila, J. K., Gatica-Perez, D., Aad, I., Blom, J., & Bornet, O.The mobile data challenge: Big data for mobile computing research.
AdNext: A Visit-Pattern-Aware Mobile Advertising System | for Urban Commercial Complexes
Keywords
● Mobile advertising
● Sequential visit patterns
● Prediction models
● Wi-Fi localization
● User survey
Kim, B., Ha, J., Lee, S., Kang, S., Lee, Y., Rhee, Y., . . . Song, J. (2011). AdNext: A visit-pattern-aware mobile advertising system for urban commercial complexes. Proceedings of the 12th Workshop on Mobile Computing Systems and Applications, Phoenix, Arizona. 7-12.
COEX Mall
● Largest commercial complex in South Korea
● 260 stores / 100.000 visitors per day
● Customer targeting
○ Spatial relevance
○ Temporal relevance
Image: Official Site of Korea Tourism http://www.visitkorea.or.
kr/enu/SI/SI_EN_3_1_1_1.jsp?cid=736121Kim, B. et al AdNext: A visit-pattern-aware mobile advertising system for urban commercial complexes. Proceedings of the 12th Workshop on Mobile Computing Systems and Applications, Phoenix, Arizona. 7-12.
AdNext System Architecture
Kim, B. et al AdNext: A visit-pattern-aware mobile advertising system for urban commercial complexes. Proceedings of the 12th Workshop on Mobile Computing Systems and Applications, Phoenix, Arizona. 7-12.
Collecting Place Visit History
● store-level localization accuracy
● identify users’ current location (using accelerometer)
● detect in/out time (location change validation)
Kim, B. et al AdNext: A visit-pattern-aware mobile advertising system for urban commercial complexes. Proceedings of the 12th Workshop on Mobile Computing Systems and Applications, Phoenix, Arizona. 7-12.
Next Visit Prediction Model 1/2
Bayesian Networks – Probabilistic graphical modelIt models the joint probability P(X, Y), where X represents features and Y represents labels. Main features:● visit place (P)● visit time (T)● visit duration (D)● gender (G) - static● age (A) - static
Kim, B. et al AdNext: A visit-pattern-aware mobile advertising system for urban commercial complexes. Proceedings of the 12th Workshop on Mobile Computing Systems and Applications, Phoenix, Arizona. 7-12.
Next Visit Prediction Model 2/2
Kim, B. et al AdNext: A visit-pattern-aware mobile advertising system for urban commercial complexes. Proceedings of the 12th Workshop on Mobile Computing Systems and Applications, Phoenix, Arizona. 7-12.
Accuracy
● Data collection
● Prediction accuracy
● Comparison
Kim, B.et al (2011). AdNext: A visit-pattern-aware mobile advertising system for urban commercial complexes. Proceedings of the 12th Workshop on Mobile Computing Systems and Applications, Phoenix, Arizona. 7-12.
Discussion
● Privacy concerns
● Energy consumption
Kim, B.,et al (2011). AdNext: A visit-pattern-aware mobile advertising system for urban commercial complexes. Proceedings of the 12th Workshop on Mobile Computing Systems and Applications, Phoenix, Arizona. 7-12.
Context-aware Media Agent for Public Spaces | Guide visitors in Museums
Authors: Ichiro Satoh
Satoh, Ichiro. Context-aware Media Agent for Public Spaces. Lansdale PA 19446: IOS Press, doi:10.3233/978-1-60750-606-5-407, 0. Print. Ichiro, Satoh, (2008) Context-Aware Agents to Guide Visitors in Museums, National Institute of Informatics, Tokyo, Japan Pages 441-455,
Portable terminals are not desirable
● requires end-users to carry players along
● require explicit input
● expensive devices to lend
● require regular maintenance
● interacting difficulties
● prevent visitors from focusing on the exhibits
Satoh, Ichiro. Context-aware Media Agent for Public Spaces. Lansdale PA 19446: IOS Press, doi:10.3233/978-1-60750-606-5-407, 0. Print. Ichiro, Satoh, (2008) Context-Aware Agents to Guide Visitors in Museums, National Institute of Informatics, Tokyo, Japan Pages 441-455,
Agent runtime system
Mobile agent
● annotation
● navigation
● user preference
Satoh, Ichiro. Context-aware Media Agent for Public Spaces. Lansdale PA 19446: IOS Press, doi:10.3233/978-1-60750-606-5-407, 0. Print. Ichiro, Satoh, (2008) Context-Aware Agents to Guide Visitors in Museums, National Institute of Informatics, Tokyo, Japan Pages 441-455,
User navigation patterns
Satoh, Ichiro. Context-aware Media Agent for Public Spaces. Lansdale PA 19446: IOS Press, doi:10.3233/978-1-60750-606-5-407, 0. Print. Ichiro, Satoh, (2008) Context-Aware Agents to Guide Visitors in Museums, National Institute of Informatics, Tokyo, Japan Pages 441-455,
Context-aware Annotation in Museum
Satoh, Ichiro. Context-aware Media Agent for Public Spaces. Lansdale PA 19446: IOS Press, doi:10.3233/978-1-60750-606-5-407, 0. Print. Ichiro, Satoh, (2008) Context-Aware Agents to Guide Visitors in Museums, National Institute of Informatics, Tokyo, Japan Pages 441-455,
Conclusion
This paper only described the design and implementation of an agent-based system for building and operating context-aware visitor-guide services in public museums.
Satoh, Ichiro. Context-aware Media Agent for Public Spaces. Lansdale PA 19446: IOS Press, doi:10.3233/978-1-60750-606-5-407, 0. Print. Ichiro, Satoh, (2008) Context-Aware Agents to Guide Visitors in Museums, National Institute of Informatics, Tokyo, Japan Pages 441-455,
Need a location-based sensor to track
the user.
Most used sensors
● RFID
● NFC
● WiFi
Probability kernel regression for WiFi localisation
How the KL-divergence kernel regression algorithm bridges the gap with other WiFi
localisation algorithm?
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.
WiFi localisation based on
● Fingerprints
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.
Fingerprint?
The received signal strength (RSS) values from multiple access points (AP) are collected at different reference points and these reference points with related RSS are
referred as fingerprints.
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.
Pattern matching process
Apply the learned model to real-time RSS sample.
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.
Fingerprint
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.
Apply different statistical algorithm
● K-nearest neighbour● Artificial neural network
● KL-divergence kernel regression algorithm
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.
Contrasting dataset
● Office building (with dense and repeated fingerprints)
● Auditorium (a large open space)
● Public space (with mixed layouts and heavy pedestrian traffic)
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.
Challenges
● Device-independent measurements
● Conditional independence of the RSS from a single AP
● Choosing parameters, like N fingerprints, N RSS sample
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.
Conclusion
Very flexible and generalise several existing WiFi localisation algorithm
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.
Ambient system
● Project goals○ Personalised recommendations for venues
(shopping, dining, etc.) in an airport terminal environment
○ Recommendations are represented in an ambient system in the public space
○ More foot traffic to airport venues○ Increased revenue of venues
Points by Breakfast NY
How do you distinguishan ambient display from a regular
display?
Designing interactivity awareness for ambient displays
● Addresses particular aspects of interaction regarding ambient displays
● How can users make out which display is interactive, and which isn't?
● How can users tell which type of interface the ambient display exposes?
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
Interaction techniques
A. Using software installed on the mobile device
B. Natural interaction via pointing and gestures
C. Touch-based interaction
D. Combining mobile phones and gestures
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
Example of public display
Clear Channel gaat digitaal op Amsterdam CS. (2012, December 4). Clear Channel Hillenaar - Buitenreclame. Retrieved October 7, 2013, from http://www.clearchannel.nl/nl/Nieuws/Laatste-nieuws/Default.aspx?PageId=535
Study
● Study to investigate perception of interactivity of displays
● Design○ Repeated-measures design, 3 independent
variables:■ Display■ Interactivity■ Reachability
○ Dependent variable:■ Response
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
Test data
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
Test results
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
Test results
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
Test results
● Accurate responses: only 54.9%
● But: participants were more accurate than being wrong or undecided (33% wrong)
● Participants were prone to mark normal displays as ambient, and vice versa
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
Factors for successful ambient awareness
● Location
● Installation
● Reachability
● Content
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
Improving ambient awareness
● Clear symbols● Instruction video● Instruction manual, text● Metaphorical pictures
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
Conclusions
"How we should interact with ambient content is still to be answered"- Vatavu R. D.
● Perceptions of ambient displays are still sometimes wrong
● Designers need to think of clever implementation of the system based on the four factors
● Future work is still needed for better understanding
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and Applications, 1-22.
How do you recommend venues using input from a user and contextual circumstances?
Location-based recommendation system using bayesian user's preference model
● Goal:○ Reflecting individual preference at a proper time
● Map-based personalised recommendation system○ Recommended restaurants nearby○ User's preference modeled by Bayesian networks
● Input from user and contextual information
Park, M., Hong, J., & Cho, S. (2007). Location-based recommendation system using bayesian user's preference model in mobile devices.4611, 1130-1139
Bayesian model
● Set of variables and their conditional dependencies
Park, M., Hong, J., & Cho, S. (2007). Location-based recommendation system using bayesian user's preference model in mobile devices.4611, 1130-1139
System design
Park, M., Hong, J., & Cho, S. (2007). Location-based recommendation system using bayesian user's preference model in mobile devices.4611, 1130-1139
User input
● User profile○ Age, Gender, Blood type, Has car,
Income, Food preference● User request
○ Class, Mood, Price, Parking Area
Park, M., Hong, J., & Cho, S. (2007). Location-based recommendation system using bayesian user's preference model in mobile devices.4611, 1130-1139
Contextual input
● Web, system, GPS and application data will be preprocessed into context log
● Live information (location, weather, time)is taken intoaccount with userrequest
Park, M., Hong, J., & Cho, S. (2007). Location-based recommendation system using bayesian user's preference model in mobile devices.4611, 1130-1139
Output
● Top 2 recommended restaurants per typeof meal○ Based on personal
preferences andcontextualcircumstances
● At least one matchon personalpreference
Park, M., Hong, J., & Cho, S. (2007). Location-based recommendation system using bayesian user's preference model in mobile devices.4611, 1130-1139
Conclusions
● Result is a proposition of a BN-based recommendation system○ It reflects user's preference using a user profile and
context information
● Usability and additional stability tests are yet to be done
Park, M., Hong, J., & Cho, S. (2007). Location-based recommendation system using bayesian user's preference model in mobile devices.4611, 1130-1139
Debate
FOR OR AGAINST #1
The Fly See Buy, personal, efficient and preference based navigation application for airports is
something I would consider using.
GREEN RED( free beer afterwards )
FOR OR AGAINST #2
Navigation by ambient displays is a useful addition to a mobile interface.
GREEN RED
FOR OR AGAINST #3
If one uses a portable device for navigation, can we speak of a ambient system (definition Weiser).
GREEN RED
Final Discussion Question
Sign Post vs Mobile Device?The mobile device is used for preference input in both cases.
Top Related