@tomchapman

The multi-touch screen of the iPhone introduced in 2007, revolutionized the handset screen, and triggered a major momentum towards touch screen for all sizes of display.

As a result, Apple ushered in a dramatic change in behaviour. We as consumers now come to expect to interact with screens.

However, this user interaction with technology is already going above the glass.

With the ability of technology, like the Microsoft Kinect to see users’ movements in space, touch less interactivity such as gestures, are being added to traditional methods in new layers of interaction. You no longer need an explicit tool or even direct manipulation to drive a user interface.

Beyond gestures, a new pattern of computing is emerging where interactions with technology will be conversational. Voice recognition technology has finally hit its tipping point of capability and the stage is being set for a generation of users to start assuming voice control - just as touch control is now assumed for any screen.

Looking forward, ubiquity of connections & connectivity will result in everything becoming interactive or ‘pervasive’. Our interactions with physical and digital will therefore become fluid and frictionless.

The following presentation will provide an analysis of Interactive Technology. We will explore present and future trends as they pertain to business, culture and society. Where available we have used current examples to bring to life each trend.By identifying change we hope to provide stimulus for readers to contemplate how present and future innovation may affect our business and clients. The key trends for Interactive Technology: • Touch• Objects• Augmented Reality / Holographics• Gestures• Facial Recognition• Eye Tracking Sound• Thought• Smell

TOUCH

LCG presents

INTERACTIVE TECHNOLOGY

LCG presentsINTERACTIVE TECHNOLOGY TOUCH

Touch Screen Smart WindowIn 2011 Samsung introduced the transparent smart window; a large, transparent one-way mirror touch screen that has a digital overlay of touch controlled widgets. Short term the smart window will disrupt traditional retail displays adding a layer of digital to the display. OLED technologies with digital overlay are being already in use as vending machines and refrigerators in Japan. In the long-term OLED touch screen technology will be used for multiple surfaces in the home and in retail environments.

TOUCH

Multi-Touch Smart WindowMoving beyond the poking action of a smart window touch screen, multi-touch tech enables a user to interact with a system with more than one finger at a time. Extending the Samsung smart window, high-spec multi-touch displays are already being researched as GM have displayed with their in-car window allowing for pinching and sweeping interactivity.

LCG presentsINTERACTIVE TECHNOLOGY

TOUCH

Multi-Touch Tables and WallsMulti-touch sensing devices such as Flatfrog’s interactive table and Adidas interactive wall can accommodate multiple users at one time, allowing for simultaneous interactive displays within retail environments, museums, tradeshows and exhibitions.

LCG presentsINTERACTIVE TECHNOLOGY

TOUCH

Enhancing Interaction on Touch SurfacesFurther developments in multi-touch are around the anatomical detection of a human finger allowing different parts to be recognized such as the tip, knuckle, nail and pad - researchers TapSense, amongst others are experimenting in this area.

LCG presentsINTERACTIVE TECHNOLOGY

TOUCH

Enhancing Interaction on Touch Surfaces - HapticsExtending touch, Disney Research have been looking at new haptics technology for enhancing touch interfaces with tactile feedback, allowing the user to feel virtual elements through touch. Leading innovator Senseg at CES 2012 demoed their haptic user experience for touch screens. Samsung Nexus One and HTC Sense include Haptic feedback. Watch out for more haptic technology in tablets and smart phones.

LCG presentsINTERACTIVE TECHNOLOGY

TOUCH

Every Surface Can Become A Touch Screen – User Defined InterfaceOmniType is a virtual projection wearable system that enables graphical, interactive, multi-touch input on arbitrary, everyday surfaces. Celluon have manufactured the ‘magic cube’ a virtual projected keyboard and multi-touch mouse.

LCG presentsINTERACTIVE TECHNOLOGY

OBJECTS

LCG presents

INTERACTIVE TECHNOLOGY

OBJECTS

Microsoft Enhance Object Recognition TablesMulti-touch screens allow for object recognition and object based play.For a few years now, multi touch tables such as Microsoft ‘Surface’ have provided a platform for developers to link interactive objects with the screen. With Amnesia Connect users can literally see-through their device and share content as easy and tactile as it can get. It is perfectly suited for sharing visual data like images and video, but works for any other type of content as well.

LCG presentsINTERACTIVE TECHNOLOGY

OBJECTS

Disney Turns iPad Into Interactive TrackAs multi-touch screen devices such as smart phones and tablets become ubiquitous, the opportunities to link physical objects with a smart device with an application have opened up business opportunities. In 2011 Disney released Appmates ‘Cars’ toys, transforming the iPad screen into a game board when the cars come into contact with the touch screen interface.

LCG presentsINTERACTIVE TECHNOLOGY

OBJECTS

Hasbro Developing zAppedTablet GamesToy company Hasbro is currently producing interactive tablet ‘zAPPed’ games such as the Game of Life, Monopoly and Battleships due to for release in June 2012.

LCG presentsINTERACTIVE TECHNOLOGY

OBJECTS

Extended the TouchScreen via Interactive ObjectsMichael Knepful a researcher at NYU has looked at the opportunity to extend touch screens via physical, mechanical and electrical devices in a number of ways that can be incorporated by many products, and opens up creative possibilities for brands that want to develop applications linked to physical objects.

LCG presentsINTERACTIVE TECHNOLOGY

OBJECTS

Microsoft Illumishare IllumiShare a research project from Microsoft allows users to interact remotely with physical objects. IllumiShare features a lamp-like peripheral device that projects light onto your desktop. Anything that appears within the lit space is then projected onto the analogous space of a fellow IllumiShare user. That setup then allows the users to interact with any object within the spaces--allowing for some amazing physical-meets-digital chess or card games, play dates or tutoring sessions

LCG presentsINTERACTIVE TECHNOLOGY

OBJECTS

Wearable TechnologyWearable technology is an area that is currently being researched. The idea of carry technology by wearing it as opposed to carrying it is intriguing. At present Microsoft Research is looking into wearable tech such as embedding RFID and NFC within clothes that can interact with other tech. They developed a printing dress that displays tweets based on what you type.

See Augmented Reality chapter for AR Glasses.

LCG presentsINTERACTIVE TECHNOLOGY

AUGMENTED REALITY

LCG presents

INTERACTIVE TECHNOLOGY

AUGMENTED REALITY

AR MarkersThe Majority of AR experiences to date are achieved using devices with a camera and touch screen. There are many applications and that incorporate AR tech allowing users to lock onto a marker and experience virtual content. Five by Five used this tech and approach for Dynamo Magician Impossible in 2011.

LCG presentsINTERACTIVE TECHNOLOGY

AUGMENTED REALITY

Markerless ARAurasma is just one company pushing the boundaries of AR. They have developed a visual browser whereby virtual content can be served up without markers, making the experience frictionless.Markerless AR has been incorporated within virtual demo apps by Bandai for its toy packaging. Sony Music developed an AR experience for their Tings Tings album cover and Starbucks created ‘Magic Cups’ app for its coffee cups for Christmas and Valentine’s. The London Eye has produced an experiential education AR app incorporating Aurasma, providing tourists with a virtual and real-world experience.

LCG presentsINTERACTIVE TECHNOLOGY

AUGMENTED REALITY

Live Broadcast ARBrands such as National Geographic, Lynx and Green Giant have incorporated AR as part of their public experiential digital out of home campaigns within shopping centres and train stations.

LCG presentsINTERACTIVE TECHNOLOGY

AUGMENTED REALITY

AR GamesAs smart phone ownership increases, pervasive entertainment in the form of AR games are becoming ever more popular. One such example is Vodaphone’s Buffer Monster game. The Sony Playstation Vita has also developed a number of AR titles for this device.

LCG presentsINTERACTIVE TECHNOLOGY

AUGMENTED REALITY

4D User-Generated Theme Park'Live Park' enables visitors to experience a virtual world (4D avatar technology) previously only seen in movies like 'Minority Report' and 'Avatar'. Mixed reality architecture allows the visitor to create an avatar and become the main character in a unique adventure that incorporates multi-touch screens, holographics, augmented reality and gesture control.

LCG presentsINTERACTIVE TECHNOLOGY

AUGMENTED REALITY

AR GlassesBy the end of 2012, Google will launch a pair of Android powered, augmented reality, heads up display glasses. Its features will primarily be locative content served up via Google Maps and Search.IDEO designer Michaël Harboun has already experimented with AR glasses. As an alternative to glasses that tell you where the nearest bar is, his ‘Transcendenz’ project allows the wearer to experience the metaphysical environment asking intelligent questions rather than receiving the answers.

LCG presentsINTERACTIVE TECHNOLOGY

AUGMENTED REALITY

Microsoft Research - HoloreflectorMicrosoft Research’s Holoreflector is a project aimed at bringing holograms to consumers. The current prototype uses Kinect and an interactive screen to produce 3D hologram interaction.

LCG presentsINTERACTIVE TECHNOLOGY

AUGMENTED REALITY

Virtual Device ProjectionA collaborative team from the University of Calgary, University of Munich, and Columbia, have figured out a way to use a smart phone to project the phone’s display on to external displays nearby. In the future you could be running a virtual projector on your device and you could borrow the display space for a while with your device.

LCG presentsINTERACTIVE TECHNOLOGY

AUGMENTED REALITY

Apple Holographic ProjectionApple has patented holographic technology, as a result tech journalists have speculated that the iPhone 5 could incorporate holographic projection features.

LCG presentsINTERACTIVE TECHNOLOGY

GESTURES

LCG presents

INTERACTIVE TECHNOLOGY

GESTURES

Motion Control Interactive DisplaysWith regard to digital out of home, the majority of gesture based interactions is focussed on motion control.It can be used for browsing through product videos outside a shop for example Cartier’s Timepiece Explorer.Raiffensenbank in Kosova have experimented with gesture controlled interactive windows where navigation enables users to browse products and interact with content at any time of the day.

LCG presentsINTERACTIVE TECHNOLOGY

GESTURES

Motion Control Interactive InstallationsDesign I/O created an interactive installation that allows children to use their arms to puppeteer larger than life creatures projected on the wall in front of them.

LCG presentsINTERACTIVE TECHNOLOGY

GESTURES

Microsoft Kinect HackingGesture control allows for touch free interactivity. Since the introduction of Microsoft Kinect in 2010, the technology has not just transformed gaming but has inspired creativity amongst many industries. As a result Kinect hacking has become very popular such as Kinect controlled skateboards, shopping trolleys and displays. Microsoft’s response to hacking was to open up their device with a dedicated SDK for Windows 7 and Kinect for PC released in March 2012.

LCG presentsINTERACTIVE TECHNOLOGY

GESTURES

Microsoft Kinect LaptopsPrototype Kinect enables laptops from ASUS which are powered by the Windows 8 operating system, boast Kinect sensors that are located at the top of the display, while a set of LEDs will run along the bottom.

LCG presentsINTERACTIVE TECHNOLOGY

GESTURES

Motion Control Tablet ApplicationBitGym have incorporated head tracking to produce a game for your tablet that can be used in the gym.

LCG presentsINTERACTIVE TECHNOLOGY

GESTURES

Motion Control Desktop GamesOmnimotec have developed browser based games for webcams that do not require Kinect or motion sensing tech. For the film Alvin & The Chipmunks 2, Omnimotec produced ‘Munkdance’ a motion controlled dance game.

LCG presentsINTERACTIVE TECHNOLOGY

GESTURES

Motion Control In-CarCar manufacturer Mercedes unveiled its Dynamic & Intuitive Control Experience (DICE) at CES 2012 featuring gesture-based controls that allows the driver to scroll, select and control a variety of functions with basic hand motions.

LCG presentsINTERACTIVE TECHNOLOGY

GESTURES

3D Motion Control Desktop DisplayMicrosoft Research have developed a 3D desktop display, controlled entirely by gestures.

LCG presentsINTERACTIVE TECHNOLOGY

FACE RECOGNITION

LCG presents

INTERACTIVE TECHNOLOGY

FACE DETECTION

Face Tracking and SubstitutionOne of the biggest areas for development with gestures is facial tracking and recognition. Developer Arturo Castro has experimented with face hack. His Faces demo allows for real-time face substitution via projection, overlaying the features of your desired clone over your own countenance. The freakiest part is that it’s even capable of moving along with your facial expressions

LCG presentsINTERACTIVE TECHNOLOGY

FACE DETECTION

Face Recognition AdvertisingImmersive labs have developed adaptive advertising technology to serve up ads dynamically tailored to the target audience using facial recognition. Plan UK used this technology to serve up ads on their billboards for women only. With television manufacturers integrating face recognition technology into their devices we will soon experience targeted TV ads.

LCG presentsINTERACTIVE TECHNOLOGY

FACE DETECTION

Face Recognition Vending MachinesUnilever Ice Cream vending machine used facial recognition technology; to determine age, gender, and emotion. The machine uses an interactive "smile-o-meter" to rate smiles; those with a big enough smile were rewarded with free ice cream.The Happiness Machine from CocaCola used facial recognition to dispense coke based on two couples kissing.

LCG presentsINTERACTIVE TECHNOLOGY

FACE DETECTION

Face Recognition Smart Phone AppsThe RecognizeMe App for iOS5 uses facial recognition for phone authentication. The creative possibility to extend smart phone apps to incorporate face recognition is an area for further exploration.

LCG presentsINTERACTIVE TECHNOLOGY

EYE TRACKING & CONTROL

LCG presents

INTERACTIVE TECHNOLOGY

EYE TRACKING & CONTROL

Eye Controlled GamesEye tracking & eye control technology make it possible for interfaces to know exactly where a user is looking.This technology has revolutionized communications for those with special needs. But more recently the technology has been used for interactive games. Innovation company Tobii produced EyeAsteroids in which players can use their eyes as a controller.

LCG presentsINTERACTIVE TECHNOLOGY

EYE TRACKING & CONTROL

Eye Control For Mobile DevicesSenseye has developed technology that tracks the movement of your eyes using the front-facing camera on mobile phones. The first version requires"EyeDock", an add-on with an inexpensive webcam and an infrared LED. The company is currently working with handset manufacturers to bring this technology to mass market in 2013.

LCG presentsINTERACTIVE TECHNOLOGY

EYE TRACKING & CONTROL

Eye Tracking Smart Phone AppsBreathalEyes iPhone app is an example of tech developed to track the movements in your eyes to determine the state of intoxication.

LCG presentsINTERACTIVE TECHNOLOGY

EYE TRACKING & CONTROL

Eye Control Interface for Windows 8Interactivity via gaze, adds an entirely new dimension to interactive media. Tobii has developed eye tracking software for Windows 8 including a photo gallery that allowed a user to navigate through pictures by shifting glance and then enlarging a chosen photo by focusing on it.

LCG presentsINTERACTIVE TECHNOLOGY

EYE TRACKING & CONTROL

Apple To Bring 3D Gaming And Eye Tracking to iPad & iPhone?Apple in 2012 filed for an eye-tracking patent for gaming, photography and video as well as a feature for its iOS. Currently Nascent.

LCG presentsINTERACTIVE TECHNOLOGY

SOUND

LCG presents

INTERACTIVE TECHNOLOGY

SOUND

Voice Control For GamesThe Kinect device features voice commands alongside motion sensing. As a result we are now experiencing voice control within games such as Mass Effect 3 allowing the player to control the squad via voice. The Nintendo Wii U console due for release in late 2012 is rumoured to feature voice control.

LCG presentsINTERACTIVE TECHNOLOGY

SOUND

Voice Control For Mobile DevicesVoice recognition and voice commands have been available as applications for Android phones for some time.When Apple announced their voice recognition app ‘Siri’, a number of hacks appeared. As a result we are now seeing apps integrating voice control for example the Hello Kitty voice control cookbook to control page turns, and location-aware app Waze which provides hands free traffic information.

LCG presentsINTERACTIVE TECHNOLOGY

SOUND

Voice Control For CarsMercedes will feature Siri for hands free voice commands within their A-Class cars, whilst the Ford SYNC in-car communications system uses Windows technology that incorporates instant voice recognition and voice activation. Among the many Siri hacks to emerge is one that will start your car via voice command.

LCG presentsINTERACTIVE TECHNOLOGY

SOUND

Voice Control For Smart TV’sSamsung are integrating voice commands within its smart TV’s as demonstrated at CES in 2012. The all new Apple Smart TV is rumoured to feature Siri voice control.

LCG presentsINTERACTIVE TECHNOLOGY

SOUND

Audio Content RecognitionAudio Content Recognition can be used to interact with television programmes, in-game content and advertisements. Automotive company Honda was one of the first brands to release a mobile app for their ‘This Unpredictable Life’ TV campaign. The app allowed viewers to interact with the TV spot on their handset, catching the animated characters as they appear on screen.Fox International produced a free companion app for Series 2 of The Walking Dead incorporating, Civolution audio recognition technology serving up exclusive content available for both live and time-shifted programming.20th Century Fox partnered with Shazam technology for TV allowing users to access exclusive content from their Alvin and the Chipmunks movie ‘Chipwrecked’, simply by tagging the film or TV spot using the application.

LCG presentsINTERACTIVE TECHNOLOGY

THOUGHT

LCG presents

INTERACTIVE TECHNOLOGY

THOUGHT

Mind Control Devices For GamesControlling both physical and virtual objects using the power of your mind is now a reality. Electronics company Emotiv manufacture a brain-computer interface system, allowing people to interact with computer applications including games via mind control. It works by picking up brain signals via electrodes mounted on the head.

LCG presentsINTERACTIVE TECHNOLOGY

SMELL

LCG presents

INTERACTIVE TECHNOLOGY

SMELL

Smell-vertisingIn the UK McCain has cooked up a novel marketing technique with a campaign that produced the smell of hot jacket potatoes wafting through bus shelters in cities across the UK. A hidden heating element warms the 3D poster of a jacket potato at the press of a button, slowly releasing the aroma of a hot baked potato.The smell was developed over three months in collaboration with a specialist scent lab to match that of the real thing.

LCG presentsINTERACTIVE TECHNOLOGY

SMELL

Smell-o-VisionSmelling your movies via a clever concept called SMELLIT, from Portuguese designer Nuno Teixeira, brings contextual smells to the viewers schnoz via 118 aroma cartridges, each triggered by reading "smell information" embedded in a DVD.Samsung R&D in collaboration with the University of California in San Diego are currently experimenting with a small device that could be easily fitted to the back of your television that would serve to deliver the essence of odour appropriate to the picture being displayed on the screen. The smell is produced via aqueous solution of ammonia. This solution is held in a compartment made of non-toxic, non-flammable silicone elastomer, which releases odours through a small hole when heat and odour pressure build up.This technology is also being tried to be implemented in smart phones!

LCG presentsINTERACTIVE TECHNOLOGY

THANKS