Location Based Services Without the Cocoa

Institutional Web Managers Workshop , 12th July 2010

Ben Butchart, Murray King

This work is licensed under a

Creative Commons Attribution-NonCommercial 2.0 England & Wales License. But note caveat.

#lbs_nococoa

Introduction Digimap Geo Mobile Scoping study

Technical evaluation User Engagement Digimap pilot (in AppStores soon! Maybe?)

Focus today for workshop on technical evaluation Our blog: http://mobilegeo.wordpress.com/ Prezi mobile scoping study

http://prezi.com/jllhxm1zzpdm Final project report:http://mobilegeo.files.wordpress.com/2010/07/digimap-

mobile-scoping-study-final-project-report1-1.doc

Thanks to JISC for funding.

#lbs_nococoa

Workshop outline

Requirements for LBS in teaching and learning. Technology concepts overview. Native, Mobile Web, Hybrid,

HTML5 Geolocation, HTML5 Canvas, HTML5 Local Storage . Building your own mobile mapping client tutorial. Build your own mobile mapping client - practical exercise . HTML5 Canvas and LBS. Building your own AR app with Layar tutorial (10 mins). Building your own AR app with Layar practical exercise ( 10

mins ). Web Approaches pros and cons. Why Native? Sustainability

#lbs_nococoa

“…delivering a map to a smart phone device within a range of educational contexts including field trips in remote areas, where network connectivity may be limited. It was also anticipated that the application might assist data collection, for example, taking pictures of rocks during a field study.”

Informal Requirements

#lbs_nococoa

Technical Requirements

1. Location: Must be able to obtain a location fix through device sensors (GPS).

2. Touch: Must be able to take advantage of touch screen user gestures (e.g pinch to zoom in and out)

3. Sensors: Should be able to access sensors and gadgets such as the camera, accelerometer and compass.

4. Local Storage: Should be able to cache data so that application can be used in remote areas with limited connectivity.

5. Portable: Should work on a range of devices.

#lbs_nococoa

Mobile App DevelopmentApproaches

Native: programming languages and tools are unique to a particular device or operating system.

Mobile Web: applications are delivered

through the mobile web browser.

Hybrid: a lightweight web browser is integrated into a skeleton native app.

#lbs_nococoa

Technologies : Mobile Web

HTML5 Geolocation: web based access to location sensors.

HTML5 Canvas: manipulate images at pixel level HTML5 Local Storage: SQLite database embedded in

browser. HTML5 Cache: Cache web resources such as images,

web pages and scripts. CSS Flash ( not supported by iPhone) AR browsers – Layar / wikitude etc/

#lbs_nococoa

Technologies : Native

iPhone: Objective-C and Cocoa Touch. Blackberry: RIM Symbian (Nokia): Symbian Foundation C

libraries. Android: Java (mostly) AR – Wikitude

#lbs_nococoa

Technologies : Hybrid

PhoneGap W3C Widgets QuickConnect Rhodes Appcelerator Titanium

#lbs_nococoa

Memory Check…

HTML5 Geolocation: web based access to location sensors.

HTML5 Canvas: manipulate images at pixel level HTML5 Local Storage: SQLite database embedded in

browser. HTML5 Cache: Cache web resources such as images,

web pages and scripts. Native: programming languages and tools are unique to

a particular device or operating system. Mobile Web: applications are delivered through the

mobile web browser. Hybrid: a lightweight web browser is integrated into a

skeleton native app.

#lbs_nococoa

Requirement Native Mobile Web Hybrid

Location sensors

Touch gestures

Sensors and gadgets

Local storage

Portable

Requirements / Approach

#lbs_nococoa

Requirement Native Mobile Web Hybrid

Location sensors yes Yes (via HTML5 geo location API)

Yes (via HTML5 geo location API)

Touch gestures yes Yes (partial) Yes (partial)

Sensors and gadgets yes No Yes (usually via framework API)

Local storage yes Yes (via HTML5 Cache and Storage API)

Yes (via HTML5 Cache and Storage API)

Portable no Yes Yes (partial)

Requirements / Approach

#lbs_nococoa

Build Your Own mobile Map App in 10 minutes

Yes, really. 10 minutes. We need some maps – Google? No. Everyone does that. We’ll use some OS

Open Data instead. We’ll use the OpenLayers API And Geolocation API

#lbs_nococoa

http://osopen.edina.ac.uk

Digimap Open Stream ( New Edina Service)

Ordnance Survery OpenData

Edina provideWMS Service

You can register for your API keyhere (free and unrestricted use for ac.uk)

#lbs_nococoa

Digimap Open Stream WMS

Ordnance Survey OpenData maps

Miniscale

1:250000 Colour Raster Vector Map District(raster)

OS StreetView

#lbs_nococoa

Digimap Open Stream WMS

#lbs_nococoa

Digimap Open Stream WMS example

http://osopen.edina.ac.uk/openstream/wms?REQUEST=GetMap&SERVICE=WMS&VERSION=1.1.1&BBOX=325000,674000,326000,675000 &WIDTH=460&HEIGHT=640&SRS=EPSG:27700&FORMAT=image/png &LAYERS=osfree&CACHE=false&TOKEN=YOUR_KEY

#lbs_nococoa

http://osopen.edina.ac.uk/openstream/wms?REQUEST=GetMap&SERVICE=WMS&VERSION=1.1.1&BBOX=325000,674000,326000,675000 &WIDTH=460&HEIGHT=640&SRS=EPSG:27700&FORMAT=image/png &LAYERS=osfree&CACHE=false&TOKEN=YOUR_KEY

Digimap Open Stream WMS example

#lbs_nococoa

Digimap Open Stream OpenLayers Example

http://osopen.edina.ac.uk/help/map

#lbs_nococoa

Digimap Open Stream OpenLayers Example Code

Example OpenLayers code here: http://osopen.edina.ac.uk/help/map

#lbs_nococoa

Digimap Open Stream OpenLayers Example Code

var os_options = { token: apikey,

format: "image/png", layers: "osfree",

cache: true};

map = new OpenLayers.Map('map', {controls: [], projection: new OpenLayers.Projection("EPSG:27700"), units: "m", maxExtent: bounds, resolutions: [1763.889,352.778,176.389,88.194….0.882,0.441]} );

var osfree = new OpenLayers.Layer.WMS( "Edina OS OpenData WMS","http://osopen.edina.ac.uk/openstream/wms", os_options );

map.addLayers([osfree]);

#lbs_nococoa

OpenLayers Example: convert latlong

<script type="text/javascript" src='coordomatic1.js'></script>

var natgrid = new Edina.EPSG_27700() ;var latlong = natgrid.toLocalSystem(55.93682192813903 , -3.180231279418976

var lonLat = new OpenLayers.LonLat(latlong[1], latlong[0]) ; map.zoomTo(10) ; map.setCenter(lonLat);

#lbs_nococoa

What’s different in mobile OpenLayers?

<script type="text/javascript" src="touch3.js" ></script>this.touchhandler = new TouchHandler(map, 2);

Touch Controls

Use of Geolocation APIif(typeof(navigator.geolocation) != "undefined"){ navigator.geolocation.getCurrentPosition(foundLocation, handleError);

}Some stylesheet stuff<meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black" /> <meta name="viewport" content="minimum-scale=1.0, width=device-width, ..>

#lbs_nococoa

Geolocation API

if(typeof(navigator.geolocation) != "undefined"){

navigator.geolocation.getCurrentPosition(foundLocation, handleError);

}

function foundLocation(position) {

lon = position.coords.longitude;lat = position.coords.latitude; var lonLat = new OpenLayers.LonLat(latlong[1], latlong[0]) ; map.setCenter(lonLat);

}

function handleError(positionError) {$("#debug").append("<p>" + "Attempt to get location failed: " +

positionError.message + " </p>") ;

};

#lbs_nococoa

HTML5 Canvas

Manipulate images at pixel level Can draw on images and retrieve pixel data

http://mobilegeo.wordpress.com/2009/11/04/html5-canvas-for-mobile-apps/

#lbs_nococoa

HTML5 Canvas

var img = new Image(255,255) ;img.src = "http://.../testimg/test1.png"  ;var newCanvas = document.getElementById("canvasimg") ;var ctx = newCanvas.getContext("2d") ;var newimg = ctx.drawImage(img, 0, 0);

<canvas id="canvasimg" width="250" height="250">This demo uses the HTML canvas object. </canvas>

1. Create canvas element

http://mobilegeo.wordpress.com/2009/11/04/html5-canvas-for-mobile-apps/Short url: http://delivr.com/11cic

2. Draw image on canvas

#lbs_nococoa

HTML5 Canvas

Blogpost : http://mobilegeo.wordpress.com/2009/11/04/html5-canvas-for-mobile-apps/Short url: http://delivr.com/11cic

if ( drawMode == true ){var xy = getxy(e, newCanvas ) ;ctx.fillRect(xy.x,xy.y,5,5) ;return ;}

3. Draw graphic on canvas image

http://mab.edina.ac.uk/testimg/canvasiphone.html

Short: http://delivr.com/11cj0

#lbs_nococoa

HTML5 Canvas

Retrieve pixel data

“the real potential for HTML5 Canvas is to get inside the image itself enabling us to extract information from it and use that information to create our own graphics on the map “

Anna Chapman:practitioner of steganography

#lbs_nococoa

HTML5 Canvas: Data Extraction

Data extraction getImageData

Demo feature selection using HTML5 canvashttp://delivr.com/11cis

var xy = getxy(e, newCanvas ) ;var imgd = c.getImageData(xy.x, xy.y, 1 ,1 ) ;var pix1 = imgd.data ; var i = 0 ; var red = pix1[i]; // red var green = pix1[i+1]; // green var blue = pix1[i+2]; // blue

!!! Browser Security restricts getImageData() to same domain as code !!!

#lbs_nococoa

HTML5 Elevation Demo 1

Blog post: http://mobilegeo.wordpress.com/2010/03/01/integrating-openlayers-and-html5-canvas/http://delivr.com/11cir

#lbs_nococoa

HTML5 Elevation Demo 2

Elevation demo: drawing a line on a map using HTML5 Canvas. At each point on the line, the height is extracted from the base terrain layer (previous slide) and plotted on the bar chart

#lbs_nococoa

HTML5 Canvas: Other data extraction use cases Feature selection: highlight features (parking lots) on

map. Reduced Map Legend: Only shows symbols in view. Speech Summary: detect features and summarize. Accessible Maps? Pro: Can all be done offline and works for raster

maps as well as vector maps Con: Mobile processors too slow for standard

techniques such as Sobel and Colour Histogram. Active research area for Edina.

#lbs_nococoa

HTML5 Local Storage

aka HTML5 Database Provides SQL-lite database in browser. Can

be used by applications to cache data.

http://unlock.edina.ac.uk/mobile.html

http://delivr.com/11clh

#lbs_nococoa

Augmented Reality

“enhancing the user’s perception and interaction with the real environment by superimposing the real world with virtual information that appear to coexist in the same space as the real world.” *

*F. Liarokapis, I. Greatbatch, D. Mountain, A. Gunesh, V. Brujic-Okretic, and J. Raper, "Mobile augmented reality techniques for geovisualisation,“ in Ninth International Conference on Information Visualisation (IV'05).    IEEE, 2005, pp. 745-751. [Online]. Available: http://dx.doi.org/10.1109/IV.2005.79

Priestnall, G. (2009) 'Landscape Visualization in Fieldwork', Journal of Geography in Higher Education,

#lbs_nococoa

AR – the virtual limb

[47] Kray C, Rohs M, Hook J, Kratz SGroup Coordination and Negotiation through Spatial Proximity Regions around Mobile Devices on Augmented Tabletops3rd IEEE Workshop on Tabletops and Interactive Surfaces (IEEE Tabletop 2008), Amsterdam, the Netherlands, October 1-3, 2008

Chris Kray, U. Newcastle, Augmented Tabletop: Visual markers displayed on screen used for performing collaborative tasks.

Could be used to share fieldtrip notes on map.

#lbs_nococoa

Ubiquitous AR

New generation of “AR browsers” for iPhone, Android etc. (Layar, Wikitude)

Building Anatomy App

Chris Lowry 3d model

Superimpose on building facade

View in Layar browser at 29m

#lbs_nococoa

Ubiquitous AR - issues

GPS accuracy – 3d image recognition

Simplifying 3d models Lack of standards ( but

innovation perhaps more important at the moment?)

Publishing platform?

#lbs_nococoa

Layar: architecture

#lbs_nococoa

Layar: set up developer account

#lbs_nococoa

Layar: create a Layar

http://publishing.layar.com/publishing/layers/

#lbs_nococoa

Layar publishing

#lbs_nococoa

Layar test: getPointOfInterest

http://publishing.layar.com/publishing/test/iwmw2010demo1

#lbs_nococoa

Layar testing: POI search failed

Probably a JSON format error

#lbs_nococoa

Check JSON with JSONLint

http://www.jsonlint.com/

#lbs_nococoa

Layar testing: Success!

#lbs_nococoa

Layar: Points of Interest JSON{ "hotspots": [ { "distance": 100, "attribution": "EDINA POI provider", "title": "Halifax Buildings, location = 53.371424, -1.505796", "lon": -1505796, "imageURL":

"http://dlib-rainbow.ucs.ed.ac.uk/layar/img/halifax.jpg", "line2": "University of Sheffield", "line3": "Endcliffe Village", "actions": [ {"uri":

"http://www.shef.ac.uk/accommodation/prospective/endcliffe", "label": "Open website" } ], "lat": 53371424, "type": 1, "id": "halifax" } ], "layer": "iwmw2010demo1", "errorString": "ok", "morePages": false, "errorCode": 0, "nextPageKey": null}

#lbs_nococoa

Points of Interest – go find ‘em!

#lbs_nococoa

Points of Interest – go find ‘em!

Endcliffe Vale FlatsLongitude -1.508054Latitude 53.371387

FroggattLongitude -1.508688Latitude 53.371907

The EdgeLongitude -1.507504Latitude 53.372711

Jonus CourtLongitude -1.506040Latitude 53.371751

Jonus court

#lbs_nococoa

Contrasting approaches to App Development

Speed AR Basic location [1] Speech control No 3rd party API Easy deployment / publishing Data cache Quality assurance Open Standards Full access to sensors [3] Touch control Developer happiness

1

2 34

5 6

7

Hybrid

Web App Native

#lbs_nococoa

Contrasting approaches to App Development

#lbs_nococoa

Why go Native? After all, Cocoa tastes good! Take advantage of latest developments in hardware /

OS Full access to sensors and gadgets Easier to integrate tasks such as viewing a map,

taking a photo, reding QR code in one app. Hybrid app developers vulnerable to “Tweak Creep”. Hybrid frameworks might get shut out by some

platforms (e.g. flash) No need to fight against web browser defaults ( text

selection, resizing images) If charging for app, native probably best option

#lbs_nococoa

Sustainability

Mobile apps technologies in constant flux. Frequent updates to OS, plaforms, hardware and consumption patterns.

Not enough to have one iPhone developer who works on occasional mobile projects.

Need constant development capacity. Consider mix of in-house ( mobile web) and

outsourced ( native ) capacity. Need to appreciate full cost of mobile development

including training, equipment and staff retention.

#lbs_nococoa

Thankyou!

Check out geo mobile bloghttp://mobilegeo.wordpress.com/

And mobile scoping reporthttp://mobilegeo.files.wordpress.com/2010/07/

digimap-mobile-scoping-study-final-project-report1-1.doc

Thanks to all those taking part in scoping study and JISC for funding.