2

Benefits through linking of analogue and digital maps

Chr istian PAUSCHERT, Emanuel RIPLINGER, Carola TIEDE, Volker COORS

ABSTRACT Despite their popularity electronic maps on mobile devices have not eliminated

the use of paper maps. Conventional paper maps can offer a quick overview due to their large format while the map content on mobile devices is adjusted to the li-mited display size. However the electronic map applications provide many addi-tional functions that paper cannot offer. The authors provide arguments for the linking of the two worlds through explaining how the advantages of both media complement each other. Hence, an approach to bridge the technological gap be-tween analogue paper and the electronic domain is presented. A technology devel-oped by Swedish company Anoto is used to enhance regular paper with an unob-trusive dot pattern enabling an electronic pen with a camera to determine its position on the paper. The potential benefits of this technology are discussed on the basis of two application scenarios that can demonstrate the feasibility of the linking of digital and analogue media and its benefits for the map user.

1. BACKGROUND AND OBJECTIVES Map applications on mobile electronic devices such as smartphones and PDAs

have been strongly propagated during recent years. The advantages that these de-vices offer are obvious. Many popular functions like voice guided navigation and automatic routing have only been possible since the advent of mobile electronic devices in combination with GNSS. Despite their popularity electronic maps have not completely eliminated the use of paper maps. Conventional paper maps can offer a quick overview due to their large format while the map content on mobile devices is adjusted to the limited display size.

Sellen and Harper (2001) investigated the question whether the paperless office

is a myth. To examine the value of paper for the user they made use of the concept of “affordances” that goes back to the ecological psychologist J. J. Gibson (Gibson 1979). Despite the fact, that paper maps are already being replaced by digital solu-tions in some domains, for example in naval navigation (DeVogel et al. 2001), one might still ask whether paperless navigation is a myth. In this particular case some incidents show that, despite the obvious benefits of the digital system, paper might

3

still be needed as a fall back solution (Backwell 2010). Reilly et al. (2006) inter-viewed students with respect to their preference for maps in paper form or on elec-tronic devices. Test persons with a preference for electronic maps mentioned the ability to cover a larger total area, to have control over the zoom level of the map and the existence of a search function as reasons for their choice. However, elec-tronic devices like smartphones have not been particularly designed for map ap-plications. Hence, the maps had to adjust to the conditions of the device that has been designed to be preferably compact. The compact dimensions of the display stand in contrast to the large formats of maps in their traditional form on paper. Ishikawa et al. (2008) examined people’s wayfinding behavior and spatial know-ledge acquisition while using GPS-based mobile navigation systems, paper maps or direct experience of routes. Users of paper maps performed significantly better in wayfinding tasks and in sketching a map of the previously unknown test area. They discuss that the small screen map may be a factor that negatively effects the user’s orientation in space compared to users with a larger map view on paper. Gartner and Hiller (2009) come to a similar conclusion. Dillemuth (2009) tested the use of maps in an empirical study. Contestants were given access to a map of a fictional area via a display while the map view had different sizes and only showed a part of the whole map. The rest of the map was accessible through pan-ning. After studying the map contestants had to answer questions concerning their spatial knowledge about the fictional area. Those with a limited map view per-formed especially worse in answering questions about relative distance and rela-tive direction compared to those who could view the whole map at once. Those test persons in Reilly et al. (2006) that were in favor of the paper map described the map view as easier accessible and easier to adjust for instance through rotating or folding. Besides these technical aspects emotional factors should be taken into account as well. Levy (2001) describes the affinity of paper documents that exists due to their tangible nature or due to the ability to simply leave notes or annotation on the medium.

Given these studies, one can keep hold of several affordances of paper maps

and compare them to the affordances of electronic mobile devices (table 1). One should consider if the affordances of paper maps and digital navigation systems together can help to better serve the goals of the user by creating combinations of the best of both the paper and digital worlds (Sellen and Harper 2001).

4

Table 1.1 Affordances of paper maps and electronic devices

In recent years there have been several research activities involved in the com-

bination with paper and digital media. An integration of functionality of electronic maps and paper maps is described by Paelke and Sester (2007). Digital interactive paper has been a subject in the Global Information Systems Group (Globis) at ETH Zürich. Thereby an event guide for a festival in Edinburgh that included a map has been developed based on Anoto technology. Besides that a digitally en-hanced version of a commercial tourist guide has been tested (Norrie 2009). At Stanford University an interaction with a map and a digital pen has been devel-oped, that enables users to access geotagged photos by tipping on the map with the pen. Equally a map to support biologists in their field work has been developed (Yeh et al. 2006). Further approaches exist that enhance maps by means of RFID tags (Reilly et al. 2006). The largest continuous Anoto pattern is exhibited in the Ars Electronica Center in Linz, Austria where a 6 x 3 meters city map serves as an interactive city- and geographic information system, called SimLinz, that enables the user to access diverse media and information like real-time data and allocate them geographically in the city (AEC 2009). A commercial application with the Anoto pattern in combination with maps is offered by the company Adapx. Their solution enables printing of maps with Anoto pattern out of ArcGIS. User records that are created with the digital pen on that map can be automatically synchro-nized with the geo data set in the GIS. The printed legend serves as a toolbar for instance to create new objects (Adapx). Compared to this the maps in the NavAD project shall allow information access as well as input of new information through the analogue medium.

5

Where experience with test users of digitally enhanced paper maps are pub-lished, they tend to be positive with respect to the underlying basic principle (Nor-rie and Signer 2005, Reilly et al. 2006, Yeh et al. 2006). In a test case of Norrie and Signer (2005) contestants were reluctant to positioning the pen on the map as they did not want to write on it. This reaction is to be taken into account when employing the pen more as a pointing tool rather than a writing tool.

2. APPROACH & METHODS Basis of the digitally enhanced paper map is the technology developed by the

Swedish company Anoto. A patented unobtrusive dot pattern (Anoto pattern) is printed on common paper. The pattern’s carbonaceous black ink reflects infra red light and this way can be read by a special electronic pen with an embedded cam-era. The Anoto pattern and the electronic pen are shown in Fig 2.1. The pen only uses a 1.8 x 1.8 mm excerpt of the pattern to determine its position on the paper with a resolution of 0.03 mm. Recorded images are analyzed at 100 frames/s while every image contains at least 6 x 6 dots of the pattern that are oriented according to an imaginary grid where the dots are slightly shifted from the intersections of the grid lines. The whole Anoto pattern can cover an area of 4.6 million km² which is about 73 x 1012 A4 pages (Kauranen 2004).

Fig. 2.1 The Anoto pattern with digital pen (Anoto)

Applications of this technology include mainly digital forms. There are systems that can recognize handwriting and allocate the writing in the digital system ac-cording to the writing position on the paper. Likewise the marking of checkboxes

6

can be recorded. These forms are used today in the medical domain, in facility management or logistics for instance (Anoto).

Certain challenges arise when printing the Anoto pattern together with a co-lored map. The black ink is reserved for the dot pattern and cannot be used in the map graphics as long as one does not use special coal free ink. Anoto recommends a limited number of certified printers for which the functionality of the pattern is guaranteed. The maps for the NavAD project however are created with means of common uncertified color laser printers without special ink. Possible interference of the map graphic with the dot pattern is minimized by adopting the map layout to the requirements of the pen as far as possible. This means that dark colors in the map are omitted or replaced by other lighter colors, which poses a challenge for the cartographic design.

The main task for the interaction with the pen and the digital map is to transfer the relative paper coordinates into absolute geographic coordinates. There are bas-ically two ways to achieve this: One can use the so called copied pattern, meaning that for all prints the same Anoto pattern space is used and the pen cannot distin-guish between different sheets or pages. This way one would have to store the geographic extends of the map field in a two-dimensional code that comes with the map like for instance a barcode, QR-Code or Data Matrix. Reading and inter-preting of the code can be implemented by using the integrated camera of mobile devices and existing software. There are also digital pens that can read barcodes. The other option would be using unique pattern by printing a different Anoto pat-tern on every map. This might seem cumbersome but due to the total size of the Anoto pattern it is not likely that the pattern will ever be used up completely. Which solution is to be used depends on the scenario or business model to be im-plemented since higher license fees might go along with the unique pattern model.

Communication between pen and mobile device is carried out via Bluetooth. The transfer of data is initiated by the user by pointing the pen at a special button printed on the paper – a so called pidget. This brings about certain limitations for the interaction with the system. The device is not giving any feedback while the user is acting with the pen on the paper. To solve this one could access the stream-ing technology by Anoto where a permanent communication channel between pen and device exists enabling real-time interaction, however not every digital pen and especially not every mobile device is capable of this. In the simple case when marking the pidget the relative coordinates are send to a server where they are de-coded and further processed before sent back to the device where a listening ap-plication will initiate the feedback on the device. In the application within the NavAD project there are three servers involved in this process. One acts as an ap-plication service handler to interpret the Anoto specific file format .PGC. This task is performed by the preexisting ActiNote® platform developed by project partner Actimage. The decoded pen data is then sent in form of an XML document to an application server that notices which request the user demands and processes the coordinates according to their application. A route request for instance might re-quire a trajectory optimization since the pen records a lot more points than re-quired for navigation. The transformation from relative to geographic coordinates

7

will also take place here. The conditioned coordinates will then be sent to the lis-tening application on the device via a push web service. This BlackBerry Push Technology is especially helpful in this scenario as it can trigger an event on the mobile device from the server side without the device having to poll for it. When working with BlackBerry the third server, the BlackBerry Enterprise Server, be-comes necessary being a communication gateway between the device and the rest of the world. The communication loop is illustrated in detail in Fig. 2.2.

Fig. 2.2 processing of pen data in the NavAD system

To demonstrate possible applications for this technology two test scenarios shall be investigated. They are tourist maps and maps for mobile data caption. The reason for the tourism scenario lies in the assumption that paper is still the main medium used by tourists when visiting a foreign city (Norrie and Signer 2005). Further, electronic devices don’t do justice to the way tourists interact with maps: They often travel in groups with a high degree of interaction and cooperative planning within the group. This social interaction is an integral part of the positive experience. Collaboration around a smartphone or PDA with a small display that cannot be read from all angles is cumbersome compared to a paper map (Brown and Chalmers 2003, Norrie and Signer 2005). Some contestants in Reilly et al. (2006) state they prefer paper maps while travelling and use electronic maps rather for travel prearrangements.

The mobile data caption scenario shall investigate how the described technolo-gy can be integrated in existing work flows, e.g. the data capture in road mainten-ance. In this case mobile workforce detects and records road damages that need to be entered into a digital system to be managed. This capturing process is often still done with pen and paper (Ahtinen et al. 2007) requiring further efforts for post processing. This can mean the scanning of forms and sketches on paper resulting in a homogeneous digital file that makes it difficult to distinguish between old and new content. The introduction of Anoto technology can simplify this post

8

processing workflow by making the scanning obsolete and delivering new data in-puts as a separate layer of data to the system.

3. RESULTS For the tourism scenario an A2 city map of Munich, Germany has been de-

signed that will interact with a digital map. To avoid problems due to different un-derlying map projections in the paper and digital map, OpenStreetMap is chosen as source of map data both for the printed map as well as for the electronic device. Since the intention is to have a map that can be printed with Anoto pattern on reg-ular laser printers without special ink, the map design had to be adjusted. To find out which map contents interfere to what extend with the pattern a vector version of an existing city map has been taken for testing and has been adjusted in an iterative manual process. In a first test print critical areas on the map were identi-fied where the map information obscures the Anoto pattern making it unreadable for the pen. In order to minimize this effect some or all of the following measures needed to be performed for these map areas: lightening of the background color, replacing of black and grey in labels, signatures and contours with blue or pale purple or omitting it where possible, reduction of the font weight, decluttering of labels and signatures. For some sections this process had to be repeated after the readability of the pattern had been checked again since one did not want to omit too much information of the map in one step. The resulting map compared with its original can be seen in Fig. 3.1a and 3.1b. The lessons learned from this process give guidance when developing a special map design based on OpenStreetMap.

Fig. 3.1a Or iginal map design (Colin and Dittmann 2005)

9

Fig. 3.1b Map design from Fig. 3.1a adjusted to the needs of the pattern

Small symbols to interact with the map are printed on the paper (Fig. 3.2), ei-ther together with the map or on a separate document. They enable access to func-tions of the map application on the mobile device which is developed using the APIs from Nutiteq (Nutiteq) and CloudMade (CloudMade). For navigation the two functions “new route” and “where am I” are implemented. The first enabling the user to point to a place on the map that will then become the destination the mobile device navigates to, starting from the current location provided by the em-bedded GPS receiver. The “where am I” button on the map gives the user a grid location on the paper map that corresponds to the location given by the GPS of the device. Further a local search is implemented where the user can point to symbols of common interests such as food or hotels etc. and mark an area on the map. The system will then search relevant objects in the intended area. A laminated version of the map prototype turned out to be useful since the lamination does not interfere the reading of the pattern and marks and writings with ball pen can be wiped off easily.

10

Fig. 3.2 Pidgets for map interaction

In the mobile data capture scenario a form with attributes that describe and classify road damage is created. The form also contains the Anoto-Pattern. Prior to an inspection the user has to select the area in his GIS and print it as a map on the form. On the street the user can draw the outline of the damaged area on the map and tick all the attributes that apply. When synchronizing the pen with a PC, the drawn shape and the attributes are processed and stored in a database.

The form uses the Anoto Copied Pattern, meaning each form uses the same coordinates to determine the position of the pen. To draw on different maps those coordinates have to be translated in the selected map-area and therefore size and parameters have to be stored when the map is printed. A plugin to Intergraphs GeoMedia GIS is developed to store this information and to provide a unique ID and print it alongside the map. It is also used to read and process the gathered in-formation from the pen after the transmission. Besides storing the attributes, the drawn geometries are simplified by the Douglas-Peucker algorithm and outlines are closed if necessary. Now the captured object and its attributes can be viewed and edited in the GIS (Fig. 3.3).

11

Fig. 3.3 Link of digitally enhanced form with GIS

4. CONCLUSION AND FUTURE PLANS A detailed compilation of previous scientific work and user studies on paper

maps compared to digital maps has been presented. Regarding this work one can state that linking tangible information like paper maps with information and inte-ractions that are familiar in the digital world is a challenge that accommodates our desire for information and security – a feeling that is supported through the tangi-ble. These thoughts were incorporated in the development of prototypes that dem-onstrate a technical solution to link paper with digital media and that indicate that the two can be linked in a beneficial way. Clearly, the prototypes are in an early stage of development at this point and still need to be tested with potential end us-ers to further examine the possible improvements in information access and input that this system offers. This project further assumes a possible near future scenario presuming that electronic mobile devices with GPS and digital pens are available to a large user group. In such a scenario of ubiquitous computing one might ask whether the paper map in particular will take a new role. As the comparison of the affordances shows, paper maps till nowadays have been designed to give over-view, orientation and information, the latter as precise and complete as possible, the affordance of giving information could probably be better fulfilled by electron-

12

ic networked devices. Hence, one can think about whether the design of present city maps for instance is overloaded for such a scenario. The use of the Anoto pat-tern implies a reduction in content on the printed map. It has to be examined whether this even has a positive influence on the ability to get orientation from the map with some users. When learning more about how users interact with paper maps and digital devices one will come to more conclusions about how paper maps and digital applications need to be designed for the two to work ideally to-gether and complement each other.

ACKNOWLEDGEMENTS The authors would like to thank the German Federal Ministry of Education and

Research (BMBF) for the funding of the project NavAD under the funding code 17N 22 09.

REFERENCES Adapx, Capturx | Turning Ink into Action. Adapx, Inc. http://www.adapx.com. Accessed 27 Jan-

uary 2011 AEC, Das neue Ars Electronica Center.

www.aec.at/content/documents/AEC_Broschuere_2009.pdf. Accessed 24 September 2010 Ahtinen A, Nurminen JK, Häkkilä J (2007) Developing a Mobile Reporting System for Road

Maintenance – User Research Perspective. In: Proceedings of the 4th international conference on mobile technology, applications, and systems and the 1st international symposium on Computer human interaction in mobile technology. Singapore, September 2007, pp 1-7. doi:10.1145/1378063.1378065

Anoto, Anoto Group AB. http://www.anoto.com. Accessed 22 September 2010 Backwell G (2010) ECDIS – Nautical Charts Without Paper – A Risky Transition Period.

Suite101. http://boats.suite101.com/article.cfm/ecdis--nautical-charts-without-paper. Ac-cessed 26 August 2010

Brown B, Chalmers M (2003) Tourism and mobile technology. In: Kuutti K, Karsten EH (eds) Proceedings of the Eighth European Conference on Computer Supported Cooperative Work. Kluwer Academic Publishers, Norwell, MA, pp 335-354

CloudMade, CloudMade - The Leading Platform for Creating and Monetizing Unique Applica-tions With Location. http://cloudmade.com/. Accessed 18 January 2011

Colin H, Dittmann S (2005) … wegweisend … Fachhochschule München. Hochschule München (ed), C. Maurer, Geislingen, Germany, April 2005.

DeVogel GF, Baccei PK, Shaw PT (2001) The United States Navy Navigating in the 21st Cen-tury. Oceans 3:1460-1465. doi:10.1109/OCEANS.2001.968048

Dillemuth JA (2009) Navigation Tasks with Small-Display Maps: The Sum of the Parts Does Not Equal the Whole. Cartographica 44(3):187-200. doi:10.3138/carto.44.3.187

Gartner G, Hiller W (2009) Impact of Restricted Display Size on Spatial Knowledge Acquisition in the Context of Pedestrian Navigation. In: GARTNER G, REHRL K (eds) Location Based Services and TeleCartography II: From Sensor Fusion to Context Models, Lecture Notes in Geoinfromation and Cartography, 2009, Section II. Springer, Berlin, pp 155-166. doi:10.1007/978-3-540-87393-8_10

Gibson JJ (1979) The Ecological Approach to Visual Perception. Houghton Mifflin, New York Ishikawa T, Hiromichi F, Osamu I, Atsuyuki O (2008) Wayfinding with a GPS-based mobile

navigation system: A comparison with maps and direct experience. Journal of Environmental Psychology 28(1):74-82. doi:10.1016/j.jenvp.2007.09.002

13

Kauranen P (2004) The ANOTO pen – Why light scattering matters. International Workshop on Microfactories. Shanghai, October 2004

Levy DM (2001) Scrolling Forward: Making Sense of Documents in the Digital Age. Arcade Publishing, 2001

Norrie MC (2009) The Link between Paper and Information Systems. In: Filipe J, Cordeiro J (eds) Enterprise Information Systems, Lecture Notes in Business Information Processing 19(1):3-14. doi:10.1007/978-3-642-00670-8_1

Norrie MC, Signer B (2005) Overlaying Paper Maps with Digital Information Services for Tour-ists. In: Frew AJ (ed) Information and Communication Technologies in Tourism 2005 1:23-33. doi:10.1007/3-211-27283-6_3

Nutiteq, Mobile Mapping Solutions | Nutiteq. http://www.nutiteq.com/. Accessed 18 January 2011

Paelke V, Sester M (2007) Design Exploration of Augmented Paper Maps. In: Proceedings of ISPRS Workshop Visualization and Exploration of Geospatial Data. Stuttgart, June 2007

Reilly D, Rodgers M, Argue R, Nunes M, Inkpen K (2006) Marked-up Maps: Combining Paper Maps and Electronic Information Resources. Personal Ubiquitous Computing 10(4):215-226. doi:10.1007/s00779-005-0043-6

Sellen AJ, Harper RHR (2001) The Myth of the Paperless Office. MIT Press, Cambridge, MA. Yeh RB, Brandt J, Klemmer SR, Boli J, Su E, Paepcke A (2006), Interactive Gigapixel Prints:

Large Paper Interfaces for Visual Context, Mobility, and Collaboration. Stanford University Computer Science Department Technical Report, October 2006.