MapStory Local How-To, DRAFT

7/29/2019 MapStory Local How-To, DRAFT

1/6

MAPSTORY LOCAL: How-ToThs ude walks you thouh the vaous ats of ceat o add to aMaStoy Local oject. You may also wat to take a look at the ovevew of

MaStoy Local, as well as the coesod vdeos. All of ths s a wok oess ad t wll e exaded o as MaStoy Local develos.

ObTAin DATA

Find and explore your local assessor's web mapping platform.

Do a web search or [Your city/county] assessor, such as Ames Iowa Assessor, and navigate to the assessors site.

Chances are, in the top ew results, you will see a link to an assessor in your city or the county youre in. Youll want to

nd the web mapping platorm, and explore the area youre in. It works just like any web mapping platorm, where

you can zoom in and out and pan and click on things. Conrm that they have a Year Built category or properties.This is the key inormation that you need.

Obtain the Data.

Look around on the site and see i there is a place you can download the data. In the best case scenario, it will be on

the site, where you can download it, and the data will have the year built columns in it. This is rare however, and

you will probably need to call the assessor or Geographic Inormation System (GIS) coordinator at the city.

Youll probably want to tell them the purpose o the project, and what youll be using the data or. Make sure that

you make it clear to them in the beginning that you are asking or the data, which will be posted online, openly.

At the least, you should ask or the shapeles or all the properties, all the buildings, all the roads and the railroads,

and the city limits. For the properties shapele, you should ask and make sure that the le they give you lists the

year built or each property. Also, ask i they have any historic data they may have some, you never know. In ourcase, they happened to have the city limits ater every annexation since the 1970s.

In some cases, they may give you a link to a place where you can download it or simply send you a le in an email,

but in many places, you may have to go through some hurdles. For one, they may be araid that the data is going

to be posted publicly due to privacy and integrity, and in some cases they may even claim that there should be a

charge or the data that is public.

But as long as it does not have anything condential according to your state law, and does not require any

processing that is signicant, beyond selecting and exporting the necessary and unrestricted data into a shapele

and emailing it to you, it is likely unlawul or them to restrict or charge or the data. It is worth the eort though be

persistent and patient and work with people, and be intelligent about who to talk to to in order to get the data. You

may have to work your way up the chain and in the worst case, talk to the ocials with the highest authority, suchas your City Councilman, or even ask ocials at your state who work with local ocials to obtain the data.

In the case o Ames, we rst asked the County GIS coordinator, who made us ll out a orm and allowed us to have

the data without there being a charge, because it was being used by a nonprot organization. But when we asked

the City or more data, they just sent it to us in an email quickly without any agreement, as they understood that

there was no problem, and had the wisdom to understand the benets o providing the data openly. For some

other non-GIS historical data that is located at the county, we are having to work with the State o Iowa ombudsman

oce to obtain some documents.

mastoy.o | @mastoy | #mastoy


2/6

Obtain Maps

This is essential to MapStory Local, but or the majority o the mapping and eort, it will come ater obtaining GIS

data. For all the places in the world where GIS data is already available, well over 90% o the eort that goes into

MapStory Local will be in less then 20% o what will be mapped everything that has existed beore what exists

now, all the buildings, properties, pathways, roads, railroads and eatures that changed or were built and removed.For this data, storytellers will have to take old maps and old inormation, and overlay them into a GIS platorm and

draw over it.

For this, you will need to obtain every image o a physical map that exists o the place you are mapping, scan them

i they are not already, and add them in a le. Eventually, this will be done in MapStory itsel, and there will be a

georeerencing or rubbersheeting tool in the site itsel. When that is the case, the site will be a place where people

put all maps, provided that they are not in copyright. I they are in copyright, that poses a problem, as you cannot

publish them openly online. You could perhaps use it on your own computer and upload the data to MapStory, and

the MapStory community will probably have to reerence the corresponding shapele instead o an image, though

that is not ideal.

To obtain maps, just do a google search online o [Your city/county] maps, such as Ames Iowa maps. You may

also want to search historical maps. It is very possible that there will be a place where people are already doing

this or your town in the case o Ames, one would come across the Ames Historical Society. In act, even i you

nd images o maps, a local historical society is the most valuable resource. They are ull o passionate, dedicated

people that collect records. Another good place will be the your municipal or county government they oten have

old maps as well. You should not only call the government oces, but make sure you talk to each department, as

each department does not always communicate or know what kind o inormation other departments have so

call the planning, public works, assessor, inspections, and various other departments to see what they have. In the

case o Ames, the City had maps that only some people knew about, and the Ames Historical Society did not even

know about it. I youre lucky, a lot o maps are already scanned, but it is likely that you will nd maps that are still

not scanned. For these, i they are larger than a scanner, they should be scanned with a large scanner. Scanning a

map in pieces does not actually work well, as the pieces do not align when you piece them together in an image

processing program. These may be hard to come by, but ask people at your historical society, library and city where

you can nd a scanner that scans maps.

In addition to complete maps, many cities or counties will have a recorder, which store all o the individual records

that they use to track property boundaries and various other things. There is a good chance that you will be able

to nd a drawing and description, with coordinates o every property that has been created and subdivided, going

back several decades. In the case o Ames, the recorder is at the county level, where there is a copy o every record

going back to the ounding o the county. It is less likely that these historical documents will be digitized, but that

is changing rapidly. In the case o Ames, there are an estimated 100,000 pages o property deed records, much

o which are not digitized, and the ones that are have been dicult to obtain, due to condentiality issues, the

ormats and medium the documents are stored, and the willingness o public ocials to do the work required to

obtain and deliver them. Once we have all the documents digitized, we can have a complete picture o how Ames

and the county was settled by those other than Native Americans rom the time the rst arms were established

and subdivided and sold o. Though it may be rare, in some places this may be easily obtained and can simply be

dumped online. Either way, as ar as human eort, that will likely be the easiest part most o the eort will be ingeoreerencing each property parcel and drawing over them, one by one.


3/6

STArT MApping

Organize the data

You may want to start mapping right away, but it is important that you establish good, simple organization.Fortunately, this is not that hard. Dont think that you can skip this and worry about it later things can quickly get

complicated, and i you do not store things right, it will become convoluted or even you, who created all o it in the

rst place. The way that storytellers would organize data would be similar, and this is one approach:

Create a older or each type: Parcels, Buildings, Roads, Railroads, Phonebooks, etc.

Add the les to them, and duplicate them. I the originals are not already in a zip ormat, zip them so that

they cannot be changed. Any time you reuse the original layer, duplicate it and unpack it.

Create a older or maps, and label the maps in a standardized ormat: [Year_Place], such as 1902_Ames_

Iowa_USA.

Take a look at the Data

Take a look at the shapeles you received in a desktop GIS application like ArcMap or QGIS, where you can see them

visualized, and make sure there are no problems opening the les. Take a look at the data table or the database le

(*.db) and see it theres a column that lists the year built dates, as well as a column that lists the dates when the

properties were sold. Also take a look at the buildings layer and see i theres a year built date there.

For each o the ollowing layers, it should take only a ew minutes to add see as a ully working mapstory, not

including the time it will take your internet connection to upload the data. Go to mapstory.org, and navigate to the

place where you upload shapeles. There will be instructions on the site to do this under How-To, which is simple

and straightorward. All you do is drag and drop the shapele and upload it, point to the column that lists the time

attributes, and then have it process.

Parcels

In your parcels layer, its possible that they may have multiple sale dates listed or each parcel i that is the case,

youll want to take the earliest date. Why? Thats the earliest date that the property existed in its current shape.

Beore that, it could have been part o a larger property that was subdivided. Go ahead and map this layer and

point to column that lists the earliest date the parcel was sold, again, using the instructions in the How-To section

at mapstory.org.

Buildings

I there is a year built column, then that will be helpul and will save you a step. But it is likely that the year built

dates will not be listed, because assessment is or a property, both the land and the buildings on it, and buildings are

simply extra shapes. So, i thats the case, youll have to do a join, where you combine two tables using a common

column as a reerence. In this case, you will need to join the building code to the corresponding parcel code in the

property layer. You can probably do that with the address or some eature in the table. But you may need to do a

spatial join. You can nd instructions online to do that in the program you are using.

Roads

It is very unlikely that there will be dates attached or the roads, and you will have to do the initial work in generating

a layer based on the proximity to other data, like buildings and parcels. With the help o some GIS developers at

Iowa State University, a tool was built that generates a roads layer using eatures. In the case o Ames, we used the

buildings shapele that was created using a join, as mentioned above, and or every street segment, the earliest


4/6

date among the adjacent parcels was added. So, or example, i there are 10 buildings along Main Street, between

two adjacent intersections, between 2nd and 3rd Streets, and the oldest building was built in 1905, then the date

will be added to that street segment rom the oldest building, and it will appear in 1905. It takes a computer about

an hour to generate it ully, and the result is a layer that roughly approximates how the roads were built in a city. The

rst roads are more sporadic, but becomes increasingly accurate as you move the timeline orward.

Its important that the right eatures are matched to the right road segments or example, i a building on the

corner o Main Street and 2nd Street is equidistant to both, then something like the address should be used to

generate the layer. But you should check the data to see how oten the address and location correspond. Sometimes

youll have buildings or parcels where the address is actually ar away. We encountered this with the ISU campus

and government buildings, which had a mailing address ar away. Furthermore, believe it or not, or those o you

who have not been outside an address paradigm like the US, they actually dont name streets in many countries,

and addresses are based on buildings and blocks. This is the case in countries like Japan and India, which are more

established societies rooted in a less mobile geography. This was made clear again with Iowa States campus, where

things are pedestrian-based. Standing on the campus, you would guess that an address would say a building

and oce number, and not a street number, because its people oriented and not based on vehicular mobility. In

these cases, roads are simply whats in between the blocks and buildings, while in most cases in the US and similar

environments, buildings and blocks are what are between the roads. In developments that were created or modern

vehicles, it is more likely that the destination was the result o the road, while in more pedestrian geographies, theroad was the result o the destination or created concurrently. Interesting, huh?

So, the roads tool was built to have two options i the eatures have addresses that mention the street number

that it accesses, then the rst option allows you want to mention those streets. The other takes a street and does it

purely based on proximity its closest proximity, not using the addresses. For the buildings or which the addresses

were not in proximity, we separated those buildings and generated roads separately using them. This is not ideal,

as it should all be done in one layer, and will be one o the rst improvements that will be made to the tool next.

Another option that may provide accuracy would be the access points to buildings. This is becoming common,

as this is oten or saety, like police or re, which might use GPS devices as aids. This would be especially useul in

places where they dont have street addresses or or areas where the addresses are not or the adjacent streets. But

or places with street addresses, it may more accurately refect the rst street, so it might be useul as a separate

column that is, unless in some places the address was changed to another adjacent street.

People and Businesses

Scanning phonebooks will take a large amount o manual labor, and it is good to estimate the amount o time it

would take. Phonebooks today are very dense, and can be estimated at 1 page per 1000 people, and 45 secs/page

pair, while earlier directories were smaller and less dense, and could be 5 times that ratio. In the case o Ames, the

number o pages only roughly doubled while the population increased rom a ew thousand around 1920 to 60,000

in 2012. All in all, we would estimate the time to be 50 hours o work, including 40 hours o scanning.

You may think that this is not worth the eort, but it is common or people to do things like this in historical

communities everywhere. People do all sorts o data entry and manual copying and preservation in historical

societies. I you approached them, they might be willing to gather a crew or do it slowly over several weeks, likea phonebook a day. We are experimenting with dierent techniques right now, using digital cameras and even

automated book scanners, to nd the easiest way o doing this reliably.

As this is built over the coming years, this will be useul by itsel or historical societies and records, but also or

understanding all sorts o social and economic trends, and tracking people with genealogical records. While most

o the inrastructure that has existed probably exists now, even i you were to digitize and scan and map every

directory, it would amount to a small raction o the 108 billion people who have ever lived. Anything urther than

directories would have to use genealogical records, which would also have to mention where the person lived.


5/6

The process to map everything in the phonebook is simple, and starts with using a good program or optical

character recognition (OCR) to convert the image to text. We have not demonstrated this with any open source OCR

programs, but it is likely that you will really want to use a proessional program, until ree programs are at the same

level. We have used ABBYY Finereader, which does an outstanding job in converting over 80% o a phonebook

accurately. I you do not wish to purchase it, you can send the images to someone in the MapStory Local community

who will OCR the images or you. You should not publish the phonebook pages online or anyone to download andonly send it to someone you are in direct contact with it is the inormation, not the images, that are not open to

distribute publicly, unless they are out o copyright. There are some eatures that you may benet rom i you have

your own copy o a proessional OCR sotware i you click on the text or example, it will take you to the part o

the image that was read, and you can see how to correct it. We have yet to gure out a good manual workfow or

correcting mistakes, but in the meantime, you can have the images available and an initial database le rom which

a shapele can be created.

Once a digital text is created, simply run the code that corresponds with the phonebook. So ar we have only added

a Polk City Directory rom 1921, and the code is available in a ree repository. The MapStory Local community will

publish and rene the code over time or each phonebook publication ormat. Eventually, a storyteller would scan a

phonebook and nd the corresponding publisher and date, veriy i the ormat corresponds using a representative

layout, and then apply the right code.

Once the code is added to a database or spreadsheet, the database is used to generate dots using an addresslocator. This is a nearly instantaneous process, using current addresses. O course street names and numbering

changes, and new address locators will have to be generated. It is likely that with both the roads and phonebook

data, storytellers would also build common historic address locators.

For anything you add to a mapstory, you will also need to remove anything that is private inormation, especially

current residents and phone numbers. There are standard ways that governments and industries have or dealing

with privacy, and we are choosing and developing the protocols that work to protect people. Typically private data

is not openly available, or visible but not searchable. We are developing ways o dealing with data, ranging rom

requiring storytellers to remove current residents or designate them as current so they are not searchable. It will

take time to develop policy and protocol, so in the meantime you must in act remove residents names and phone

numbers entirely, and only show the addresses, while or businesses, you can add whatever inormation you wish

rom phonebooks. Until the protocols are developed, have the original copy saved that contains all the data.

Annotations, Information

You should add annotations and inormation to a mapstory. The instructions or annotations are all online, under

the How-To section. Eventually, eatures will also have inormation that people can add, but this is ar away. In the

meantime, or any given layer, you can create a separate column that says inormation, and add it to the shapele

beore you upload it.

Georeferencing or Rubbersheeting

This is where you take all the maps you gathered, and overlay them into the geography you are intending. You can

nd instructions online or doing this in various programs. MapStory will have this in the site itsel eventually, along

with good instructions, but in the meantime, you will have to use documentation ound elsewhere.Simply take each map and a layer, probably your parcels and roads layers to map it to, and reerence points. Many

maps are drawn to precisely to scale, but or others you will have to learn practice through trial and error and get it

as close as you can.


6/6

EDiTing DATA

Eventually, editing can be done in MapStory itsel, and all edits will be tracked and can be reverted easily with the

click o a button. In the meantime, there will be a workfow that will resemble the workfow that will eventually

be done within the site. We are developing this workfow, so this will be an early overview that may have someshortcomings.

For any layer, you will probably do some manual editing, but the rst layer that would require the most would be

the roads layer. We will use that as an example, but the instructions are the same or any layer:

Prepare and add everything:

Add the generated roads layer

Lock the original year built column that was generated by the roads tool; i possible, make the column

read-only, such that any changes would require someone copying the le and editing a new one. Regardless

o whether that capability is not available, make sure there is an original zipped duplicate o the generated

roads shapele.

Create a separate column or manually entered dates.- two columns that are Start Date and End Date

Add a column or labeling changes. This will reer to the layer or map that the manually entered date is

based on.

Create a Notes column. This is or notes where the reason something changed was not by reerring to non-

mapped resource or some other reasoning.

Add any rubbersheeted maps. This can o course be done as you rubbersheet maps, on the fy.

Start editing:

Make all the roads one color, say black.

I using year, say 1902, create color scheme, where, say red

MapStory Local How-To, DRAFT

Documents

Transcript of MapStory Local How-To, DRAFT