Reclaiming our story

using digital archives to preserve the history of Japanese American incarceration

IEEE Big Data 2018: 3rd CAS workshop 12 December 2018

Geoff Froh, Deputy Director

Credits:“A woman with her catch during a fishing trip,” Courtesy of Densho and the Fujii and Mori Families Collection, https://ddr.densho.org/ddr-densho-321-1127

Credits:“Picnic,” Courtesy of Densho and the Tamura Collection, https://ddr.densho.org/ddr-densho-124-1“Man digging potatoes,” Courtesy of Densho and Manzanar Historic Site Collection, https://ddr.densho.org/ddr-densho-153-4“Sister and brother in front of a car,” Courtesy of Densho and Akimoto Collection, https://ddr.densho.org/ddr-densho-129-2

Credits: “Film advertisement: The Yellow Menace (September 23, 1916),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-287"Japanese Exclusion League Will Finish Organization To-Day,” Courtesy of Densho and the San Francisco Chronicle Collection, https://ddr.densho.org/ddr-densho-69-32“Opposed to Jap Labor. Local Unions Ask for an Investigation of the Influx of Brown Men. (April 19, 1900),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-4“Pacific Coast at Jap's Mercy, Says Senator. Works, of California, Says Million Men Are Needed to Protect Pacific Coast -- Would Give Them Land. (December 16, 1915),” Courtesy of Densho and the Seattle Times Collection, ddr-https://ddr.densho.org/densho-56-277“Aliens Pour Through the Mexican Gateway (July 2, 1907),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-91“Alien Land Ban Tightened! Cropping Contracts Are Invalid. Persons Ineligible to Citizenship Cannot Make Farming Agreements, U.S. Supreme Court Holds. (November 19, 1923),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-383“Japanese Barred by New Bill. Immigration Quotas Reduced. House Committee Approves Important Provisions of Measure, Including One Writing 'Gentlemen's Agreement' Into Law. (February 6, 1923),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-378“Trouble Caused by Japanese Question. Immigration Inspector C. W. Snyder of Los Angeles, Says Orientals Are to Blame For Much of Bad Feeling (July 17, 1907),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-95"Japanese Bring Vile Diseases,” Courtesy of Densho and the San Francisco Chronicle Collection, https://ddr.densho.org/ddr-densho-69-34“California Idea of Japanese Situation: Cartoon Published in San Francisco Indicating Extreme Jingo Viewpoint (May 9, 1913),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-231

WCCA Seattle Exclusion Order No. 17

Credit:“Mass removal,” Courtesy of Densho and the Bainbridge Island Japanese American Community Collection, https://ddr.densho.org/ddr-densho-34-175

Credit:“Tule Lake concentration camp,” Courtesy of Densho and Bain Collection, https://ddr.densho.org/ddr-densho-2-33

Densho

• Founded by members of the Japanese American community in 1996

• Original purpose

• Capture the stories before those who experienced the incarceration are gone

• Use those stories to educate, promote, and advance the ideals of democracy and to encourage civic engagement

Credit:Excerpted from “Kara Kondo Visual History,” Courtesy of Densho, https://ddr.densho.org/ddr-densho-1000-139

Credit:“Family in front of barracks,” Courtesy of Densho and Manzanar National Historic Site Collection, https://ddr.densho.org/ddr-densho-153-7/

Credits:“Scene the Pictorial Magazine Vol. 3 No. 1 (May 1951),” Courtesy of Densho and Scene and Cue Collection, https://ddr.densho.org/ddr-densho-266-30"Camp road,” Courtesy of Densho and Bigelow Collection, https://ddr.densho.org/ddr-densho-156-1“Children on a teeter totter,” Courtesy of Densho and Bigelow Collection, https://ddr.densho.org/ddr-densho-156-128"Application for Leave Clearance,” Courtesy of Densho and the Yamanaka Collection, https://ddr.densho.org/ddr-densho-188-5“442nd Regimental Combat Team soldier,” Courtesy of Densho and the Frances Haglund Collection, https://ddr.densho.org/ddr-densho-275-10“Two Japanese Americans picking strawberries,” Courtesy of Densho and the Bain Collection, https://ddr.densho.org/ddr-densho-2-26“Citizen's indefinite leave card,” Courtesy of Densho and Yoshihara Collection, https://ddr.densho.org/ddr-densho-126-4“Manzanar Free Press Vol. II No. 46 (November 5, 1942),” Courtesy of Densho and Manzanar Free Press Collection, https://ddr.densho.org/ddr-densho-125-6

WRA Form 26 (“Individual Record”)

Form 26REFERENCE COPY OF TECHNICAL DOCUMENTATION FOR ACCESSIONED ELECTRONIC RECORDS(Copied: May 11, 2005)Records About Japanese Americans Relocated During World War II RecordGroup 210: Records of the War Relocation Authority

Credit:“Card Punch Operators working on population cards, Negro section,” National Archives and Records Administration, Retrieved from https://prologue.blogs.archives.gov/2014/05/19/executive-orders-9980-and-9981-ending-segregation-in-the-armed-forces-and-the-federal-workforce/

Final Accountability Rosters (FAR)

Final Accountability Roster, Heart Mountain Relocation Center, 1946.

Issues with registry data

• Form WRA-26• Truncated and summarized

data values• Transcription and reformatting

errors• Electronic version created from

incomplete, secondary dataset

• FAR• Legibility of originals• Schema and syntax differs

between camps• Variable quality of microfilm

transfer• Multiple versions of roster

Credit:“Japanese Americans waiting in line for inoculations,” Courtesy of Densho and Dorothea Lange Collection, https://ddr.densho.org/ddr-densho-151-131

Names Registry FAR Data Prep Workflow

Raw PDF Decompose to page sets Transcribe Normalized

Excel filesReconcile and

Q/AConsolidated

data file

Names Registry data prep workflow

Names Registry Consolidation Workflow

Marshal raw CSV files

Create pseudo-ID, filter private data, and

perform secondary QA

Index in ElasticSearch and publish cleaned CSV files

to web

FAR

WRA

Names Registry data prep workflow

Names Registry v1

Names Registry: looking ahead

• True name authority + family graph

• More data sources• Military service lists• DoJ, INS, US Army and

other facility registers

• Integration with Densho Digital Repository, Encyclopedia and other website projects

• Public API and LOD service

Credit:“Group of Issei hiking,” Courtesy of Densho and Ouchi Family Collection, https://ddr.densho.org/ddr-densho-182-19

Processing at scale

• Acquiring raw data and determining “good enough”

• Finding better tools and novel approaches

• Cultivating different staff skills

Credit:“Interior of Mitsuwado store,” Courtesy of Densho and Mamiya Collection, https://ddr.densho.org/ddr-densho-13-32

Privacy and ethics

• Evaluating regulatory and legal requirements

• Implementing privacy policies and managing access

• Balancing individual vs. community concerns

Credit:“Grandfather waiting for bus,” Courtesy of Densho and the Dorothea Lange Collection, https://ddr.densho.org/ddr-densho-151-46

Credit:“Masao Sakagami looking out at a view of mountains,” Courtesy of Densho and the Masao Sakagami Collection, https://ddr.densho.org/ddr-densho-201-470

Thank you!

Questions or comments?

Contact me at: Geoff Froh <geoff.froh@densho.org>

Image creditsSlide 2:“A woman with her catch during a fishing trip,” Courtesy of Densho and the Fujii and Mori Families Collection, https://ddr.densho.org/ddr-densho-321-1127

Slide 3:“Picnic,” Courtesy of Densho and the Tamura Collection, https://ddr.densho.org/ddr-densho-124-1“Man digging potatoes,” Courtesy of Densho and Manzanar Historic Site Collection, https://ddr.densho.org/ddr-densho-153-4 “Sister and brother in front of a car,” Courtesy of Densho and Akimoto Collection, https://ddr.densho.org/ddr-densho-129-2

Slide 4: “Film advertisement: The Yellow Menace (September 23, 1916),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-287 "Japanese Exclusion League Will Finish Organization To-Day,” Courtesy of Densho and the San Francisco Chronicle Collection, https://ddr.densho.org/ddr-densho-69-32 “Opposed to Jap Labor. Local Unions Ask for an Investigation of the Influx of Brown Men. (April 19, 1900),” Courtesy of Densho and the Seattle Times Collection, https:// ddr.densho.org/ddr-densho-56-4“Pacific Coast at Jap's Mercy, Says Senator. Works, of California, Says Million Men Are Needed to Protect Pacific Coast -- Would Give Them Land. (December 16, 1915),” Courtesy of Densho and the Seattle Times Collection, ddr-https://ddr.densho.org/densho-56-277“Aliens Pour Through the Mexican Gateway (July 2, 1907),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-91“Alien Land Ban Tightened! Cropping Contracts Are Invalid. Persons Ineligible to Citizenship Cannot Make Farming Agreements, U.S. Supreme Court Holds. (November 19, 1923),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-383“Japanese Barred by New Bill. Immigration Quotas Reduced. House Committee Approves Important Provisions of Measure, Including One Writing 'Gentlemen's Agreement' Into Law. (February 6, 1923),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-378“Trouble Caused by Japanese Question. Immigration Inspector C. W. Snyder of Los Angeles, Says Orientals Are to Blame For Much of Bad Feeling (July 17, 1907),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-95"Japanese Bring Vile Diseases,” Courtesy of Densho and the San Francisco Chronicle Collection, https://ddr.densho.org/ddr-densho-69-34“California Idea of Japanese Situation: Cartoon Published in San Francisco Indicating Extreme Jingo Viewpoint (May 9, 1913),” Courtesy of Densho and the Seattle Times Collection, https://ddr.densho.org/ddr-densho-56-231

Slide 6: “Mass removal,” Courtesy of Densho and the Bainbridge Island Japanese American Community Collection, https://ddr.densho.org/ddr-densho-34-175

Slide 7: “Tule Lake concentration camp,” Courtesy of Densho and Bain Collection, https://ddr.densho.org/ddr-densho-2-33 Slide 9: Excerpted from “Kara Kondo Visual History,” Courtesy of Densho, https://ddr.densho.org/ddr-densho-1000-139

Slide 10: “Family in front of barracks,” Courtesy of Densho and Manzanar National Historic Site Collection, https://ddr.densho.org/ddr-densho-153-7/

Image credits continuedSlide 11:“Scene the Pictorial Magazine Vol. 3 No. 1 (May 1951),” Courtesy of Densho and Scene and Cue Collection, https://ddr.densho.org/ddr-densho-266-30 "Camp road,” Courtesy of Densho and Bigelow Collection, https://ddr.densho.org/ddr-densho-156-1“Children on a teeter totter,” Courtesy of Densho and Bigelow Collection, https://ddr.densho.org/ddr-densho-156-128"Application for Leave Clearance,” Courtesy of Densho and the Yamanaka Collection, https://ddr.densho.org/ddr-densho-188-5“442nd Regimental Combat Team soldier,” Courtesy of Densho and the Frances Haglund Collection, https://ddr.densho.org/ddr-densho-275-10“Two Japanese Americans picking strawberries,” Courtesy of Densho and the Bain Collection, https://ddr.densho.org/ddr-densho-2-26“Citizen's indefinite leave card,” Courtesy of Densho and Yoshihara Collection, https://ddr.densho.org/ddr-densho-126-4“Manzanar Free Press Vol. II No. 46 (November 5, 1942),” Courtesy of Densho and Manzanar Free Press Collection, https://ddr.densho.org/ddr-densho-125-6

Slide 13: “Card Punch Operators working on population cards, Negro section,” National Archives and Records Administration, Retrieved from https://prologue.blogs.archives.gov/ 2014/05/19/executive-orders-9980-and-9981-ending-segregation-in-the-armed-forces-and-the-federal-workforce/

Slide 15: “Japanese Americans waiting in line for inoculations,” Courtesy of Densho and Dorothea Lange Collection, https://ddr.densho.org/ddr-densho-151-131

Slide 19: “Group of Issei hiking,” Courtesy of Densho and Ouchi Family Collection, https://ddr.densho.org/ddr-densho-182-19

Slide 20: “Interior of Mitsuwado store,” Courtesy of Densho and Mamiya Collection, https://ddr.densho.org/ddr-densho-13-32

Slide 21: “Grandfather waiting for bus,” Courtesy of Densho and the Dorothea Lange Collection, https://ddr.densho.org/ddr-densho-151-46

Slide 22: “Masao Sakagami looking out at a view of mountains,” Courtesy of Densho and the Masao Sakagami Collection, https://ddr.densho.org/ddr-densho-201-470