IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is...
-
date post
19-Dec-2015 -
Category
Documents
-
view
215 -
download
2
Transcript of IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is...
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Hildelies Balk, IMPACT Project Director, KB National Library of the Netherlands
Overview of the IMPACT Project
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
2
Overview of this presentation
Challenges in digitisation of historical printed text
IMPACT project and objectives
IMPACT Achievements
IMPACT Centre of Competence
How can we work together with YOU
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
3
KB Digital Library Programme Goal: Offer everyone access to everything published in and about the
Netherlands through the internet
2013: 10% of the publications published in and about the Netherlands available in digital form (60 M pages by KB, 13 M pages by third parties)
Offer our full text collections in such a way that they can be immediately used by researchers
Example projects:Historical Newspapers – http://kranten.kb.nlDutch Parliamentary Papers – http://www.statengeneraaldigitaal.nl/Early European Books (Proquest) , 18th and 19th century books (Google), other projects - http://www.kb.nl/hrd/digitalisering/index-en.html
Timeframe covered: 1618 - 1995
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
4
So we offer this….
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
5
With this message ….
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
6
OCR problems
Damaged pages, bleed through, difficult layout, historic fonts …
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7
Warping of paper (due to humidity)
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
8
Bleed through & Shine through
Bad printing: blurred, broken, faded
characters
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
9
Gothic print types
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
10
Annotations in the text
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
11
Complicated layout
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
12
Language Challenges: Spelling variants, orthographical variants, inflected forms…and more
Historical variants of the Dutch word ‘wereld’ (world):
werelt weerelt wereld weerelds wereldt werelden weereld werrelts waerelds weerlyt wereldts vveerelts waereld weerelden waerelden weerlt werlt werelds sweerels zwerlys swarels swerelts werelts swerrels weirelts tsweerelds werret vverelt werlts werrelt worreld werlden wareld weirelt weireld waerelt werreld werld vvereld weerelts werlde tswerels werreldts weereldt wereldje waereldje weurlt wald weëled
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
13
Institutional Challenge: lack of knowledge and expertise inefficiency
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
14
Answering the challenges – IMPACTIMPACT – Improving Access to Text (2008-2011) Large-scale integrating research project Consortium of 26 partners
Good mix of public and private partners Users, researchers and industry work together to find solutions Each established in a large international network
Coordinated by the National Library of the Netherlands (KB) Co-funded by EU (FP7 ICT Work Programme)
From 2012: sustainable Centre of Competence with alternative resources
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
15
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
16
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
17
IMPACT objectives
Significantly improve mass digitisation of historical printed text by:
Innovate OCR software and language technology → tools for each step in the digitisation workflow from scan to publication
Share expertise and building capacity across Europe Ensure that tools and services will be sustained after the end of the
project
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
18
IMPACT Achievements: summary On market: Improved commercial OCR Ready for real life testing:
– Adaptive OCR engine– Tools for OCR correction with volunteer involvement – Computerlexica for nine languages – Digitisation Framework with evaluation tools and dataset– Knowledge bank with guidelines and learning resources– Service for for print space recognition
For future development:– Novel Approaches to preprocessing, OCR and post correction– Tools for lexion building
Added value: Unique network bringing together experts from different communities Centre of Competence for digitisation to start 1 january 2012
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
19
IMPACT Achievements:
Examples
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
20
Preprocessing: Novel Approaches to image enhancement
Border removal and dewarping by NCSR and USAL
before after
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
21
OCR: Improved commercial engine on market: ABBYY FR10
Historic European font:
FRE10 recognition of historic fonts:
25% more accurate than FRE9 38% more accurate than FR XIX
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
22
OCR correction: two effective tools ready for implementation
Both make use of volunteer involvement
CONCERT by IBM: collaborative correction feeds back into Adaptive OCR
→ promising pilots by libraries
LMU Post correction tool based on language input → pilot to start soon
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
23
Language: lexica for nine languages
Correction of Long S with IMPACT lexicon for historical Dutch
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
24
Post Processing: Print space recognition
Functional Extension Parser by UIBK
Recognition of the structure of book pages
Enrichment of OCR results with structural information
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
25
Evaluation: IMPACT Framework
Modular and transparent method for evaluating specific workflows
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
26
Evaluation: IMPACT Dataset Over half a million representative pages of digitised historical texts (newspapers,
books, pamphlets, typewritten material) from the collections of 11 European libraries, with unique IDs and metadata
Invaluable resource for future research in OCR and language technology.
Twitter: @impactocr, #impactproject
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
27
Centre of Competence in digitisation
New community: Bridges the gap between – content holders with digitisation programmes and – scientific communities in the area of pattern recognition, language technology, image
processing Mission: making Europe’s heritage accessible in digital form Focus on practical solutions Provides support in the implementation of the innovative IMPACT solutions for
improving access to text Provides tools and services for further advancement of the State of the Art in the
field Organises Conferences/workshops
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
28
How to join
Three levels of membership : Open (registration) access to forum, part of content Basic membership (fee): access to all facilities, reduced fee for conferences Premium membership (fee): member of the Board, privileges such as free entry to
conferences
Want to sign up? Mail to [email protected] for information on membership Join us now already on LinkedIn Follow us on Twitter (@impactocr) Access through www.impact-project.eu
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
29
Houston: our ideas on working togetherLow hanging fruit: Sharing open source solutions Evaluating them in our framework with Ground Truth Building a good set of use cases for all available tools Sharing case studies on digitisation problems
Adressing the big remaining challenge: Getting the tools to work in real life environments Bridging the gap between techy solution and content holders workflow
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
30
Questions?
www.impact-project.eu
Thank you!
Twitter: @impactocr, #impactproject