DE Conferentie 2005 Egon v.d. Broek
-
Upload
digitaal-erfgoedconferentie -
Category
Education
-
view
308 -
download
1
description
Transcript of DE Conferentie 2005 Egon v.d. Broek
Multilevel Access to our Cultural Heritage (MACH)
Multilevel Access to our Cultural Heritage (MACH)
People who participated in the project
• Egon L. van den Broek (AI, VU Amsterdam /
NICI, RU Nijmegen)• Thijs Kok (ICIS, RU Nijmegen)
• Eduard Hoenkamp (NICI, RU Nijmegen)• Theo E. Schouten (ICIS, RU Nijmegen)• Peter J. Petiet (AI, VU Amsterdam)• Louis G. Vuurpijl (NICI, RU Nijmegen)
Multilevel Access to our Cultural Heritage (MACH)
In addition…The people who worked on the foundation of M4ART
• Frans Gremmen (RTOG, RU Nijmegen)• Maarten A. Hendriks (freelance)• Peter Kisters (freelance)• Harco C. Kuppens (ICIS, RU Nijmegen)• Eva M. van Rikxoort (ISI, UMC Utrecht)• Charles M. de Weert (Soc. Fac., RU Nijmegen)
Multilevel Access to our Cultural Heritage (MACH)
NWO research line ToKeN
• Two projects within the ToKeN research line:– Eidetic– VindIT
• A few of the aims of the projects– Access and enhancement of our cultural heritage – Combining Computer Science and Cognitive Science– Combining (text-based) Information Retrieval (IR) and Content-Based
Image Retrieval (CBIR)– Launching a CBIR system for the National Gallery of Arts (Het
Rijksmuseum)
Multilevel Access to our Cultural Heritage (MACH)
Key notions of this talk
• Aim: Access and enhancement of knowledge• Domain: Cultural heritage; more specific: The National
Gallery of Arts (Het Rijksmuseum)• Characteristics:
– Digital photographs of objects– Annotations of these objects; i.e., descriptions of the artist, the
material, the content/theme of the object etc. • For … both laypersons (i.e., non-experts) and experts• How: (text-based) Information Retrieval (IR) and Content-
Based Image Retrieval (CBIR)• Additionally, facilitating:
– The user’s understanding of CBIR– Comparison of IR and CBIR
Multilevel Access to our Cultural Heritage (MACH)
Current state-of-the-art in museum access through the Internet
Four examples:
Virtual Catalogue for Art History
The National Gallery of Art (Het Rijksmuseum)
The hermitage museum (powered by IBM)
Multilevel Access to our Cultural Heritage (MACH)
Multilevel Access to our Cultural Heritage (MACH)
Multilevel Access to our Cultural Heritage (MACH)
Multilevel Access to our Cultural Heritage (MACH)
Multilevel Access to our Cultural Heritage (MACH)
Multilevel Access to our Cultural Heritage (MACH)
Multilevel Access to our Cultural Heritage (MACH)
Why this aim in this domain?
• Attract people to museums through Internet• Make museums better accessible for laypersons• Make museums better accessible for experts• Providing more information through new methods of access• Gaining more understanding of CBIR through using it
• Future aims:– Interaction between users and the system through agent technology– Extracting information of artists directly from their art
Multilevel Access to our Cultural Heritage (MACH)In
form
ati
on
Re
trie
va
l (e
.g.,
as
wit
h G
oo
gle
)
Multilevel Access to our Cultural Heritage (MACH)
Color-Based Image Retrieval:3 methods of querying
• Browsing the catalogue
• Providing an image from the Internet, by way of an URL
• Uploading a local image from your desktop
Multilevel Access to our Cultural Heritage (MACH)
Multilevel Access to our Cultural Heritage (MACH)
From simple to complex user interfaces
M4ART can be described using 2 dimensions:
1) Retrieval method; either IR or CBIR
2) Complexity of querying;
- straightforward queries
- specified/combined queries
Multilevel Access to our Cultural Heritage (MACH)
Complex IR querying (1)
• Exclusion of terms
• Boolean search, using AND and/or OR operators
• Utilizing specific tags (fields) of the annotations
Multilevel Access to our Cultural Heritage (MACH)C
om
ple
x IR
qu
ery
ing
(2
)
Multilevel Access to our Cultural Heritage (MACH)
Complex CBIR querying (1)
• Color space
• Quantization scheme; i.e., how any colors do you want to distinguish?
• Distance measure
• Optionally: include texture
Multilevel Access to our Cultural Heritage (MACH)C
om
ple
x C
BIR
qu
ery
ing
(2
)
Multilevel Access to our Cultural Heritage (MACH)
Gaining understanding …
• Showing the features …
• Compare IR and CBIR techniques
Multilevel Access to our Cultural Heritage (MACH)S
ho
win
g t
he
fe
atu
res
Multilevel Access to our Cultural Heritage (MACH)D
ista
nc
e m
atr
ix
Multilevel Access to our Cultural Heritage (MACH)
Advanced schemes incorporated
• 11 color categories quantization scheme
• Parallel-Sequential Texture Analysis
Multilevel Access to our Cultural Heritage (MACH)
Color spaces (and their quantizations)
• RGB• HSV • YUV/YIQ• CIE XYZ• CIE LUV• 11 colors ...
Multilevel Access to our Cultural Heritage (MACH)
Its all only in 11 color categories
• Most famous: Berlin & Kay (1969) with their book
“Basic color terms: Their universals and evolution”
• But already 15 years before them Brown and Lenneberg (1954) presented their work in the paper: “A study in language and cognition”
• More recently:– Derefeldt, Swartling, Berggrund, and Bodrogi (2004)– Moroney (2004)– Regier, Kay, and Cook (2005)
Multilevel Access to our Cultural Heritage (MACH)
The practical use of fundamental research: Color categories
• People perceive in color categories
• People talk in color categories
• Human memory uses color categories
• Robust to inter and intra personal variability
• Computationally inexpensive
Multilevel Access to our Cultural Heritage (MACH)
THE 11 colors
• Red• Green• Blue
• Yellow• Purple• Pink• Orange• Brown
BlackGrayWhite
Multilevel Access to our Cultural Heritage (MACH)
Texture defined …
• Cross and Jain (1983): There is no universally accepted definition for texture
• Bovik, Clarke, and Geisler (1990): an exact definition of texture either as a surface property or as an image property has never been adequately formulated.
• Gonzales and Woods (2002) state: No formal definition of texture exists, intuitively this descriptor provides measures of properties such as smoothness, coarseness and regularity.
Multilevel Access to our Cultural Heritage (MACH)
Texture Analysis
• Palm (2004) noted: “The integration of color and texture is still exceptional.” Others: Drimbarean and Whelan (2001) and Mäenpää and Pietikäinen (2004)
• Techniques applied in literature, so far:– Global color analysis (e.g., color histograms)– Local gray-scale texture analysis (e.g., co-occurrence matrix)– Local color-based texture analysis (e.g., color correlogram)– Global color analysis combined with local gray-scale texture analysis (e.g.,
parallel approach)
• What is used in M4ART: Parallel-sequential texture analysis– Global and local color analysis in parallel: the color histogram and the color
correlogram combined.
Multilevel Access to our Cultural Heritage (MACH)
Gray versus Color
Multilevel Access to our Cultural Heritage (MACH)
Parallel-Sequential texture analysis*
* More info? Broek, E. L. van den and Rikxoort, E. M. van (2005). Parallel-sequential texture analysis. Lecture Notes in Computer Science (Advances
in Pattern Recognition), 3687, 532-541.
Multilevel Access to our Cultural Heritage (MACH)
Back to C-BAR: Some last remarks
• Technical specifications
• Freely accessible
• Questions?
Multilevel Access to our Cultural Heritage (MACH)
Technical specifications
• W3C HTML 4.01 validated
• W3C CSS validated
• Running on an Apache HTTP server
• PHP 5.1 powered
• Running on a MySQL database
Multilevel Access to our Cultural Heritage (MACH)
Freely accessible
• All feedback = welcome!
• How to get there …
Use Google query on ‘M4ART’ or go through:
http://cai.nici.ru.nl/M4ART/
Multilevel Access to our Cultural Heritage (MACH)
Thank you for your attention!
Questions?