Multimedia Information Retrieval

Prof Stefan Rüger

Multimedia and Information SystemsKnowledge Media Institute

The Open Universityhttp://kmi.open.ac.uk/mmis

Multimedia Information Retrieval

1. What are multimedia queries?

2. Fingerprinting

3. Metadata & piggy-back retrieval

4. Automated image annotation

5 Visual content-based retrieval I

6 Visual content-based retrieval II

7. Evaluation

8. Browsing, search and geography

- Information Visualisation

- User interaction

- Visual search and relevance feedback

- Browsing: lateral and geographic

Alternatives to the list

How else could results be displayed?

Sammon maps

Dendro map

Radial

Query point moving

Query point moving

Weight space movement

10

11

12

13

Connect similar objects

Colour

Text

Structurefocus [with Heesch ECIR 2004, CIVR 2005][with Heesch et al ACM MM 2006,TRECVID 2003-2004][PhD thesis Heesch, nominated forbest UK thesis 2005]

Browsing example

TRECVID topic 102: Find shots from behind the pitcher in a baseball game as he throws a ball that the batter swings at

In your mind

[Alexander May: SET award 2004: best use of IT – national prize][dataset TRECVID 2003]

Empora.com

Empora.com

Geotemporal search

Geography

Wikipedia

[Logo trademark and © by Wikimedia Foundation, Inc]

English language

[with Overell, GIR 2007; MMKM 2008]

English

[with Overell, GIR 2007; MMKM 2008]

Russian

[with Overell, GIR 2007; MMKM 2008]

German

[with Overell, GIR 2007; MMKM 2008]

French

[with Overell, GIR 2007; MMKM 2008]

Spanish

[with Overell, GIR 2007; MMKM 2008]

Portugese

[with Overell, GIR 2007; MMKM 2008]

Chinese

[with Overell, GIR 2007; MMKM 2008]

Japanese

[with Overell, GIR 2007; MMKM 2008]

[with Overell, GIR 2007; MMKM 2008]

Conclusion

Are we all little Steinbergs?

S Overell and S Rüger: View of the world according to Wikipedia: are we all little Steinbergs?International Journal of Computational Science, 2(3), pp 193–197, 2011. DOI: 10.1016/j.jocs.2011.05.006

Mobile interactions

• SnapTell• Spot&Search

SnapTell

Spot&Search: OU Art

Spot&Search: OU Art

Feedback

• Please fill in the feedback form• I want to know what worked (and what didn't!)

Multimedia Information Retrieval

1. What are multimedia queries?

2. Fingerprinting

3. Metadata & piggy-back retrieval

4. Automated image annotation

5 Visual content-based retrieval I

6 Visual content-based retrieval II

7. Evaluation

8. Browsing, search and geography

- Information Visualisation

- User interaction

- Visual search and relevance feedback

- Browsing: lateral and geographic

Thank you!

• You have been a fantastic crowd• Thank you for your interest and all the interaction• Big thanks to NII, Professor Shin'ichi Satoh,

Professor Noriko Kando and the professional international affairs and educational support team, in particular Ms Tsukushi Inagaki, for being such a great host!