Access 2011: Big Data in Libraries
-
Upload
robotninja -
Category
Technology
-
view
113 -
download
0
Transcript of Access 2011: Big Data in Libraries
![Page 1: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/1.jpg)
BIG DATABIG DATA
![Page 2: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/2.jpg)
![Page 3: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/3.jpg)
"datasets that grow so large that they become
difficult to work with using relational
databases and within a tolerable elapsed time"
![Page 4: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/4.jpg)
BIG DATA IS BIGBIG DATA IS BIG
![Page 5: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/5.jpg)
LIKE, REALLY BIGLIKE, REALLY BIG
![Page 6: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/6.jpg)
FACEBOOK: 140 BILLION PHOTOS
HUMAN GENOME: 3 BILLIONBASE PAIRS
GOOGLE: 50 BILLIONWEB PAGES
WORLDCAT: 1.5 BILLIONITEM RECORDS
![Page 7: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/7.jpg)
NOT REALLYNOT REALLY
![Page 8: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/8.jpg)
EUROPEANA: 20 MILLION(715K / COUNTRY)
LIBRARY OF CONGRESS:
1.9 MILLION
CANADIANA: 1 MILLION
LIBRARY AND ARCHIVES CANADA:
3.5 MILLION(ARCHIVAL DESCRIPTIONS)
![Page 9: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/9.jpg)
BIG DATABIG DATAIS COMPLICATEDIS COMPLICATED
![Page 10: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/10.jpg)
1966
![Page 11: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/11.jpg)
1976
![Page 12: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/12.jpg)
≠
![Page 13: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/13.jpg)
≠
![Page 14: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/14.jpg)
NOT REALLYNOT REALLY
![Page 15: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/15.jpg)
ಠ_ಠ
![Page 16: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/16.jpg)
![Page 17: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/17.jpg)
![Page 18: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/18.jpg)
SCALABILITYSCALABILITY
![Page 19: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/19.jpg)
● ICA-AtoM (LAMP)
● BENCHMARK 3.5M RECORDS (current largest: < 100K)
● 100% OPEN SOURCE SOFTWARE
● COMMODITY HARDWARE
![Page 20: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/20.jpg)
![Page 21: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/21.jpg)
CAN WE DO IT?CAN WE DO IT?
![Page 22: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/22.jpg)
WRITE SPEEDWRITE SPEED
![Page 23: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/23.jpg)
READ SPEEDREAD SPEED
![Page 24: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/24.jpg)
WRITE MEMORYWRITE MEMORY
![Page 25: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/25.jpg)
READ MEMORYREAD MEMORY
![Page 26: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/26.jpg)
NOSQL vs. SQLNOSQL vs. SQL(a.k.a. ODM vs. ORM)
● 4x - 10x FASTER
● 50% - 90% LESS MEMORY
![Page 27: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/27.jpg)
![Page 28: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/28.jpg)
![Page 29: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/29.jpg)
RELATIONAL DATABASESSCALE WELL
IF YOUR DATAIS NOT HIERARCHICAL
SOLRSCALES WELL
IF YOU HAVE INFINITE RAM
BEWARE THEDOGMA OF SQL
NOSQL IS AVIABLE OPTION
THINK SIDEWAYS SCALE OUT →
![Page 30: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/30.jpg)
![Page 31: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/31.jpg)
THE CLOUD IS A LIETHE CLOUD IS A LIE
![Page 32: Access 2011: Big Data in Libraries](https://reader033.fdocuments.us/reader033/viewer/2022052907/559036581a28ab0f2d8b47c8/html5/thumbnails/32.jpg)
“big data is less about size, and more about
freedom”
open source tools+ distributed design= new opportunities