Multi Domain Monitoring NORDUnet 2008 Espoo, 2008-04-09 Jon Kåre Hellan, UNINETT R&D.
Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.
-
Upload
ella-larsen -
Category
Documents
-
view
212 -
download
0
Transcript of Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.
![Page 1: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/1.jpg)
Storage Services
Let the data flow!NorduNet 2008, .fi, 9 April 2008
Jan Meijer
![Page 2: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/2.jpg)
2
Storage importance increasing
![Page 3: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/3.jpg)
3
Something is brewing...
Storage as (a) utility service(s) within the
European higher ecudational and
research community
![Page 4: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/4.jpg)
4
Spot the differences
high capacity network as a utility
compute cycles as a utility
data storage as a utility
![Page 5: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/5.jpg)
5
Run that buzz by me again, please?
Storage as (a) utility service(s)
institutional / national / international scale
combining centralization, distribution, federation
creating a location and administrative domain agnostic ’make it so’ storage layer
![Page 6: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/6.jpg)
6
So where does this fit in?
![Page 7: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/7.jpg)
7
Drivers ’stuff’ as a service, the $500M data centers technology: because we can storing is cheaper then re-acquiring the
data. Sometimes reacquiring is impossible data explosion enabling new applications with old data limited pool of experts: institutional,
national, within Europe need to deal with an ever increasing
complexity and array of technologies increased reliability demands Do more with less money Storage as an enabler
![Page 8: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/8.jpg)
8
the storage mix
Tiered storage HSM Tapes don’t spin..
![Page 9: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/9.jpg)
9
Storage mix: Centralize, distribute, specialize
![Page 10: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/10.jpg)
10
Revisiting that ’utility service’ buzz
’Half’ services: sites doing end-user frontends, end-user support
Block storage: iSCSI, FC (..), InfiniBand?
File systems: CIFS, (p)NFS, GPFS, ....
Archiving, data curation Full services: disaster
recovery, off site backup, storage frontends, ..?
![Page 11: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/11.jpg)
11
Relevant tech trends
SSDs! Fast, high throughput, reliable
Improved understanding of storage reliability.
Storage middlewares, server virtualization
iSCSI Increasing demands for data
security Increasing pressure on availability
of data, services
![Page 12: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/12.jpg)
12
Lucky us! Fast networks!
1 PetaByte = 8,000,000,000,000,000 bits
10 Gbit = 10,000,000,000 bits 1 day is 86400 sec
And this matters why? NorStore has 2x600TB netto in phase 1 1 minute of JPEG2000 video = 6GB LHC > 1 PB/year raw data production Data is kept!
![Page 13: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/13.jpg)
13
Other challenges
proof of data ownership single name spacing, tight integration
between disparate storage elements what with storing data across borders? predictability (measurements) reliability of our storage archiving & data curation! AAI-enabling of storage access methods energy consumption
![Page 14: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/14.jpg)
14
What are *we* doing?
Investigating what makes sense for our community
Coordinating, stimulating collaboration between sites (network!)
Learning...and putting that to use in
NorStore Experimenting
![Page 15: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/15.jpg)
15
File Exchange
![Page 16: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/16.jpg)
16
File exchange + EduGain + NorStore
![Page 17: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/17.jpg)
17
Digital notary experiment
Investigating: Proof of data ownership Protecting data integrity Deployment options
![Page 18: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/18.jpg)
18
What are folks doing?
.hu: national storage service, iSCSI
.pl: national storage service project
national research data storage NREN storage consolidation LHC PetaFlops = PetaBytes ...
![Page 19: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/19.jpg)
19
Where is this going?
People want to share their data People want to collaborate across
organizational borders People want to have access to
their data The technology has to support this
Virtualizing storage into a service across institutional boundaries is
needed
![Page 20: Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer.](https://reader035.fdocuments.us/reader035/viewer/2022081602/55149ea5550346d36e8b58d4/html5/thumbnails/20.jpg)
20
To close with
http://www.terena.org/activities/tf-storage/
http://www.norstore.no/
Thank you
jan dot meijer at uninett dot no
BoF?