Languages
Pages
Legal
2. Topics Review Community and Evolution of GlusterFS Feature overviewGranular lockingReplication Improvements (AFR)Unified file and object storageHDFS compatibility06/13/12 3. 1. Quick Review06/13/12 4. Simple EconomicsSimplicity, scalability, less costVirtualized Multi-Tenant Automated CommoditizedScale on Demand In the Cloud Scale Out Open Source06/13/12 5. What is GlusterFS, Really? Gluster is a unified, distributedstorage system DHT, stackable, POSIX, Swift, HDFS06/13/12 6. What Can You Store?Media Docs, Photos, VideoVM Filesystem VM Disk ImagesBig Data Log Files, RFID DataObjects Long Tail Data06/13/12 7. 2. Community and GlusterFSEvolution06/13/12 8. Community-led Features2009 GlusterFS easier to use2010 CLI, shell, glusterd2011 Marker framework, geo-replication06/13/12 9. GlusterFS in 2011Scale-out NASDistributed and replicatedNFS, CIFS and native GlusterFSUser-space, stackable architecture A good platform to build on06/13/12 10. GlusterFS in 2011: TheGaps Object storage popularized by S3Simplicity bias GET & PUTCombined with RESTful APIUsed mostly in web-based applications06/13/12 11. GlusterFS in 2011: TheGaps Big data, semi-structured data No Hadoop, MapReduce capabilities Structured data (databases)No MongoDB, Oracle, MySQL capability06/13/12 12. GlusterFS in 2011: TheGaps VM image hosting difficultiesDifficulty in self-heal, rebalancing Small filesPHP-based web sites, primary email storage06/13/12 13. 3. Feature Overview06/13/12 14. GlusterFS in 2012:Filling the Gaps Better replication Granular locking Proactive self-healing Quorum enforcement Synchronous translator API06/13/12 15. Granular Locking Server fails, comes back Files evaluated Block-by-block until healedBlocks compared Virtual Disk 1-1 Virtual Disk 1-2Virtual Disk 2-1 Virtual Disk 2-2GlusterFS GlusterFS Server 1Server 206/13/12 16. Proactive Self-healingPerformed server-to-serverRecovered node queries peersServer 1 - good Server 3 - good / Symlink 1 Hidden | Symlink 2DistributedSymlink 3 ReplicatedServer 2 - recoveredServer 4 - good File 1 File 1 File 2 File 2 File 3Self-healing File 3 06/13/12 17. Split Brain Nodes cannot see each other, but canall still write Often due to network outages Sometimes results in conflicts Up to 3.2, GlusterFS had no concept ofquorum06/13/12 18. Quorum Enforcement Which node has valid data? If quorum, keep writing, else stop Configurable optionServer 1 Server 2 Server 3-No quorum -Quorum-Quorum -Stops writing-Keeps writing -Keeps writing BrokenConnection06/13/12 19. Quorum EnforcementAfter connection restored, self-heal kicks off Replica 1Replica 2 Replica 3-No quorum -Quorum-Quorum -Stops writing Self-heal-Keeps writing -Keeps writing-No quorum -Quorum-Quorum -Stops writing-Keeps writing -Keeps writing06/13/12 20. GlusterFS in 2012:Filling the Gaps Synchronous translator API Unified File and Object Storage (UFO) HDFS-compatible storage layer06/13/12 21. Synchronous TranslatorAPI GlusterFS runs asynchronouslynon-blocking I/O, for performance Writing code for async I/O confusing06/13/12 22. Synchronous TranslatorAPI 3.3 introduces synchronous translatorsEasier to writeGreat for non-core operationsEg. background scrubbing06/13/12 23. Unified File and Object(UFO) S3, Swift-style object storage Access via UFO or Gluster mountHTTP Request Client AccountVolumeProxyID=/dir/sub/sub2/fileContaine Directoryr NFS or ClientFile ObjectGlusterFS Mount06/13/12 24. Unified File and Object(UFO) Your gateway to the cloud Your data, accessed your way06/13/12 25. HDFS Compatibility Run MapReduce jobs on GlusterFS Add unstructured data to HadoopHadoop ServerGlusterF GlusterF SS GlusterF GlusterF Local DiskSSHDFSConnector06/13/12(Jar file) 26. Thank you! AB Periasamy Office of the CTO, Red Hat [email protected] John Mark Walker Gluster Community [email protected]
Top Related