State of Gluster Performance
-
Upload
glusterorg -
Category
Technology
-
view
526 -
download
0
Transcript of State of Gluster Performance
![Page 1: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/1.jpg)
State of Gluster PerformanceManoj Pillai - [email protected] Hat Perfomance and Scale Engineering
![Page 2: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/2.jpg)
Agenda● Motivation● Gluster performance by workload class● EC for video storage use-cases● Work in Progress
![Page 3: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/3.jpg)
Motivation● Understand gluster performance characteristics, for
different workload classes and workload parameters● Identify areas for improvement
![Page 4: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/4.jpg)
Configuration● Systems:
○ 12 drives per server
○ Replica-2 results with single 12-disk RAID-6 device on each server
○ EC results with brick on each drive○ Upto 6 servers, 4 clients in these tests○ 1 10GbE link each
● Software versions○ Glusterfs-3.8.2-1.el7.x86_64, RHEL-7
● Standard tuning○ lookup-optimize, client-io-threads: on○ event-threads=4
![Page 5: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/5.jpg)
Sequential I/O: Replica Volume
● With sufficient number of requests concurrently in flight, close to network line speed on both writes and reads.
![Page 6: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/6.jpg)
Sequential I/O: Disperse Volume
● Similar to dist-rep, but larger gap in throughput between low thread count and high thread count.
![Page 7: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/7.jpg)
Seq I/O: Comparison of Volume Types
● Replica-2 performs better for low thread count, especially on writes.
● Disperse excels for writes with higher no. of threads.
![Page 8: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/8.jpg)
Seq Write: Comparison with kNFS
● kNFS better on single-thread tests, gluster does well with higher number of threads.
![Page 9: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/9.jpg)
Seq Read: Comparison with kNFS
● kNFS dominates on single-thread test, hits line speed
![Page 10: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/10.jpg)
Random Read: Comparison with kNFS
● “1-cached” is single thread test where server-side cache has not been dropped.○ Good test of latency in the I/O path
![Page 11: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/11.jpg)
Random Write: Comparison with kNFS
● All of these tests bottleneck on disk○ RAID-6 not a good choice for random writes.
![Page 12: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/12.jpg)
Random Read: Comparison of Volume Types
![Page 13: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/13.jpg)
Random Write: Comparison of Volume Types
● Replica-2 suffers possibly because of RAID-6 poor performance on random writes.
![Page 14: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/14.jpg)
Small File Read: Comparison of Volume Types
● Disperse at a clear disadvantage in these tests
![Page 15: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/15.jpg)
Small File Create: Comparison of Volume Types
● Small file performance still under investigation
![Page 16: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/16.jpg)
EC for Video Storage Use-Cases
● Lot of interest seen in this area● Significantly lower cost for EC, good performance
![Page 17: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/17.jpg)
Work In Progress
● Rebalance Performance○ Crawl, migration and throttle settings
● Small-file Performance○ Performance degradation with increasing number of
files
● Tiering performance for small files○ Small file performance with EC cold tier
![Page 18: State of Gluster Performance](https://reader031.fdocuments.us/reader031/viewer/2022021813/5871aea71a28abda6a8b6293/html5/thumbnails/18.jpg)
Theme
Thanks!!