Columbro - Alfresco 4 Scalability and Benchmarking
Transcript of Columbro - Alfresco 4 Scalability and Benchmarking
![Page 1: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/1.jpg)
Alfresco Scalability Benchmarking
Before telling how cool Alfresco is, you better prove it!
![Page 2: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/2.jpg)
Few items we’ll talk about…
• Scalability and benchmarking in the ECM context
• Alfresco 4 is rocket scalable. And we got proofs
• If your software scales, your BM framework must scale more!
![Page 3: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/3.jpg)
Scalability and benchmarking
What do you know about it?
![Page 4: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/4.jpg)
One size fits all…
![Page 5: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/5.jpg)
How scalable is your ECM?
“ECM systems can be scalable or they can fail to scale well. They can have modular architectures that allow you to simply add more
elements as required, rather than multiply the entire system as things expand. They can be scalable in that they have built in high availability, automatic
failover support, run on enterprise grade application servers and databases. They can be scalable because they have been tested and proven to handle very
high volumes (hundreds of millions of documents) in the repository and/or tested and proven to handle very high throughput rates (tens of thousands per
hour or minute). There are many ways in which an ECM system can scale or not. But the biggest element determining whether the system can scale
is your usage of it”
http://www.realstorygroup.com/Blog/1403-Scalable-ECM
![Page 6: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/6.jpg)
Alfresco ECM Solutions
![Page 7: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/7.jpg)
Factors at Alfresco Scalability
Just so we talk the same language:
![Page 8: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/8.jpg)
What you need to know about Alfresco 4?
Key scalability changes:
![Page 9: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/9.jpg)
The game changer(s) – Node Creation
Alfresco 3.x
![Page 10: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/10.jpg)
Game changer(s) – Node Creation
Alfresco 4.x
NOTE: Solr will index with a default period of 15s
![Page 11: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/11.jpg)
Game changer(s) – Node Search
Alfresco 3.x
![Page 12: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/12.jpg)
The game changer(s) – Node Search
Alfresco 4.x
NOTE: Solr needs to track ACLs!
![Page 13: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/13.jpg)
![Page 14: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/14.jpg)
Alfresco 4 Scalability Benchmarks
The journey to the ultimate knowledge of Alfresco
![Page 15: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/15.jpg)
Why benchmarking?
![Page 16: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/16.jpg)
Verba volant, scripta manent
• Alfresco Product• Validation of Alfresco 4 scalability• Product bottlenecks
• Alfresco Field• Sizing• Achieve even more stellar use-cases
• Customers & partners• Provide reference milestones• Allow contextualized benchmarks
![Page 17: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/17.jpg)
Benchmark projects overview
BM-0001 – Alfresco 4 Scalability – 4.0.0BM-0002 - 3.4 vs. 4.0 (Share Scenario - 360 u – 1 node)– 3.4.8 4.0.0BM-0003 - SOLR vs. Lucene (Share Scenario) – 4.0.0BM-0004 - Uncover repository limit using simple Site-based data – 4.0.2BM-0005 - Measure Alfresco Cloud signup rates up to 125K users - CloudBM-0006 - Measure Activiti workflow service performance – 4.0.2BM-0007 - Measure Alfresco workflow API performance – 4.0.2BM-0008 - Simulate 125K users on Alfresco Cloud – Cloud
BM-0009 - Define optimal tuning and extrapolate sizing information for large scale Share Enterprise deployment - 4.1.1.x
![Page 18: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/18.jpg)
BM-0001 and BM-0009
• Common factors• Benchmark Lab• Enterprise Collaboration Scenario• Technologies (Alfresco Enterprise + Jmeter)
• Differences• Load testing scripts• Database tier• Repository content
• Both provided useful insight!
![Page 19: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/19.jpg)
The benchmark lab (HW)
![Page 20: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/20.jpg)
BM-0001 – Alfresco 4 Scalability benchmark
• You can never forget your first time!
• Objective• Statistic analysis of Alfresco Scalability • Best effort pre-tuning• Scenario• Search intensive Collaboration scenario• 10s think time • Implemented with Jmeter
• Async requests• Memory intensive
![Page 21: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/21.jpg)
BM-0001 – Scenario
http://svn.alfresco.com/repos/alfresco-open-mirror/benchmark/scripts/SHARE/share-0001/V4.0.0/
![Page 22: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/22.jpg)
BM-0001 – Scalability data points
![Page 23: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/23.jpg)
BM-0001 – Architecture
![Page 24: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/24.jpg)
BM-0001 - Software
![Page 25: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/25.jpg)
BM-0001 – Scalability results
![Page 26: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/26.jpg)
In other words…
![Page 27: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/27.jpg)
BM-0001 take-aways
• 1100 concurrent users on 10M docs! • With high search %, load is mostly on Solr• Share is lightweight, repo not loaded• Solr can be memory intensive• Make sure you give enough memory!• Scale out when needed!
• Dedicated Alfresco for Solr tracking beneficial
![Page 28: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/28.jpg)
The “Alfresco 4 Scalability blueprint”
• Broad intro to Alfresco 4 Scalability • Scalability analysis of BM-0001 results• Architectural options• Tuning and configuration details• Load analysis• Sizing and performance reference• Available for Enterprise Customers &
Partners at http://support.alfresco.com
![Page 29: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/29.jpg)
BM-0009 – Real life collaboration
• Errare humanum est, perseverare diabolicum!• Objective• Create a real repository and scenario
• 2*Alfresco + 2*Solr• Find optimal tuning / sizing for large concurrency
• Scenario• Much less search intensive than BM-0001• 15s think time • Implemented with Jmeter
• More Async requests• Less Memory intensive
![Page 30: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/30.jpg)
BM-0009 – Scenario
https://svn.alfresco.com/repos/alfresco-enterprise/benchmark/scripts/SHARE/share-0002/4.0.2/
![Page 31: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/31.jpg)
BM-0009 – Repository details
• Share Sites: 10 000 • Avg members per site: 100 + 3 random
groups• #files per site: 10 2MB docs + 1k docs of
1kB • 3 level deep folder structure, each folder of the
hierarchy contains 5 documents and 5 folders
• Users in repo: 50 000• Groups: 30 000 hierarchical (depth=7)• Storage used: 1TB • Number of nodes: >10M• Number of assocs: >10M child assocs
![Page 32: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/32.jpg)
BM-0009 – Architecture
![Page 33: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/33.jpg)
BM-0009 - Software
![Page 34: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/34.jpg)
BM-0009 – Scalability results
![Page 35: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/35.jpg)
Once again…
![Page 36: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/36.jpg)
BM-0009 take-aways
• Did I already say Alfresco ROCKS? • Even on a high end realistic scenario, avg
time 1.2s with 500 concurrent users (with 2*Alfresco and 2*Solr)
• Dedicated Alfresco tracking beneficial mostly for operational purposes (similar performance)
• Adding Solr nodes allows further degrees of scalability
![Page 37: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/37.jpg)
Lessons learned (tuning tips)
As we are not in marketing…numbers are not enough!
![Page 38: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/38.jpg)
Make sure house is clean!
•Allow enough Open files
•Allow enough connections to your DB
•Allow enough Tomcat threads (but don’t exaggerate!)
•Disable your anti-virus!
•Check your balancer, and first test without it!
![Page 39: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/39.jpg)
Areas you might want to tune
•Alfresco Repository•db.pool.* (and DB connections accordingly)•(only if your know what you’re doing) L2-caches and transactional caches •solr.maxConnections
•Solr•Solr caches in solrcore.properties•#tracking threads (default=3). I/O bound so don’t exaggerate!•alfresco.maxConnections•mergeFactor in solrconfig.xml
![Page 40: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/40.jpg)
Dedicated vs. shared tracking
![Page 41: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/41.jpg)
Dedicated vs. shared tracking
![Page 42: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/42.jpg)
Benchmark gotchas
• Jmeter is very memory intensive• 30G to scale to 1100 users!• Not fit for cloud-scale• Jmeter does not parse Javascript• Complex hacked implementation to mimic Share• Full emulation is impossible!
• We need a scalable, distributed, flexible framework to run cloud-scale benchmarks!
![Page 43: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/43.jpg)
Alfresco Benchmark framework
I.e. cool people doing even cooler stuff to prove they are cool!
Derek Hulley, Repository Lead Engineer
![Page 44: Columbro - Alfresco 4 Scalability and Benchmarking](https://reader035.fdocuments.us/reader035/viewer/2022062407/55cf9e01550346d033b04219/html5/thumbnails/44.jpg)
Gabriele ColumbroPrincipal Architect Consulting Services
@mindthegabz
Derek HulleyFounding Engineer and Repository Team Lead