Alfresco Day Roma 2015: Full Stack Load Testing
-
Upload
alfresco-software -
Category
Technology
-
view
643 -
download
1
Transcript of Alfresco Day Roma 2015: Full Stack Load Testing
Full Stack Load Testing Load Testing On Alfresco 5.1
Derek Hulley Alfresco Founding Engineer
Introduction
• Resource Limits
• Load Test Accuracy
• Load Test Software
Introduction
Resource Limitations
Limiting Factors • Some Basic Server Limitations
– Memory consumption – CPU usage – IOPS / Bandwidth / IO
• Other Server Limitations – Monitoring (getting at logs, charts, monitors, etc) – Backups – CRON jobs
• Features – Supported file types – Auditing – Jodconverter – Clustering
Resources
Moving Parts
Resources
Caches
Start with a guess and try it out in anger Alfresco and
Share 20%
Transforma4ons
10%
SOLR 30%
Database 30%
Content 5%
Load Balancer 5%
Resource Guess 0
Alfresco and Share Transforma2ons SOLR
Database Content Load Balancer
Resources
Monitor all Run for days Resolve Issues Repeat Adjust resources
Focuse.g. SOLR
Resources
Alfresco and Share 20%
Transforma4ons
10%
SOLR 30%
Database 30%
Content 5%
Load Balancer 5%
Resource Guess 0
Alfresco and Share Transforma2ons SOLR
Database Content Load Balancer
Resource Focus Example: SOLR • CPU: underused
• Memory: underused
• IO: underused
• Network: Keep and eye on this!
Resources
Reduce spend and iterate …
Resources
Alfresco and Share 23%
Transforma4ons
12% SOLR 18%
Database 35%
Content 6%
Load Balancer 6%
Resource Guess 2
Alfresco and Share Transforma2ons SOLR
Database Content Load Balancer
Load Test Accuracy
Load Test Risk Factors: Scale
• Increases Cost – Your Time – Getting the data in does not normally produce results
• Bulk Loading – Different hardware for this phase? – Simulate bulk loading as accurately as necessary
• Number of Documents – Easier to account for with experience – Data storage costs – Backup/Restore procedures
• Users and Groups – LDAP sync – Complex Group Hierarchy – Site Group Permissions – Mass-authentication
Load Test Accuracy
Load Test Risk Factors: Documents • Document Types
– Image thumbnail generation – Text to PDF for preview – Document library listings (many transformations a.s.a.p)
• Document Size – Network and IO costs and limits – CPU used for streaming – PDF to text memory consumption – External converters
• More CPU or more infrastructure
Load Test Accuracy
Load Test Risk Factors: API Usage Profile • Load Profile
– Burst load – Trickle load
• Data Structure – Paging – Sorting – Permissions
• The Internet – CMIS across the Atlantic – Southern hemisphere client apps
Load Test Accuracy
Load Test Risk Factors: UI Complexity • Share APIs
– /private • Upload File
– Stream binary – Then all of these at once:
• Register user activity • Generate thumbnail • Generate PDF preview and display • Get and display document metadata • Get and display document version history
• Replay Doom (You Only Live Once)
Load Test Accuracy
Load Test Risk Factors: UI Complexity Load Test Accuracy
Load Test Risk Factors: UI Complexity • Page-Object Pattern: Browsers do what they do
Login Event Processor Login Page Dashboard Page Browser
processEvent loginAs
Send keystrokes
Login button click DashboardPage
Tim
ed
render Wait For: Activities
Wait For: My Sites
Wait For: My Sites
Wait For: My Sites
Wait For: My Docs
Wait For: etc
Increasing Complexity
Server
x100
Load Test Accuracy
Load Test Software Alfresco Benchmark Server and Tests
References • Martin Bergljung’s walkthrough • Alfresco Wiki: Benchmark_Testing_with_Alfresco • GitHub: AlfrescoBenchmark
Test Software
Alfresco Benchmark Suite
Test Software
Benchmark Server
Tomcat 7
Rest API
MongoDB Config Data
Services
MongoDB Test Data
UI
Benchmark Driver (xN) Benchmark Driver (xN) Benchmark Driver
Tomcat 7 Extras (Selenium)
Servers / APIs Servers / APIs
Load Balancer
Servers / APIs
Test
Services
Rest API
Load Tests: /tests/ent-signup • Create users • User base reused
– Prerequisite for other tests
Test Software
Load Tests: /tests/dataload • Reuse user base • Create
– Sites – Site Members
• Site data – Folders – Files uploaded using CMIS or
Test Software
Load Tests: /tests/dataload • AlfrescoBenchmark/alfresco-text-gen
• Text data generated in server • Option to push data
into ContentStore
Test Software
Load Tests: GitHub benchmark-cmis • Burst load • Easy to extend
Test Software
Load Tests: FTP sample files vs spoofing • FTP server
– A few thousand samples – Use for Share tests
• Transformations • Text extractions • Renditions
• Spoofing – Stress SOLR – document library size
• Search: searchterms.txt
Test Software
Share Load Testing: /tests/share • Concurrent Users:
– A new user logs in every N seconds – The user logs out after spending Y seconds – Number of concurrent users C = Y/N
• Selenium Driver Overload – Use Windows – 5s to start a browser on EC2 m3.2xlarge – Up to 40 browsers per load driver on EC2 m3.2xlarge – Drivers required D = max( 5s/N, C/40) – m3.2xlarge Windows à spot price of $0.3/hour – 500 concurrent user à20 drivers à $6/hour
Test Software
Summary
Summary • Build up load testing software and skills • Design tests around existing tools
– Understand risks and where to extend or introduce new tests • Iterate
Summ
ary
Questions
Thank you!
Condividi su
#AlfrescoDayRoma