Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved....
Transcript of Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved....
![Page 1: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/1.jpg)
![Page 2: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/2.jpg)
2 © 2014 Pivotal Software, Inc. All rights reserved. 2 © 2014 Pivotal Software, Inc. All rights reserved.
Apache Tomcat
Load-balancing and Clustering
Mark Thomas, 20 November 2014
![Page 3: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/3.jpg)
3 © 2014 Pivotal Software, Inc. All rights reserved.
Introduction
Apache Tomcat committer since December 2003 – [email protected]
Tomcat 8 release manager
Member of the Servlet, WebSocket and EL expert groups
Consultant Software Engineer @ Pivotal
Currently focused on Apache Tomcat 9
![Page 4: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/4.jpg)
4 © 2014 Pivotal Software, Inc. All rights reserved.
Agenda
Terminology
Reverse proxies
Load-balancing
Clustering
![Page 5: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/5.jpg)
5 © 2014 Pivotal Software, Inc. All rights reserved.
Terminology
![Page 6: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/6.jpg)
6 © 2014 Pivotal Software, Inc. All rights reserved.
Reverse Proxy
issues.apache.org
httpd instance
Jira
(issues.apache.org/jira)
Tomcat instance
Bugzilla
(issues.apache.org/bugzilla)
httpd instance
![Page 7: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/7.jpg)
7 © 2014 Pivotal Software, Inc. All rights reserved.
Reverse Proxy
Looks like a single host to the clients
Usually multiple hosts
Different services on different hosts – May also be geographically distributed
Can be used to add features – e.g. Use httpd as a reverse proxy for Tomcat to add Windows
authentication (no longer necessary)
![Page 8: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/8.jpg)
8 © 2014 Pivotal Software, Inc. All rights reserved.
Load-balancing
www.apache.org
geoip
US
www.us.apache.org
Europe
www.eu.apache.org
![Page 9: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/9.jpg)
9 © 2014 Pivotal Software, Inc. All rights reserved.
Load-balancing
Looks like a single host to the clients
Multiple hosts
Each host is the same
Each host is independent – No shared state between the hosts
– May share common services (e.g. authentication, database)
Node failure may be visible to users
![Page 10: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/10.jpg)
10 © 2014 Pivotal Software, Inc. All rights reserved.
Load-balancing
Lots of options for distributing the load – Hardware load-balancer
– Round-robin DNS
– Software load-balancer ▪ httpd
▪ pen
– geoip
![Page 11: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/11.jpg)
11 © 2014 Pivotal Software, Inc. All rights reserved.
Clustering
Load-balancer
Instance 02
Instance 01
Replicated
Session
State
![Page 12: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/12.jpg)
12 © 2014 Pivotal Software, Inc. All rights reserved.
Clustering
Load-balancing plus
Node failure is transparent to users
This transparency comes at a (usually significant) cost
![Page 13: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/13.jpg)
13 © 2014 Pivotal Software, Inc. All rights reserved.
Putting it all together
![Page 14: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/14.jpg)
14 © 2014 Pivotal Software, Inc. All rights reserved.
Reverse Proxies
![Page 15: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/15.jpg)
15 © 2014 Pivotal Software, Inc. All rights reserved.
Agenda
Protocol selection
httpd module selection
Tomcat connector implementation selection
Troubleshooting
Demonstration – Time permitting
Questions
![Page 16: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/16.jpg)
16 © 2014 Pivotal Software, Inc. All rights reserved.
Protocol Selection
Two options – AJP
– HTTP
Best choice depends on circumstances – No clear winner
Both support persistent connections – On a fast LAN or the same machine this makes little difference
![Page 17: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/17.jpg)
17 © 2014 Pivotal Software, Inc. All rights reserved.
AJP
Protocol Selection
Not a binary protocol – Common headers and values encoded
– Other values in plain text
– Request and response bodies in plain text
Request headers must fit in a single AJP message – Default 8192
– Max 65536
![Page 18: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/18.jpg)
18 © 2014 Pivotal Software, Inc. All rights reserved.
AJP
Protocol Selection
Supports passing of SSL termination information
Does not directly support encryption – IPSec, VPN, SSH tunnel, etc.
Supports ping to validate connection status
![Page 19: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/19.jpg)
19 © 2014 Pivotal Software, Inc. All rights reserved.
HTTP
Protocol Selection
Clear text protocol – Easy to read
No limit on request header size
No dedicated ping
![Page 20: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/20.jpg)
20 © 2014 Pivotal Software, Inc. All rights reserved.
HTTP
Protocol Selection
Does not directly support providing SSL termination
information – Can be added by httpd using custom headers
– Can be processed by Tomcat using the SSLValve
Supports encryption via HTTPS
![Page 21: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/21.jpg)
21 © 2014 Pivotal Software, Inc. All rights reserved.
AJP vs. HTTP
Protocol Selection
If terminating SSL at httpd and you need SSL the
information – Use AJP
If you need to encrypt the httpd to Tomcat channel – Use HTTP
![Page 22: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/22.jpg)
22 © 2014 Pivotal Software, Inc. All rights reserved.
AJP vs. HTTP
Protocol Selection
If you need both – Use HTTP
– It is (usually) easier to pass SSL information over HTTP than it is to
encrypt AJP
If you need neither – Pick the one you are more familiar with – debugging problems will
be easier
![Page 23: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/23.jpg)
23 © 2014 Pivotal Software, Inc. All rights reserved.
httpd Module Selection
Avoid – mod_jk2
– mod_jserv
– mod_webapp
– anything else not explicitly mention below
Consider – mod_jk
– mod_proxy
– (mod_rewrite)
![Page 24: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/24.jpg)
24 © 2014 Pivotal Software, Inc. All rights reserved.
mod_rewrite
httpd Module Selection
You can replace most of httpd.conf with mod_rewrite
directives
That doesn’t mean that you should
It is generally more efficient to use the dedicated directive
There are times (complex load balancing rules) where I’ve
used mod_rewrite
![Page 25: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/25.jpg)
25 © 2014 Pivotal Software, Inc. All rights reserved.
mod_rewrite
httpd Module Selection
mod_jk and mod_proxy can route based on environment
variables
Use mod_rewrite and/or mod_setenvif to determine the
routing info
Set the routing configuration with mod_jk / mod_proxy
![Page 26: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/26.jpg)
26 © 2014 Pivotal Software, Inc. All rights reserved.
mod_jk
httpd Module Selection
Only supports AJP
Developed by the Tomcat committers – More frequent releases than httpd
– Features developed in mod_jk first
Non-httpd style configuration
More complex URL mappings are simpler to write
Binaries only provided for Windows
![Page 27: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/27.jpg)
27 © 2014 Pivotal Software, Inc. All rights reserved.
mod_jk
httpd Module Selection
Doesn’t directly support URL re-writing
Make sure you are using the latest documentation – http://tomcat.apache.org/connectors-doc/
The status worker can be used for monitoring and
management
![Page 28: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/28.jpg)
28 © 2014 Pivotal Software, Inc. All rights reserved.
mod_proxy
httpd Module Selection
Supports AJP and HTTP
Included as standard with httpd
Uses httpd style configuration
More complex URL mappings are trickier to write
Built-in support for URL re-writing (not all use cases)
Binaries provided for most platforms
![Page 29: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/29.jpg)
29 © 2014 Pivotal Software, Inc. All rights reserved.
mod_jk vs/mod_proxy
httpd Module Selection
If you need the latest features – mod_jk
If you have complex mapping rules – Consider mod_jk
Not on Windows and don’t want to have to compile the
module – mod_proxy
![Page 30: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/30.jpg)
30 © 2014 Pivotal Software, Inc. All rights reserved.
mod_jk vs/mod_proxy
httpd Module Selection
If you will be load-balancing – mod_jk’s management interface is probably better
Already using one of these – Carry on
– The costs of changing will probably out-weight the benefits
If you have a free choice – Use mod_proxy, the configuration style will be more familiar
![Page 31: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/31.jpg)
31 © 2014 Pivotal Software, Inc. All rights reserved.
Tomcat Connector Selection
BIO – Default for all version to Tomcat 8
– Removed from Tomcat 9 onwards
– 100% Java Blocking IO
NIO – Default from Tomcat 8 onwards
– 100% Java non-blocking IO ▪ Waiting for next request
▪ Reading HTTP request headers
▪ SSL handshake
![Page 32: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/32.jpg)
32 © 2014 Pivotal Software, Inc. All rights reserved.
Tomcat Connector Selection
NIO2 – Introduced in Tomcat 8
– 100% Java non-blocking IO ▪ Waiting for next request
▪ Reading HTTP request headers
▪ SSL handshake
APR/native – Apache APR based native code with JNI providing non-blocking IO
▪ Waiting for next request
![Page 33: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/33.jpg)
33 © 2014 Pivotal Software, Inc. All rights reserved.
Tomcat Connector Selection
All connectors block (or simulate blocking) during – Request body read
– Response body write
SSL – BIO, NIO & NIO2 use JSSE
– APR/native uses OpenSSL
– OpenSSL is significantly faster
Sendfile – NIO, NIO2 and APR/native support sendfile
![Page 34: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/34.jpg)
34 © 2014 Pivotal Software, Inc. All rights reserved.
Tomcat Connector Selection
Comet – Will be removed in Tomcat 9
– NIO, NIO2 and APR/native support Comet
WebSocket – All connectors support WebSocket
– httpd does not support WebSocket when acting as a reverse proxy
– BIO fakes the non-blocking support
![Page 35: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/35.jpg)
35 © 2014 Pivotal Software, Inc. All rights reserved.
BIO vs. NIO vs. NIO2 vs. APR/native
Tomcat Connector Selection
If you use SSL – APR/native
Stability – NIO, BIO
Scalability – NIO, NIO2, APR/native
![Page 36: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/36.jpg)
36 © 2014 Pivotal Software, Inc. All rights reserved.
Thread Exhaustion
Troubleshooting
Need to understand threading models
httpd prefork MPM – 1 thread per process
– MaxRequestWorkers processes
– Maximum of 1 * MaxRequestWorkers threads
![Page 37: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/37.jpg)
37 © 2014 Pivotal Software, Inc. All rights reserved.
Thread Exhaustion
Troubleshooting
httpd worker MPM – ServerLimit processes
– ThreadsPerChild threads for each process
– Maximum of ServerLimit * ThreadsPerChild threads
Thread == concurrent request
![Page 38: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/38.jpg)
38 © 2014 Pivotal Software, Inc. All rights reserved.
Thread Exhaustion
Troubleshooting
Each httpd thread may create a connection to each Tomcat
instance
Therefore, 2 httpd instances each with 400 threads – Maximum of 800 connections to each Tomcat instance
– The connections are NOT distributed between the Tomcat instances
– Connections are persistent by default
![Page 39: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/39.jpg)
39 © 2014 Pivotal Software, Inc. All rights reserved.
Thread Exhaustion
Troubleshooting
Connections may have low utilization
BIO requires a thread per connection
BIO connector may run out of threads even when Tomcat is
almost idle
![Page 40: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/40.jpg)
40 © 2014 Pivotal Software, Inc. All rights reserved.
Thread Exhaustion: Solutions
Troubleshooting
Use NIO connector as it is non-blocking between requests
Don’t use persistent connections between httpd and Tomcat
Ensure each Tomcat instance has >= threads than total
httpd threads
Configure timeouts – I have seen cases where httpd tried to use a timed out connection
Use distance to create preferred groups
![Page 41: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/41.jpg)
41 © 2014 Pivotal Software, Inc. All rights reserved.
Thread Exhaustion: Example
Troubleshooting
Reverse proxy for ASF Jira had more threads than Tomcat
Didn’t take much load for Tomcat to run out of threads
No component was particularly loaded
Tomcat, Java, network I/O all blamed
5 second fix (edit maxThreads in server.xml)
(OK, and several minutes for Jira to restart)
![Page 42: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/42.jpg)
42 © 2014 Pivotal Software, Inc. All rights reserved.
Broken Links
Troubleshooting
Easiest way to create a lot of hassle for yourself – ProxyPass /foo http://localhost:10180/bar
Easiest way to avoid the hassle – ProxyPass /foo http://localhost:10180/foo
Don’t change the context path
![Page 43: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/43.jpg)
43 © 2014 Pivotal Software, Inc. All rights reserved.
Broken Links
Troubleshooting
Often marketing wants http://name.com rather than
http://name.com/app
Consider a simple redirect from / to /app – /app becomes visible to end users once they use the app
– Much easier to implement and maintain
Deploy your application as ROOT – Use ROOT##label if you need to add a version number or similar
![Page 44: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/44.jpg)
44 © 2014 Pivotal Software, Inc. All rights reserved.
Broken Links: What can go wrong
Troubleshooting
Redirects – Redirect to wrong path
Cookie paths – Cookies are not returned by client
Links – Created for wrong URL
Custom headers (e.g. Spring MVC)
![Page 45: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/45.jpg)
45 © 2014 Pivotal Software, Inc. All rights reserved.
Broken Links: Solutions
Troubleshooting
Fixing redirects – Don’t change the context path
– ProxyPathReverse will fix some but not all HTTP headers
Fixing cookie paths – Don’t change the context path
– ProxyPassReverseCookiePath /bar /foo
![Page 46: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/46.jpg)
46 © 2014 Pivotal Software, Inc. All rights reserved.
Broken Links: Solutions
Troubleshooting
Fixing links – Don’t change the context path
– mod_sed, mod_substitute, mod_proxy_html
– Fragile solution and a significant maintenance overhead
Fixing custom headers – Don’t change the context path
– mod_headers
![Page 47: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/47.jpg)
47 © 2014 Pivotal Software, Inc. All rights reserved.
Security Issues
Troubleshooting
Need to be careful when terminating HTTPS at httpd
Tomcat needs to know if request was received over HTTPS – Sessions must not transition from HTTPS to HTTP
– Cookies created over HTTPS must be marked as secure
mod_jk and mod_proxy_ajp just handle this
mod_proxy_http does not
![Page 48: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/48.jpg)
48 © 2014 Pivotal Software, Inc. All rights reserved.
Security Issues: Solutions
Troubleshooting
Custom headers and the RemoteIpValve
Two HTTP connectors – HTTP traffic proxied to connector with secure=“false”
– HTTPS traffic proxied to connector with secure=“true”
![Page 49: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/49.jpg)
49 © 2014 Pivotal Software, Inc. All rights reserved.
Miscellaneous
Troubleshooting
Virtual host selection – ProxyPreserveHost on
Client IP based security – RemoteIpValve
![Page 50: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/50.jpg)
50 © 2014 Pivotal Software, Inc. All rights reserved.
Demonstration
Time Permitting
![Page 51: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/51.jpg)
51 © 2014 Pivotal Software, Inc. All rights reserved.
Questions
![Page 52: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/52.jpg)
52 © 2014 Pivotal Software, Inc. All rights reserved.
Load-balancing
![Page 53: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/53.jpg)
53 © 2014 Pivotal Software, Inc. All rights reserved.
Agenda
Terminology
Request distribution
Managing state
Failover
Demonstration – Time permitting
Questions
![Page 54: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/54.jpg)
54 © 2014 Pivotal Software, Inc. All rights reserved.
Terminology
Sticky sessions
Without clustering, session is created only on node that
handled request
On next request, the load-balancer could send user to a
different node where the session doesn’t exist
Sticky sessions is a mechanism (there are several) that
ensures the user returns to the node holding their session
![Page 55: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/55.jpg)
55 © 2014 Pivotal Software, Inc. All rights reserved.
Request Distribution
Many ways to select node to handle request
mod_proxy – Number of requests
– Number of bytes returned
– Number of current requests
mod_jk – As mod_proxy plus
– Number of sessions (estimate)
![Page 56: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/56.jpg)
56 © 2014 Pivotal Software, Inc. All rights reserved.
Request distribution
Client IP – Last octet
Account number – Last digit 0-3, 4-6, 7-9
Customer type – More important customers get priority
![Page 57: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/57.jpg)
57 © 2014 Pivotal Software, Inc. All rights reserved.
Managing State
Stateless applications are the simple solution
Application state – State includes authentication
Options – HTTP session
– Database
– Request parameters
Load-balancing is impacted by HTTP state
![Page 58: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/58.jpg)
58 © 2014 Pivotal Software, Inc. All rights reserved.
Managing State
Sticky sessions are used for HTTP State
Session ID – Something in the session ID identifies the correct node
– Users could change this
Dedicated cookie – Users could change this
Property of client such as IP – Beware of ISP that use forward proxies
![Page 59: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/59.jpg)
59 © 2014 Pivotal Software, Inc. All rights reserved.
Managing State
Application property – Account number
– Account type
Often overlaps with load-balancing algorithm
![Page 60: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/60.jpg)
60 © 2014 Pivotal Software, Inc. All rights reserved.
Failover
Load-balancer needs to know the state of the nodes
Nodes need to taken off-line for maintenance – Known in advance
– Several options
Nodes will fail – Not (usually) predictable
– Need to be able to detect dynamically
What is the impact on users?
![Page 61: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/61.jpg)
61 © 2014 Pivotal Software, Inc. All rights reserved.
Maintenance
Failover
More transparent to users means – More complex configuration
– Process takes longer
Need to drain node of users – How long can an HTTP session last?
– At what point do you stop the node anyway?
Can Tomcat’s parallel deployment feature help?
![Page 62: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/62.jpg)
62 © 2014 Pivotal Software, Inc. All rights reserved.
Unexpected
Failover
Typically there is no separate management channel
between Tomcat instances and load-balancer – There is with mod_cluster from JBoss
Need to detect failed nodes so failover can happen as early
as possible
![Page 63: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/63.jpg)
63 © 2014 Pivotal Software, Inc. All rights reserved.
Unexpected
Failover
Can use a ‘failed’ request to detect a failed node
Is a 500 response because the server crashed or because
of an application bug?
Is a timeout because the server crashed or because it is just
a long running request?
Applications that can have long running requests take at
least that long to detect failures.
![Page 64: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/64.jpg)
64 © 2014 Pivotal Software, Inc. All rights reserved.
Unexpected
Failover
Monitoring user initiated requests to detect node failure is
fragile
Load-balancer triggered request to known, working, ‘simple’
page – More reliable
– Still an HTTP request with the associated overhead
Protocol pings are even faster
![Page 65: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/65.jpg)
65 © 2014 Pivotal Software, Inc. All rights reserved.
TODO Content
![Page 66: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/66.jpg)
66 © 2014 Pivotal Software, Inc. All rights reserved.
Demonstration
Time Permitting
![Page 67: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/67.jpg)
67 © 2014 Pivotal Software, Inc. All rights reserved.
Questions
![Page 68: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/68.jpg)
68 © 2014 Pivotal Software, Inc. All rights reserved.
Clustering
![Page 69: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/69.jpg)
69 © 2014 Pivotal Software, Inc. All rights reserved.
Agenda
When to cluster
Clustering components
Configuration choices
Debugging
Demonstration – Time permitting
Questions
![Page 70: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/70.jpg)
70 © 2014 Pivotal Software, Inc. All rights reserved.
When To Cluster
Ideally, never – Adds configuration complexity
– Requires additional processing
– Debugging is lot harder
What do you really need? – Load-balancing plus sticky sessions
– If a node fails, sessions will be lost
Clustering should be the last resort
![Page 71: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/71.jpg)
71 © 2014 Pivotal Software, Inc. All rights reserved.
Components
Manager
Channel
Valves Deployer Listeners
Membership
Receiver
Sender
Interceptors
Cluster
![Page 72: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/72.jpg)
72 © 2014 Pivotal Software, Inc. All rights reserved.
Components
Cluster – Container for all cluster related configuration
– May be placed within Engine or Host
Manager – Controls how data is replicated between nodes
Channel – Communication between cluster nodes
![Page 73: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/73.jpg)
73 © 2014 Pivotal Software, Inc. All rights reserved.
Components
Membership – Tracks which nodes joining and leaving the cluster
Sender – Sends cluster messages to other nodes
Receiver – Receives messages from other nodes
Interceptors – Valves for cluster messages
![Page 74: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/74.jpg)
74 © 2014 Pivotal Software, Inc. All rights reserved.
Components
Listeners – Receive notifications of cluster messages
– Managers support LifecycleListener
– Standard session listeners remain available
Valves – Inserted into the request processing pipeline
Deployer – Cluster wide deployment of web applications
![Page 75: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/75.jpg)
75 © 2014 Pivotal Software, Inc. All rights reserved.
Manager
Configuration Choices
Two options – Delta manager
– Backup manager
![Page 76: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/76.jpg)
76 © 2014 Pivotal Software, Inc. All rights reserved.
Manager
Configuration Choices
Delta manager – Default
– Replicates every change to every node ▪ Maximum reliability
– Network traffic proportional to the square of the number of nodes ▪ Doesn’t scale to large numbers of nodes
– Fail-over can be to any node
![Page 77: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/77.jpg)
77 © 2014 Pivotal Software, Inc. All rights reserved.
Manager
Configuration Choices
Backup manager – Sessions have a primary node and a backup node
▪ Need to use sticky sessions
– Backup node selected on a round-robin basis from all other nodes
– There is NOT a single backup node
– Every node knows the primary node and backup node for every
session
– Network traffic proportional to the number of nodes
– Failover is more complicated
![Page 78: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/78.jpg)
78 © 2014 Pivotal Software, Inc. All rights reserved.
Manager
Configuration Choices
Node A
Primary Sessions:
30*A
Backup sessions:
10*B’, 10*C’, 10*D’
Node D
Primary Sessions:
30*D
Backup sessions:
10*A’, 10*B’, 10*C’
Node B
Primary Sessions:
30*B
Backup sessions:
10*A’, 10*C’, 10*D’
Node C
Primary Sessions:
30*C
Backup sessions:
10*A’, 10*B’, 10*D’
![Page 79: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/79.jpg)
79 © 2014 Pivotal Software, Inc. All rights reserved.
Manager
Configuration Choices
Node A
Primary Sessions:
30*A
Backup sessions:
10*B’, 10*C’, 10*D’
Node D
Primary Sessions:
30*D
Backup sessions:
10*A’, 10*B’, 10*C’
Node B
Primary Sessions:
30*B
Backup sessions:
10*A’, 10*C’, 10*D’
Node C
Primary Sessions:
30*C
Backup sessions:
10*A’, 10*B’, 10*D’
![Page 80: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/80.jpg)
80 © 2014 Pivotal Software, Inc. All rights reserved.
Manager
Configuration Choices
Node D fails
Sessions will be distributed to other nodes – As soon as node failure is detected
If new node was the backup – It becomes the primary
– A new backup node is selected
– Session is copied to new backup node
![Page 81: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/81.jpg)
81 © 2014 Pivotal Software, Inc. All rights reserved.
Manager
Configuration Choices
If new node was not the backup – It becomes the primary
– The backup node remains the same
– Session is copied from the backup node
Sessions are re-distributed amongst remaining nodes
![Page 82: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/82.jpg)
82 © 2014 Pivotal Software, Inc. All rights reserved.
Manager
Configuration Choices
Node A
Primary Sessions:
40*A
Backup sessions:
20*B’, 20*C’
Node D
Primary Sessions:
30*D
Backup sessions:
10*A’, 10*B’, 10*C’
Node B
Primary Sessions:
40*B
Backup sessions:
20*A’, 20*C’
Node C
Primary Sessions:
40*C
Backup sessions:
20*A’, 20*B’
![Page 83: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/83.jpg)
83 © 2014 Pivotal Software, Inc. All rights reserved.
Membership
Configuration Choices
Two options – Multicast
– Static
![Page 84: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/84.jpg)
84 © 2014 Pivotal Software, Inc. All rights reserved.
Membership
Configuration Choices
Multicast membership – Requires multicast to be enabled on the network
– Can be difficult to debug problems
– Scales more easily
Static – Simple to debug
– Adding nodes gets time consuming as cluster grows
![Page 85: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/85.jpg)
85 © 2014 Pivotal Software, Inc. All rights reserved.
sendOptions
Configuration Choices
Delta manager – channelSendOptions on Cluster
Backup manager – mapSendOptions on Manager
Synchronous or asynchronous
![Page 86: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/86.jpg)
86 © 2014 Pivotal Software, Inc. All rights reserved.
sendOptions
Configuration Choices
Synchronous – Request processing does not complete until session data has been
sent
– What is meant by sent? ▪ On the TCP stack
▪ Received by the other node
▪ Processed by the other node
– Next request to a different node will see updated sessions
![Page 87: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/87.jpg)
87 © 2014 Pivotal Software, Inc. All rights reserved.
sendOptions
Configuration Choices
Asynchronous – Request processing continues while session data is sent
– Next request to a different node may or may not see updated
sessions
![Page 88: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/88.jpg)
88 © 2014 Pivotal Software, Inc. All rights reserved.
Summary
Configuration Choices
Manager – Delta or Backup
– (Sticky sessions)
Membership – Multicast or static
Send options – Synchronous or asynchronous
![Page 89: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/89.jpg)
89 © 2014 Pivotal Software, Inc. All rights reserved.
Cluster Configuration
Debugging
Need to know – Session ID
– Current route
– Which node handled the request
I use a simple JSP page that shows all of the above
![Page 90: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/90.jpg)
90 © 2014 Pivotal Software, Inc. All rights reserved.
Cluster Configuration
Debugging
Quickly test behaviour is as expected – Is the route correct for the current node
– Is load-balancing happening as expected
– Is fail-over happening as expected
Keep in mind how reverse proxy handles failed nodes
![Page 91: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/91.jpg)
91 © 2014 Pivotal Software, Inc. All rights reserved.
Application Problems
Debugging
Just like trying to debug any other application problem – But harder
Can the issue be replicated in a non-clustered environment?
Approach depends a lot on the application
![Page 92: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/92.jpg)
92 © 2014 Pivotal Software, Inc. All rights reserved.
Application Problems
Debugging
Network / failover issues – Look at the access logs (need session IDs)
– Look at error logs
– May need to look at network traffic
Application issues – Logging, logging and more logging
– Need to be able to fine tune logging
![Page 93: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/93.jpg)
93 © 2014 Pivotal Software, Inc. All rights reserved.
Demonstration
Time Permitting
![Page 94: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/94.jpg)
94 © 2014 Pivotal Software, Inc. All rights reserved.
Questions
![Page 95: Apache Tomcat - events.static.linuxfound.org · © 2014 Pivotal Software, Inc. All rights reserved. 2 Apache Tomcat Load-balancing and Clustering Mark Thomas, 20 November 2014](https://reader035.fdocuments.us/reader035/viewer/2022081517/5f73a6670b98f75b6e15efa4/html5/thumbnails/95.jpg)