Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All...
Transcript of Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All...
![Page 1: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/1.jpg)
Lars Noldan, Systems Administrator
Speed DaemonsBSDCAN 2011
![Page 2: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/2.jpg)
BSDCan, 2011Who am I?
• e-mail: [email protected]
• IRC: LarsN on Freenode
• Phone: 317-861-5948 x609
• Twitter: @Absenth
• Lars Noldan
• Six Feet Up
• Systems Administrator
![Page 3: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/3.jpg)
BSDCan, 2011
What Requirements?Every Project Starts With Requirements
![Page 4: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/4.jpg)
BSDCan, 2011Technical Requirements• Site Must Maintain 99.9% Uptime.
• Pages Load in 5-10 Seconds on Broadband Connections.
• Process User Registration in 15-20 Seconds on Broadband Connections.
• Must Support Between 5,000 and 50,000 Registered Users.
• Must Integrate With Existing Systems
![Page 5: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/5.jpg)
BSDCan, 2011Software Requirements• Zope / Plone (Enterprise CMS)
• With RelStorage (MySQL Backed)
• Solr (Powered By Apache Lucene)
• Apache Tomcat Engine
![Page 6: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/6.jpg)
BSDCan, 2011
Open Source Solutions!Using FOSS To Solve The Puzzle
![Page 7: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/7.jpg)
BSDCan, 2011Web Proxy Chain• Nginx - Web Server
• Light Weight
• Extremely Fast
• Varnish - Reverse Proxy Cache
• High Performance
• Extremely Flexible
• HAProxy - Load Balancer
• Session Aware
• Configurable
![Page 8: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/8.jpg)
BSDCan, 2011Application Servers
• Python 2.4
• Zope 2.10 - Python Application Server
• Plone 3.3.5 - Enterprise Content Management
• Apache Tomcat - Java Application Server
• Solr 1.4 - Enterprise Search Software
![Page 9: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/9.jpg)
BSDCan, 2011Database Servers
• MySQL 5.1 - Database Server
![Page 10: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/10.jpg)
BSDCan, 2011
Network TechnologiesCan’t Build a Cluster Without a Network
![Page 11: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/11.jpg)
BSDCan, 2011Network Technologies• FreeBSD 8.1-Release AMD64
• PF - Enterprise Grade, FOSS Firewall
• CARP - Common Address Redundancy Protocol
• LAGG - Link Aggregation via LACP
• ifstated - Network Interface State Engine
![Page 12: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/12.jpg)
BSDCan, 2011
Speed Daemons!High Performance Clusters
![Page 13: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/13.jpg)
BSDCan, 2011Simplified Cluster
Proxy01
![Page 14: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/14.jpg)
BSDCan, 2011Simplified Cluster
Proxy01
Application01
![Page 15: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/15.jpg)
BSDCan, 2011Simplified Cluster
Proxy01
Application01
Database01
![Page 16: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/16.jpg)
BSDCan, 2011Simplified Cluster
Proxy01
Application01
Database01
Nginx - 80/443Varnish - 3180HAProxy - 3380
![Page 17: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/17.jpg)
BSDCan, 2011Simplified Cluster
Proxy01
Application01
Database01
Nginx - 80/443Varnish - 3180HAProxy - 3380
Zope - 8080-8088Solr - 6080
![Page 18: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/18.jpg)
BSDCan, 2011Simplified Cluster
Proxy01
Application01
Database01
Nginx - 80/443Varnish - 3180HAProxy - 3380
Zope - 8080-8088Solr - 6080
MySQL - 3306
![Page 19: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/19.jpg)
BSDCan, 2011
Somebody Order a Double?
Let’s Kick This Cluster Into High Gear
![Page 20: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/20.jpg)
BSDCan, 2011Expanded Cluster
Proxy01
Application01
Database01
Proxy02
![Page 21: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/21.jpg)
BSDCan, 2011Expanded Cluster
Proxy01
Application01
Database01
Proxy02
Application02
![Page 22: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/22.jpg)
BSDCan, 2011Expanded Cluster
Proxy01
Application01
Database01
Proxy02
Application02
Database02
![Page 23: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/23.jpg)
BSDCan, 2011
Wire It UpA Little Cable Never Hurt Anyone. Right?
![Page 24: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/24.jpg)
BSDCan, 2011Proxy Servers
Proxy01 Proxy02
![Page 25: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/25.jpg)
BSDCan, 2011Proxy Servers
Proxy01 Proxy02
12.12.12.3 12.12.12.212.12.12.1
•Each Proxy Server Has a Unique Public IP Address.•Both Proxy Servers Share a Public IP Address via CARP
![Page 26: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/26.jpg)
BSDCan, 2011Proxy Servers
Proxy01 Proxy02
12.12.12.3 12.12.12.212.12.12.1
•ifstated - Runs On Both Servers•Both Servers Constantly Monitor Each Other
![Page 27: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/27.jpg)
BSDCan, 2011Proxy Servers
Proxy01 Proxy02
12.12.12.3 12.12.12.212.12.12.1
•In The Event Of Any Service Interruption ifstated Forces CARP To Fail Over Making Proxy02 Primary•This Works Bi-Directionally
![Page 28: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/28.jpg)
BSDCan, 2011Proxy Servers
Proxy01 Proxy02
12.12.12.3 12.12.12.212.12.12.1
•pfsync Runs On Both Proxy Servers. •This Keeps The Statefull Firewalll In Sync Between The Two Servers
![Page 29: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/29.jpg)
BSDCan, 2011Application Servers
Application01 Application02
![Page 30: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/30.jpg)
BSDCan, 2011Application Servers
Application01 Application02
10.1.0.3 10.1.0.210.1.0.1
•These Servers Only Have Private Addresses•Zope Listens On All Addresses•Solr is Configured To Listen On The Shared CARP Address
![Page 31: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/31.jpg)
BSDCan, 2011Application Servers
Application01 Application02
10.1.0.3 10.1.0.210.1.0.1
•These Servers Share Solr Search Data•ifstated - Does Not Monitor These Servers
![Page 32: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/32.jpg)
BSDCan, 2011Database Servers
Database01 Database02
![Page 33: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/33.jpg)
BSDCan, 2011Database Servers
Database01 Database02
10.1.0.13 10.1.0.1210.1.0.11
•These Servers Only Have Private Addresses•MySQL Listens On All Addresses•PF Is Used To Limit Access To MySQL
![Page 34: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/34.jpg)
BSDCan, 2011Database Servers
Database01 Database02
10.1.0.13 10.1.0.1210.1.0.11
•ifstated - Runs Custom Scripts To Facilitate MySQL Master/Slave Replication State Changes•Data Is Kept In Sync By MySQL Replication
![Page 35: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/35.jpg)
BSDCan, 2011
Data StorageOur Solutions For Storing and Shipping Data
![Page 36: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/36.jpg)
BSDCan, 2011Storage SolutionsProxy01
Application01
Database01
Proxy02
Application02
Database02
Stor01 Stor02
![Page 37: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/37.jpg)
BSDCan, 2011Storage SolutionsProxy01
Application01
Database01
Proxy02
Application02
Database02
Stor01 Stor02
•Hardware:•LSI MegaRaid - Raid Card Using JBOD•2 OS Drives - Mirror•2 Intel SSD Drives - Mirrored ZIL•7 1TB Spindles - RaidZ2 (5TB)
![Page 38: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/38.jpg)
BSDCan, 2011Storage SolutionsProxy01
Application01
Database01
Proxy02
Application02
Database02
Stor01 Stor02
•Configuration v1.0:•FreeBSD 8.0-Release•Several File Backed iSCSI Mounts•Numerous NFS Shares•Snapshots At 5m, 15m, 1h•Stor01 Primary•Stor02 Replication Target
![Page 39: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/39.jpg)
BSDCan, 2011Storage SolutionsProxy01
Application01
Database01
Proxy02
Application02
Database02
Stor01 Stor02
•Problems Encountered:•iSCSI Performance Abysmal•Kernel Race Condition During Snapshot Replication
![Page 40: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/40.jpg)
BSDCan, 2011Storage SolutionsProxy01
Application01
Database01
Proxy02
Application02
Database02
Stor01 Stor02
•Configuration v2.0:•OpenSolaris - 2008.11•Several File Backed iSCSI Mounts•Numerous NFS Shares•Snapshots At 5m, 15m, 1h•Stor01 Primary•Stor02 Replication Target
![Page 41: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/41.jpg)
BSDCan, 2011Storage SolutionsProxy01
Application01
Database01
Proxy02
Application02
Database02
Stor01 Stor02
•Future Configuration v3.0:•FreeBSD 9.0-Release•Several File Backed iSCSI Mounts•Numerous NFS Shares•Snapshots At 5m, 15m, 1h•Stor01 Primary•Stor02 Replication Target
![Page 42: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/42.jpg)
BSDCan, 2011Storage SolutionsProxy01
Application01
Database01
Proxy02
Application02
Database02
Stor01 Stor02
•Future Configuration v3.0:•iet - iSCSI Target Replacement•zpool - v28•Kernel Race Conditions Fixed in 8.2
![Page 43: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/43.jpg)
BSDCan, 2011
Does It Work?A Quick Review
![Page 44: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/44.jpg)
BSDCan, 2011Quick Review
• Customer Requirements Have Been Met
• Over 1.5 Years Of Hosting 99.995% Uptime
• High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions
• FreeBSD Rocks!
![Page 45: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/45.jpg)
BSDCan, 2011Who is Six Feet Up?
• Web Development & Hosting Provider
• Focus on CMS, KMS, and Web Apps
• Located in Fortville, IN
• Major Player in Open Source Technologies such as:
• Plone (CMS) - http://www.plone.org
• KARL (KMS) - http://www.karlproject.org
![Page 46: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/46.jpg)
BSDCan, 2011Where Can You Find Us?
• HTTP://www.sixfeetup.com
• Twitter: @sixfeetup
• Conferences and Events
• IndyPy - Indianapolis Meetup Group
• PyCon - Atlanta, GA
• BSDCan - Ottawa, Canada
• KMWorld - Washington, DC
• Plone Symposium East - State College, PA
• Plone Conference 2011, San Francisco, CA
• OSCON - Portland, OR
• Ohio Linux Fest - Columbus, OH
![Page 47: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/47.jpg)
BSDCan, 2011
Want To Work With Me?
http://www.sixfeetup.com/about-us/jobsor
E-Mail a Resume To: [email protected]
![Page 48: Speed Daemons€¦ · • High Performance / High Availability Cluster Can Be Built Using All OpenSource Solutions • FreeBSD Rocks!](https://reader035.fdocuments.us/reader035/viewer/2022071515/613761620ad5d206764894b6/html5/thumbnails/48.jpg)
Check out
sixfeetup.com/demos