Php & cloud computing
-
Upload
corley-srl -
Category
Documents
-
view
4.240 -
download
0
description
Transcript of Php & cloud computing
PHP & Cloud Computing
PHPTOSTART 2013Walter Dal Mut - @walterdalmut - @corleycloud
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Whoami
• Walter Dal Mut• Startupper
• Corley S.r.l.• http://www.corley.it/
• UpCloo Ltd.• http://www.upcloo.com/
• Electronic Engineer• Polytechnic of Turin
• Social• @walterdalmut - @corleycloud - @upcloo
• Websites• walterdalmut.com – corley.it – upcloo.com
• Corley S.r.l.• Scalable CMS
• www.xitecms.it• Load and Stress tests
• www.corley.it• Consultancy
• PHP• AWS• Distributed Systems• Scalable Systems
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
AWS - Application Management Solutions
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Why scalability
Source Jeff Barr AWS slideshare.net/…
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Also daily…
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Provision with fixed hardware
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Thanks to Cloud Computing
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Are you wasting monies?
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Scalability effects on your business
• Scale capacity on demand• Turn fixed costs into variable costs• Always available• Rock-solid reliability• Cost-effective• Reduce time to market• Focus on product and core competencies
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Distributed systems
Scalability != PerformanceScalability != High Availability
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Scalability is the ability of a systemto handle
a growing amount of workin a capable manner
orits ability to be enlarged toaccommodate that growth
Scalable PHP environments – Initial Step
Web Instance MySQL89.30.31.32/32
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Scalable PHP environments – autoscaling
Elastic Load BalancerWeb
Instance
MySQL
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Scalable PHP environments – autoscaling
Elastic Load BalancerWeb
Instance
MySQL
Web Instance
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Scalable PHP environments – autoscaling
Elastic Load BalancerWeb
Instance
MySQL
Web Instance
Web Instance MySQL
Read Replica
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
AWS (Elastic) Load Balancer
• You can distribute incoming traffic across your web instances• When it detects unhealthy load-balanced web instances, it no longer
routes traffic to those web instances and spreads the load across the remaining healthy web instances.
• Elastic?• automatically scales its request handling capacity in response to incoming
application traffic.
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Autoscaling
• Scale out web instances seamlessly and automatically when demand increases
• Replace unhealthy or unreachable instances to maintain higher availability of your applications.
• Scale dynamically based on your Cloud Watch metrics, or predictably according to a schedule that you define..
• Run on-demand or spot instances
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Metrics and Autoscaling
EC2 Instance CloudWatch
Autoscaling
Scaling rules
Elastic Load Balancer
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Metrics and Autoscaling
EC2 Instance CloudWatch
Autoscaling
Scaling rules
Alarms
Elastic Load Balancer
EC2 Instance
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Metrics and Autoscaling
EC2 Instance CloudWatch
Autoscaling
Scaling rules
Elastic Load Balancer
EC2 Instance
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Metrics and Autoscaling
EC2 Instance CloudWatch
Autoscaling
Scaling rules
Alarms
Elastic Load Balancer
EC2 Instance
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Metrics and Autoscaling
EC2 Instance CloudWatch
Autoscaling
Scaling rules
Elastic Load Balancer
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Scale with metrics, scheduled or both?• How to scale? It depends by your application
• Scheduled scale operations are useful when you have predictable spikes• Metrics when you have heavy spike application and you need to adjust your
infrastructure automatically• Mix previous in mixed scenarios
• Which metric use in order to scale?• It depends by your application design
• CPU heavy scale on CPU load metric• Network heavy scale on network load metric• Etc.
• Remember that applications don’t consume only CPU but also other resources
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
We have to discuss about:
• Session Management• If we open and close servers runtime we have to maintain PHP sessions in order
to handle user logins and other application features related to the end user session.
• Database connections• All MySQL connectors (PDO, MySQLi, [mysql_*]) handle just one connection…
No “x” RDB connection at the same time
• Software maintenance• How we can ran the same application version on all our infrastructure?
• What about logs? How we can collect VM logs in order to centralize the log management?
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Session Management
• Fortunately with PHP you can use different strategies to centralize sessions
• Memcache/Memcached (AWS ElastiCache)• Redis• DynamoDB (AWS)• Etc.
• We can modify the session_handler in order to use a different storage (php.ini, your application bootstrap, etc.)
session.save_handler = memcache
session.save_path = “tcp://1.cache.group.domain.tld:11211”
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
MySQL – Master/Slave?
• Master/Slave is a model of communication where one device or process has unidirectional control over one or more other devices
• MySQL can scales out all your read (SELECT) operations on different slave servers.
• AWS RDS (Relational Database Service) helps you to create read replicas directly in AWS web console with a couple of clicks!
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
PHP Application MySQL READ Replicas
MASTER
SLAVE 1
SLAVE 2
Application
SELECT ONLY
SELECT ONLY
INSERT/UPDATE/DELETE/SELECT
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
new PDO(“mysql:dbname=name;host=127.0.0.1”);
new mysqli(“127.0.0.1”, “user”, “pwd”, “name”);
mysql_connect(“127.0.0.1”, “user”, “pwd”, “name”);
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
How we can handledifferent READ only instances?
Can we send all write operationsonly on the Master instance?
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
MySQL Native Driver for PHP
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
MySQL Native Driver
• MySQL native driver?• Available from PHP >= 5.3• You have to compile your PHP with “mysqlnd” support
• --with-mysqli=mysqlnd --with-pdo=mysqlnd --with-mysql=mysqlnd
• Delegate to “mysqlnd_ms” the master/slave management• http://php.net/manual/en/book.mysqlnd.php
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Create the mysqlnd_ms configuration file{ "myapp": { "master": { "master_0": { "host": "localhost", "port": "3306" } }, "slave": { "slave_0": { "host": "192.168.2.27", "port": "3306" } } }}
The simple JSON configuration is divided in two main section
• Master• Slaves
“myapp” is the hostname that we use instead the real mysql host address.
Eg.• mysql_connect(“myapp”,
“user”, “passwd”);• new Mysqli(“myapp”, “user”,
“passwd”);• new
PDO(“mysql:dbname=testdb;host=myapp”);
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Maintain softwares
Elastic Load BalancerWeb
Instance
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Maintain softwares
Elastic Load BalancerWeb
Instance
Web Instance
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Maintain softwares
Elastic Load BalancerWeb
Instance
Web Instance
Web Instance
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Maintain softwares
Elastic Load Balancer
Web Instance
Web Instance
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Maintain softwares
Elastic Load Balancer
Web Instance
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
How we can stay fresh?
• If servers starts and stops continuously we have to find solutions to stay updated also on software
• When a server starts, it has to create a valid environment in order to provides web pages. Strategies?
• Compile and bundle all softwares in one instance image• It is very simple but all software becomes old very quickly and when you have to release an
update you have to compile a new image and update all load balancers configurations. It is a long and complex operation
• Use EC2_USER_DATA feature provided by AWS• You can run a shell script when your instances bootstraps. It is more flexible because you can
create a skeleton (PHP + libraries) and download all software runtime during the boot operation
• Typically you will use both in conjunction
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Cool Up/Down – Inhibits alarms
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Scales in action
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Run commands on multiple servers
• If you ran many servers execute commands could be hard. You can use tools to run commands on a server list
• Capistrano (Ruby)• https://github.com/capistrano/capistrano
• Fabric (Python)• https://github.com/fabric/fabric• Use CLOTH for AWS EC2 instances
• https://github.com/garethr/cloth
• Chef• Puppet• Etc…
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Fabric is very simple
#! /usr/bin/env python
from __future__ import with_statementfrom fabric.api import *
from fabric.contrib.console import confirm from cloth.tasks import * env.user = "root"env.directory = '/mnt/wordpress'env.key_filename = ['/home/walter/Amazon/wp-cms.pem'] @taskdef reload(): "Reload Apache configuration"
run('/etc/init.d/apache2 reload') @taskdef tail(): "Tail Apache logs"
run('tail /var/log/syslog')
EC2 instances are dynamic with don’t know addresses, for that reason we can use tagging system to execute commands on a group of instances
fab nodes:"^production.*" tail
Execute the “tail” command on all instances with a name that starts with “production.”
Eg.• production.web-1• production.log• production.mongodb
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Centralize all logs
• We create and destroy instances thanks to alarms but when we close an instance we lose immediately all instance/application logs
• How we can manage logs?• The simplest way is to use Rsyslog clusters
• Rsyslog is an opensource software that forwarding log messages in an IP network
• Rsyslog implement the basic syslog protol• That means that we can configure apache logs to “syslog” instead using
normal text files.• In this way we can collect all logs in one group of VM and work on these files
later thanks to other technologies
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Log management
• Collecting logs is not the latest operation because you have to analyse and reduce information• Move logs to S3 bucket – Time based• Analyze logs with Hadoop
• Map Reduce on the cloud with Elastic Map Reduce service (EMR)• Use script languages on top of Hadoop in order to simply the log analysis
• HIVE – Data Warehouse infrastructure (data summarization)• Pig – High level platform for creating MapReduce program
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Cloud Conference 2013 – 18 April
• www.cloudconf.it• @_CloudConf_• https://www.facebook.com/pages/Cloud-Conf-Italia/140265592804313?ref=hl
• Data analysis• Hadoop – MapReduce Framework• Apache Pig – MapReduce Scripting Language for Hadoop• Apache Hive – Data Warehouse System for Hadoop
PUG User Coupon Code:
pug-user
Walter Dal Mut [email protected] @walterdalmut @corleycloud @upcloo
Thanks for listening
Any questions?