Use Somebody Else's Infrastructure - Utilizing Amazon S3 and EC2
-
Upload
jonathan-weiss -
Category
Technology
-
view
11.175 -
download
0
description
Transcript of Use Somebody Else's Infrastructure - Utilizing Amazon S3 and EC2
Use Somebody Else's Infrastructure:
Utilizing Amazon S3 and EC2
Jonathan Weiss
Peritor Wissensmanagement GmbH, www.peritor.comWeb 2.0 Expo Berlin, 2007
2
Peritor
• Consultancy from Berlin, Germany
• Specialized on Architecture, Scaling andRuby on Rails
• A lot of our clients use Amazon WebServices
3
www.peritormail.com
We run a Webmail portal on top of Amazon Web Services
4
Starting Point
One machine:
• Apache
• Ruby on Rails / PHP / Perl / Java / …
• MySQL
5
Worst Case Populariy
6
A Difficult Path
7
Problem: Backup
• High availability
• Redundancy
• Very big data sets
8
Problem: File System
• Important files have to be accessed by many servers
• NFS / Samba not practical
9
Problem: Messaging
• Many servers on different locations have to communicate
• Creating a reliable and scalable message queue is really hard
10
Problem: Spontaneous Traffic
11
Problem: Load Fluctuation
12
Don‘t reinvent the wheel!
13
Amazon Web Services
Simple Storage Service - S3
Redundant Data Store
$0,15 per GB data permonth
$0,10 - $0,20 per GBtraffic
Virtual server per hour
$0,10 per CPU hour
$0,10 - $0,20 per GBtraffic
Elastic Compute Cloud - EC2
…….
E-Commerce..
Reliable Message System
$0,10 per 1,000 messages
$0,10 - $0,18 per GBtraffic
Simple Queue Service - SQS
14
S3 - Simple Storage Service
• Redundant storage - as much as you like
• max. 5 GB per object
• Organized in „Buckets“
• Web Service API for uploads
• Downloads via
• Web Service
• HTTP / HTTPS
• BitTorrent
15
S3 - Buckets
• Unique over all S3
• Contains manykey-value-metadata tupel
• Cannot contain other buckets!
• Key can contain „/“
MyBucket_name
S3
foto_5.jpg
backups/januar.zip
fotos/2007/001.png
www.peritor.com
site/screen.css
site/script.js
FreeBSD-6.2.iso
16
S3 with AWS::S3Upload
Download
17
EC2 - Elastic Compute Cloud
• Based on XEN virtualization
• On demand virtual servers - controlled with Web Service API
• Use your favorite Linux distro (Linux 2.6.16),Amazon Machine Images (AMI) are stored on S3
• ACLs for hosts/ports access control
18
EC2 ToolsList availabe images
Start a new instance
Login with SSH
Shutdown instance
19
amazon-ec2 gemSetup
Usage
20
SQS - Simple Queue Service
• Distributed, highly scalable and reliable message system
• Unlimited number of queues and messages
• Locking / releasing built-in
• 256 KB message payload
• Basic permission model
21
SQS gemSetup
Usage
22
How does this solvemy problems?
23
S3 - Backup
• s3sync.rb
• Brackup
• Jungle Disk
• S3 FUSE
• s3DAV
• Duplicity
• S3Browser
• Firefox S3 Organizer extension
• …
24
s3sync.rb
Backup
Restore
25
S3 Asset Host
26
S3 Asset Host
Setup DNS
Rails configuration
27
S3 Asset Host
welcome.rhtml template
Output
28
S3 - Authenticated User Data
29
attachment_fu Rails plugin
Setup
30
attachment_fu Rails plugin
Upload
Download
31
On-Demand Computing with EC2
Time based, e.g. with cron
32
On-Demand Computing with EC2
Load based, e.g. with Monit
33
On-Demand Computing with EC2
34
EC2 for extra capacity
35
Peritor Wissensmanagement GmbH
Lenbachstrasse 212157 Berlin
Internet: www.peritor.comEmail: [email protected]
© Peritor Wissensmanagement GmbH - All Rights Reserved
Phone: +49 (0)30 69 40 11 94Fax: +49 (0)30 69 40 11 95