Journey Through the Cloud - Digital Media
-
Upload
amazon-web-services -
Category
Technology
-
view
492 -
download
1
description
Transcript of Journey Through the Cloud - Digital Media
Journey through the Cloud:
Digital Media
Ian Massingham – Technical Evangelist @IanMmmm
Common use cases & stepping stones into the AWS cloud Learning from customer journeys
Best practices to bootstrap your projects
Journey through the cloud
Amazon Web Services provides a set of secure and on-demand storage, archive, computing, transcoding, streaming, and
application services for media companies
Digital Media
Why AWS for Digital Media Applications? Media Ingest Media Storage & Archive Media Content Delivery Useful Resources
Agenda
Why AWS for Digital Media Applications?
Customers Using AWS for Digital Media Applications
Global Infrastructure
Fundamental Characteristics of the AWS Cloud
Convert CAPEX to OPEX
Lower Total Cost
No Need to Guess Capacity
No Undifferentiated Heavy Lifting
Security is Our No.1 Priority Comprehensive Security Capabilities to Support Virtually Any Workload
People & Procedures
Platform Security
Network Security
Physical Security
Validated by security experts
Collaboration on Enhancements Every Customer Benefits
AWS Services Stack in a Media Workflow
AWS Direct Connect
Elastic Load!Balancing!
AWS Import/ Export
Amazon S3
AWS Storage Gateway
Amazon Glacier!
Amazon EBS
Amazon!CloudFront!
Amazon CloudSearch!
Amazon SNS!
Amazon SQS!
Amazon SWF!
Amazon Elastic
Transcoder!
Amazon EC2! Amazon
EMR!
Amazon!DynamoDB!
Amazon VPC!
Amazon RDS!
Amazon Redshift
Amazon ElastiCache
AMI!
Amazon!Route 53!
Ingest Store Deliver Process
Amazon EC2!
Media Ingest
AWS Direct Connect Dedicated bandwidth between your
site and AWS
AWS Storage Gateway On-premises storage federation with
Amazon S3 and Amazon Glacier
AWS Import/Export Physical transfer of media into
and out of AWS
AWS Ingest Options
AWS Ingest Options – Parallel Uploads
1. Multipart upload!2. Request rate optimization!3. TCP window scaling!4. TCP selective acknowledgement!
AWS has customers that ingest roughly 1 PB per day!
AWS Ingest Options – Direct Connect
• Reduces costs for bandwidth-heavy workloads!
• Private connectivity to AWS!• Physical connection – 1 Gbps or
10 Gbps port!• Logical connections (802.1q
VLANs) !• Public: To AWS cloud (Amazon
EC2, Amazon S3 etc.)!• Private: To VPCs!
• Consistent network performance!• Compatible with all AWS services!
aws.amazon.com/directconnect/!
• Rapidly move data into and out of AWS!
• Portable storage device shipment to AWS!
• eSATA!• USB 2.0 and 3.0 !• 2.5 and 3.5 inch internal SATA
hard drives!• Supports!
• Amazon Elastic Block Store (EBS)!• Amazon Simple Storage Service
(S3)!• Amazon Glacier!
AWS Ingest Options – AWS Import/Export
aws.amazon.com/importexport/!
• On-premises, virtual iSCSI storage appliance!
• Local cache enables low latency access to data!
• Gateway – stored volumes!• Gateway – cached volumes!
• Copies data in the form of Amazon EBS snapshots to Amazon S3!
• Leverage Amazon S3 server-side encryption!
• Recover to Amazon EBS / Amazon EC2!
AWS Ingest Options – AWS Storage Gateway
aws.amazon.com/storagegateway/!
Aspera Up to 1Gb/s per instance to AWS
Signiant High-speed, network-efficient file
transfer –up to 200X faster than FTP with 95+% network efficiency
CloudBeam SaaS-based file transfer into and
out of AWS
Partner Ingest Options
Moving Big Data into the Cloud with Tsunami UDP
Media Storage & Archive
Amazon Elastic Block Store (EBS
Amazon Glacier Amazon Simple Storage Service (S3)
AWS Storage and Archive Options
Long-term object archive Extremely low cost per gigabyte
99.999999999% durability
Highly scalable object storage 1 byte to 5 TB in size
99.999999999% durability
High-performance block storage device 1 GB to 1 TB in size
Mount as drives to instances with snapshot/cloning functionalities
Very Fast ‘disks’ Fast web object storage Slow, rare access
AWS Storage Options Amazon Elastic Block Store (EBS)
• High I/O block storage for Amazon EC2!• Predictably scale to 1000s of IOPS per
Amazon EC2 instance !• Automatic replication within the Availability
Zone!• 10x more reliable than commodity disk
drives!• Point-in-time snapshots!
• Amazon S3 durability (11-9s) !• Point-in-time snapshots across regions!
aws.amazon.com/ebs/!
AWS Storage Options Amazon Simple Storage Service (S3)
• Designed for 99.999999999% durability!• Authentication & authorisation mechanisms
ensure that data is kept secure!• Multiple encryption options!• Standard & Reduced redundancy storage!
aws.amazon.com/s3/!
Amazon Simple Storage Service (S3) Security Controls
• Bucket- and object-level permissions!• Owner only access (by default)!
• Signed URLs/query string authentication!• IAM policies!• Versioning (MFA delete)!• Detailed access logging!
aws.amazon.com/s3/!
✔Access Logs!
AWS Archive Options Amazon Glacier
• $0.01 - GB per month!• Retrievals:!
• 5% of monthly average storage (pro-rated daily) free!
• Synchronous in!• 3–5 hour asynchronous retrieval!• Designed for 99.999999999% durability!• AES 256 encryption at rest!• Highly scalable and reliable!• Authentication & authorisation mechanisms
ensure data is kept secure!
aws.amazon.com/glacier/!
AWS Archive Options Object Lifecycle Management
• Seamlessly move data from !Amazon S3 → Amazon Glacier!
• 3-5 hour asynchronous retrieval!• Data lifecycle policies!• $0.01 per GB for Amazon Glacier costs!
Media Content Delivery
Using AWS for Media Streaming
• Native infrastructure services (Amazon S3, Amazon Elastic Transcoder, Amazon CloudFront, Amazon EC2)
• Partner solutions in AWS Marketplace
Well Formed Ecosystem
• Pay-as-you-go pricing model • Capacity when you need it • Global footprint
Cost Effective Scaling
• MPAA security best practices Media Specific Capabilities
Encode to Take Advantage of Commonality Amongst Devices
Ingest!
Amazon S3! Amazon Elastic Transcoder!
Lowest number of encodes that span maximum number of devices H.264 video, AAC audio Low quality, high quality & 720p HD compression
Video Streaming with Amazon CloudFront
• All HTTP-based streaming protocols (HLS, HDS, Smooth Streaming, MPEG-DASH) supported!
• Video streaming-specific caching optimizations!
• Scalable to handle popular content!• Global reach (50+ edge locations)!• Native streaming support for HLS!• Joint solutions with streaming media ISVs!• Granular customer access logs!
aws.amazon.com/cloudfront/!
Amazon CloudFront Security Controls
• CloudFront’s private content feature!• Only deliver content to securely signed
requests !
• HTTPS ONLY requests/delivery!• CloudFront origin access identity!• Signed URL verification!
• Policy based on a timed URL or a CIDR block of the requestor!
• HTTPS ONLY origin fetches!• Trusted signers!• Access logs ! Amazon S3
(Media Storage)
Amazon CloudFront
End User
HTTP
________
HTTPS ONLY
Delivery EC2 Instances
Security Group
Signed Request
Amazon S3 (Logs Storage)
Amazon CloudFront Media Streaming Tutorials
aws.amazon.com/cloudfront/streaming/!
Monitor and Measure Customer Experience
CloudFront !Access Logs!
Amazon S3 Bucket!
Amazon Elastic MapReduce!
Maximizing Audience Engagement
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Maximizing Audience Engagement in Media Delivery
Usman Shakeel, Amazon Web Services Shobana Radhakrishnan, Engineering Manager at Netflix
November 14, 2013
The Challenge
Mountains of Raw Data
Real-time Processing Dashboards/ User Personalization/
User Experience
Back-end Processing Storage/DWH
Ingest & Stream
Processing
Amazon EMR Amazon Redshift
Batch
Real-ti
me
The Solution
Mountains of Raw Data
Real-time Processing
Back-end Processing Storage/DWH
Ingest & Stream Amazon S3
SQS Amazon DynamoDB
Kafka Canal
Audience Engagement
Amazon EMR Amazon Redshift Storm Spark
Amazon S3 Amazon RDS Amazon DynamoDB Amazon RedShift
EMR
Maximizing Audience Engagement in Media Delivery (MED303) from AWS re:Invent 2013 !
Where to go to learn more
Intro to Digital Media in the AWS Cloud »
Streaming from the AWS Cloud »
Storage and Archive in the AWS Cloud »
Encoding in the AWS Cloud »
AWS Case Study: Netflix »
Scalable Media Processing in the Cloud »
Resources
AWS Training & Certification Certification
aws.amazon.com/certification
Demonstrate your skills, knowledge, and expertise
with the AWS platform
Self-Paced Labs
aws.amazon.com/training/ self-paced-labs
Try products, gain new skills, and get hands-on
practice working with AWS technologies
aws.amazon.com/training
Training
Skill up and gain confidence to design, develop, deploy and
manage your applications on AWS
http://reinvent.awsevents.com/livestream-reg.html!
Ian Massingham – Technical Evangelist @IanMmmm
@AWS_UKI for local AWS events & news
@AWScloud for Global AWS News and Announcements ©Amazon.com, Inc. and its affiliates. All rights reserved.
We typically see customers start by trying our services
Get started now at : aws.amazon.com/getting-started
You can get started for free…
Get started now at : aws.amazon.com/free
Design your application for the AWS Cloud
More details on the AWS Architecture Center at : aws.amazon.com/architecture