Amazon S3 Advanced Features - wmich.edu · l Offsite Backup via S3 & Glacier l Onsite File Servers...
5
7/12/17 1 Amazon S3 Advanced Features l Prefixes and Delimiters - Organize and emulate hierarchical file systems. l e.g., logs/2016/January/server42.log - Used with IAM to set permissions, share, etc. l Storage Classes - S3 Standard l high durability, high availability, low latency l Short or long term, frequently accessed Amazon S3 Advanced Features l Storage Classes (cont) - S3 Standard-IA, Infrequently Accessed l As Standard for durability, and latency l Long lived, infrequently accessed data l Lower GB/month cost than Standard l Minimum size, 128KB l Minimum duration, 30 days - S3 RRS, Reduced Redundancy l Lower durability (99.99% vs. 99.9999999%) l Reduced cost l Derived or easily reproducible data Amazon S3 Advanced Features l Storage Classes (cont) - Glacier l Long term, secure, durable l No real time access, low availability l Retrieval time, serveral hours l Extremely low cost l Retrieval charged after 5%, in addition to storage
Transcript of Amazon S3 Advanced Features - wmich.edu · l Offsite Backup via S3 & Glacier l Onsite File Servers...
7/12/17
1
Amazon S3 Advanced Features
l Prefixes and Delimiters - Organize and emulate hierarchical file systems.
l e.g., logs/2016/January/server42.log
- Used with IAM to set permissions, share, etc.
l Storage Classes - S3 Standard
l high durability, high availability, low latency l Short or long term, frequently accessed
Amazon S3 Advanced Features
l Storage Classes (cont) - S3 Standard-IA, Infrequently Accessed
l As Standard for durability, and latency l Long lived, infrequently accessed data l Lower GB/month cost than Standard l Minimum size, 128KB l Minimum duration, 30 days
- S3 RRS, Reduced Redundancy l Lower durability (99.99% vs. 99.9999999%) l Reduced cost l Derived or easily reproducible data
Amazon S3 Advanced Features
l Storage Classes (cont) - Glacier
l Long term, secure, durable l No real time access, low availability l Retrieval time, serveral hours l Extremely low cost l Retrieval charged after 5%, in addition to storage
7/12/17
2
Amazon S3 Advanced Features
l Object Lifecycle Management - Automated storage tiering - Creates workflows for transitioning data from hot
frequently access to cold long term storage
l Encryption - S3 Secure Sockets Layer (SSL)
l Inflight https encryption
- S3 Server Side Encryption (SSE) l 256-bit AES
Amazon S3 Advanced Features
l Encryption Key Management - SSE-S3, AWS Managed Keys
l Every object encrypted with unique key l Key encrypted with separate master key l Master key issued monthly, rotated by AWS l Keys stored separately and secure hosts
- SSE-KMS, Customer Managed Keys l As SSE-S3, but customer manages master key l IAM permissions for master key access l Auditing, key usage and user access l Security, tracks failed access attempts
Amazon S3 Advanced Features
l Encryption Management (cont) - SSE-C, Customer Provided Keys
l Client maintains own keys l AWS does encryption/decryption
- CSE, Client Side Encryption l Encrypting data before upload to AWS
- Use AWS KMS managed master key - Use client side master key
l End to End control of encryption
7/12/17
3
Amazon S3 Advanced Features
l Versioning - Protects against malicious or accidental deletion - Objects restorable to any previous state - Turned on at bucket level - Can’t be turned off once enabled, only suspended
l MFA Delete - Requires authentication for permanent deletion - Requires authentication for changing versioning
Amazon S3 Advanced Features
l Pre-Signed URLs - Grants limited time permission to download objects
l Multipart Upload - Upload of large objects in parts - Should be used for uploads larger than 100MB - Must be used for uploads larger than 5GB - Low level API, manually split upload - High level API, automatically split upload
Amazon S3 Advanced Features
l Range GETs - Retrieve only part of an S3 or Glacier object - Useful for large objects and poor connectivity
l Cross Region Replication - Asynchronous replication - Source bucket in region A to Dest bucket region B - Requires versioning, both ends - Enable AWS replication of objects in IAM - Commonly used to reduce latency.
7/12/17
4
Amazon S3 Advanced Features
l Logging - Off by default, enabled at the bucket in S3.
l Event Notifications - Track and respond to actions taken on S3 objects - Run workflows, send alerts - Setup at bucket level - Configure through Console, REST API and/or AWS
SDK.
Amazon S3 Advanced Features
l Best Practices - Storage, Hybrid IT Environement
l Offsite Backup via S3 & Glacier l Onsite File Servers and Database Servers
- Bulk (Blob) Storage l Indexed via Amazon DynamoDB or RDS
- Higher Request Rates l Use hashing as a prefix to keys
Amazon Glacier
l Details - Extremely durable, %99.999999999 - Replacement of traditional tape solutions
l Archives - 40TB each - Unlimited number of archives. - Automatically Encrypted - Immutable, can not be changed
7/12/17
5
Amazon Glacier
l Vaults - Contain archives - 1000 vaults per account - IAM dictates access
l Vault Locks - Vault lock policy, enforces compliance - WORM, Write Once Read Many - Once locked, policy can not be changed
Amazon Glacier
l Data Retrieval - 5% of your data free per month - After, charged based on maximum rate - Data retrieval policy can limit to free
l S3 vs. Glacier - S3 max store 5TB, Glacier Archive 40TB - S3 Keys, Glacier Archive ID (auto generated) - S3 encryption optional, Glacier automatic
