File Upload 2015
-
Upload
choon-keat-chew -
Category
Software
-
view
3.020 -
download
0
Transcript of File Upload 2015
![Page 1: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/1.jpg)
File Upload 2015@choonkeat
choonkeat.com jollygoodcode.com
![Page 2: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/2.jpg)
How’s that predefined styles doing for you?
has_attached_file :asset, styles: { thumb: "100x100#"}
![Page 3: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/3.jpg)
How’s that predefined styles doing for you?
has_attached_file :asset, styles: { thumb: "100x100#", medium: "320x>"}
![Page 4: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/4.jpg)
How’s that predefined styles doing for you?
has_attached_file :asset, styles: { thumb: "100x100#", medium: "320x>", large: "1024x>"}
![Page 5: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/5.jpg)
How’s that predefined styles doing for you?
has_attached_file :asset, styles: { thumb: "100x100#", medium: "320x>", big: "640x>", large: "1024x>"}
Names starting to lose meaning…
![Page 6: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/6.jpg)
How’s that predefined styles doing for you?
has_attached_file :asset, styles: { thumb: "100x100#", thumb2x: "200x200#", medium: "320x>", medium2x: "640x>", big: "640x>", big2x: "1280x>", large: "1024x>", large2x: "2048x>"}
![Page 7: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/7.jpg)
How’s that predefined styles doing for you?
has_attached_file :asset, styles: { thumb: "100x100#", thumb2x: "200x200#", medium: "320x>", medium2x: "640x>", big: "640x>", big2x: "1280x>", large: "1024x>", large2x: "2048x>"}
Reprocess all the production files, each time, we make changes.
404 while rake runs or do at midnight?
![Page 8: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/8.jpg)
How’s that transformation juggling doing for you?
![Page 9: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/9.jpg)
How’s that transformation juggling doing for you?
class MyUploader < CarrierWave::Uploader::Base version :thumb do process resize_to_fill: [280, 280] end
version :small_thumb, from_version: :thumb do process resize_to_fill: [20, 20] endend
![Page 10: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/10.jpg)
How’s that transformation juggling doing for you?
class MyUploader < CarrierWave::Uploader::Base version :thumb do process resize_to_fill: [280, 280] end
version :small_thumb, from_version: :thumb do process resize_to_fill: [20, 20] endend
Did your users wait in the foreground or background?
![Page 11: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/11.jpg)
How’s that file path config doing for you?
![Page 12: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/12.jpg)
How’s that file path config doing for you?
class Avatar < ActiveRecord::Base has_attached_file :image, url: '/system/:class/:attachment/:id/:hash-:style.:extension', hash_secret: Rails.application.secrets.paperclipend
![Page 13: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/13.jpg)
How’s that file path config doing for you?
class Avatar < ActiveRecord::Base has_attached_file :image, url: '/system/:class/:attachment/:id/:hash-:style.:extension', hash_secret: Rails.application.secrets.paperclipend
You sure this is the format?
Or will they need to change?
![Page 14: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/14.jpg)
How’s that file path config doing for you?
class Avatar < ActiveRecord::Base has_attached_file :image, url: '/system/:class/:attachment/:id/:hash-:style.:extension', hash_secret: Rails.application.secrets.paperclipend
![Page 15: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/15.jpg)
How’s that file path config doing for you?
class Avatar < ActiveRecord::Base has_attached_file :image, url: '/system/:class/:attachment/:id/:hash-:style.:extension', hash_secret: Rails.application.secrets.paperclipend
![Page 16: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/16.jpg)
How’s that file path config doing for you?
CarrierWave.configure do |config| config.permissions = 0666 config.directory_permissions = 0777 config.storage = :fileend
![Page 17: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/17.jpg)
How’s that file path config doing for you?
CarrierWave.configure do |config| config.permissions = 0666 config.directory_permissions = 0777 config.storage = :fileend
Did you configure your MySQL data file in your app too?
![Page 18: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/18.jpg)
How’s that file path config doing for you?
class Avatar < ActiveRecord::Base self.table = { name: "avatars", data: "/var/lib/mysql/data/avatars.MYD", index: "/var/lib/mysql/data/avatars.MYI" }end
Did you configure your MySQL data file in your app too?
![Page 19: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/19.jpg)
How’s that form validation error dance doing for you?
![Page 20: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/20.jpg)
How’s that form validation error dance doing for you?
• User chooses a file
• Submit & wait for file to upload ⌛…
• Validation error: “Username is already taken!”
• Re-render form
![Page 21: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/21.jpg)
How’s that form validation error dance doing for you?
• User chooses a file
• Submit & wait for file to upload ⌛…
• Validation error: “Username is already taken!”
• Re-render form
Where dat file go?
![Page 22: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/22.jpg)
How’s that form validation error dance doing for you?
http://stackoverflow.com/questions/5198602/not-losing-paperclip-attachment-when-model-cannot-be-saved-due-to-validation-err
Closed: Won’t Fix
![Page 23: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/23.jpg)
How’s that form validation error dance doing for you?
http://stackoverflow.com/questions/5198602/not-losing-paperclip-attachment-when-model-cannot-be-saved-due-to-validation-err
![Page 24: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/24.jpg)
How’s that form validation error dance doing for you?
http://stackoverflow.com/questions/5198602/not-losing-paperclip-attachment-when-model-cannot-be-saved-due-to-validation-err
Answer: use CarrierWave
![Page 25: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/25.jpg)
How’s that form validation error dance doing for you?
![Page 26: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/26.jpg)
How’s that form validation error dance doing for you?
“Is there a standard approach? This seems like a very common use case.”
![Page 27: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/27.jpg)
How’s the schema pollution doing for you?
![Page 28: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/28.jpg)
How’s the schema pollution doing for you?
class StoreMetadata < ActiveRecord::Migration def change add_column :users, :profile_image_filename, :string add_column :users, :profile_image_size, :integer add_column :users, :profile_image_content_type, :string endend
![Page 29: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/29.jpg)
How’s the schema pollution doing for you?
class StoreMetadata < ActiveRecord::Migration def change add_column :users, :profile_image_filename, :string add_column :users, :profile_image_size, :integer add_column :users, :profile_image_content_type, :string endend
![Page 30: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/30.jpg)
How’s that multiple files doing for you?
![Page 31: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/31.jpg)
How’s that multiple files doing for you?
class Post < ActiveRecord::Base has_many :images, dependent: :destroyend
![Page 32: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/32.jpg)
How’s that multiple files doing for you?
class Post < ActiveRecord::Base has_many :images, dependent: :destroyend
class Image < ActiveRecord::Base belongs_to :post attachment :fileend
![Page 33: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/33.jpg)
How’s that multiple files doing for you?
class Post < ActiveRecord::Base has_many :images, dependent: :destroyend
class Image < ActiveRecord::Base belongs_to :post attachment :fileend
![Page 34: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/34.jpg)
How’s that multiple files doing for you?
class Post < ActiveRecord::Base has_many :images, dependent: :destroyend
class Image < ActiveRecord::Base belongs_to :post attachment :fileend
Is this what you want or just what you’re accustomed to?
![Page 35: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/35.jpg)
How’s Amazon Lambda doing for you?
• User chooses a file
• Submit & wait for file to upload ⌛…
• Success!
• Render page with thumbnail…
How many thumbnails - 404?
![Page 36: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/36.jpg)
How’s Amazon Lambda doing for you?
• User chooses a file
• Submit & wait for file to upload ⌛…
• Success!
• Render page with thumbnail…
Direct upload to AWS?
Cancel form submit - delete files & thumbnails?
Deep integration & assumption
![Page 37: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/37.jpg)
How’s Dragonfly doing for you?
http://markevans.github.io/dragonfly/
![Page 39: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/39.jpg)
http://thecooperreview.com/10-tricks-appear-smart-meetings/
10 Tricks to Appear Smart During Meetings
![Page 40: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/40.jpg)
Take a step back
![Page 41: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/41.jpg)
Take a step back• We want to store a bunch of attributes in a model
• e.g. Title, Body, Tags, Photo
![Page 42: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/42.jpg)
Take a step back{photo}{title}{body}
![Page 43: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/43.jpg)
Take a step back<img src={photo}><h1>{title}</h1>{body}
![Page 44: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/44.jpg)
Take a step back• Why should photo be a disproportionately
complicated attribute in my Article model?
• stored file path
• conversion
• background job
• aws config
• clean up on delete
![Page 45: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/45.jpg)
Take a step back• Why should photo be a disproportionately
complicated attribute in my Article model?
• stored file path
• conversion
• background job
• validation error dance
• aws config
![Page 46: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/46.jpg)
Take a step back• Frankly photo_url is best; least intrusive
![Page 47: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/47.jpg)
Take a step back• Frankly photo_url is best; least intrusive
• Problems
• Remote url 404? (not exclusive to your app)
• Asking users to give us a URL is a hard sell
• Need to render other sizes
• Filter by meta data
![Page 48: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/48.jpg)
Take a step back• Frankly photo_url is best; least intrusive
• Problems
• Remote url 404? (not exclusive to your app)
• Asking users to give us a URL is a hard sell
• Need to render other sizes
• Filter by meta data
![Page 49: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/49.jpg)
Take a step back• Frankly photo_url is best; least intrusive
• Problems
• Remote url 404? (not exclusive to your app)
• Asking users to give us a URL is a hard sell
• Need to render other sizes
• Filter by meta data
![Page 50: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/50.jpg)
Take a step back• Frankly photo_url is best; least intrusive
• Solutions
• Exclusive server for your app
• Upload to that server
• On-the-fly resize based on URL
• Store url with meta data: photo_json instead?
![Page 51: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/51.jpg)
Just add server
![Page 52: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/52.jpg)
• PostgreSQL, MySQL
• Redis
• Memcached
• SMTP server (Mail)
You are already
Generic server to do specialised work
Not specific to your business logic
![Page 53: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/53.jpg)
• Not a new pattern
• Mostly commercial services
• Maybe it has to be Free & Open Source to become a default pattern
Image server
![Page 54: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/54.jpg)
Want• Move the “concern” out of my app
• photo is a regular attribute
• configure my app & forget it exist
![Page 55: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/55.jpg)
Want• Move the “concern” out of my app
• photo is a regular attribute
• configure my app & forget it exist What would my app look
like in a better world?
![Page 56: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/56.jpg)
My app: Bare minimumcreate_table "users" do |t| t.string "name" t.json "avatar" t.json "photos"end
![Page 57: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/57.jpg)
My app: Bare minimumcreate_table "users" do |t| t.string "name" t.json "avatar" t.json "photos" # multiple files in a columnend
![Page 58: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/58.jpg)
My app: Bare minimumImage serverRails appBrowser
{ avatar: #<File..> }
![Page 59: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/59.jpg)
My app: Bare minimumImage serverRails appBrowser
{ avatar: #<File..> }
{“path”:“x.jpg”, “geometry”:“200x600”}
#<File..>
![Page 60: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/60.jpg)
My app: Bare minimumImage serverRails appBrowser
{“path”:“x.jpg”, “geometry”:“200x600”}
#<File..>
user.avatar={“path”: “x.jpg”…}user.save
<img src=“x.jpg”>
{ avatar: #<File..> }
![Page 61: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/61.jpg)
My app: Bare minimumImage serverRails appBrowser
{“path”:“x.jpg”, “geometry”:“200x600”}
<img src=“x.jpg”>
#<File..>
GET x.jpg
#<File..>
user.avatar={“path”: “x.jpg”…}user.save
{ avatar: #<File..> }
![Page 62: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/62.jpg)
My app: Bare minimum• Browser render <file> field; regular form submit
• Receive binary param, uploads to attache server and stores the json response instead
• Your app render <img src> requesting for image in certain size, e.g. http://example/200x/file.png
![Page 63: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/63.jpg)
Image serverRails appBrowser#<File..>
{“path”:“x.jpg”, “geometry”:“200x600”}
Progressive Enhancement
![Page 64: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/64.jpg)
Image serverRails appBrowser#<File..>
{“path”:“x.jpg”, “geometry”:“200x600”}
Progressive Enhancement
{ avatar: {“path”:“x.jpg”, … }
![Page 65: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/65.jpg)
{ avatar: {“path”:“x.jpg”, … }
Image serverRails appBrowser#<File..>
{“path”:“x.jpg”, “geometry”:“200x600”}
user.avatar={“path”: “x.jpg”…}user.save
<img src=“x.jpg”>
Progressive Enhancement
![Page 66: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/66.jpg)
• Browser render <file> field; regular form submit
• Receive binary param, uploads to attache server and stores the json response instead
• Your app render <img src> requesting for image in certain size, e.g. http://example/200x/file.png
Progressive Enhancement
![Page 67: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/67.jpg)
• Browser render <file> field; regular form submit
• Receive binary param, uploads to attache server and stores the json response instead
• Your app render <img src> requesting for image in certain size, e.g. http://example/200x/file.png
• JS upload directly to attache server; “Direct upload” in AWS parlance
• No binary in params; receive and store json attribute
Progressive Enhancement
• When after_update & after_destroy removes obsolete file from attache via delete API
![Page 68: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/68.jpg)
• Just use Ruby; just use your framework
• Pre-render multiple sizes
• fetch the urls, server will generate and cache
• Validation
• validating a regular json attribute
How do I…
![Page 69: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/69.jpg)
• Move the “concern” out of my app
• photo is a regular attribute
• configure my app & forget it exist
Want (cont’d)
![Page 70: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/70.jpg)
Want (cont’d)• Move the “concern” out of my app
• photo is a regular attribute
• configure my app & forget it exist
• Separate, standalone server
• Minimal / zero ongoing administration
![Page 71: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/71.jpg)
Want (cont’d)• Move the “concern” out of my app
• photo is a regular attribute
• configure my app & forget it exist
• Separate, standalone server
• Minimal / zero ongoing administration
How does this server work?
![Page 72: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/72.jpg)
Attache File Server• HTTP server with simple APIs
• upload
• download
• delete
• Rack app + ImageMagick
• Go? Node? C++? PHP?
• GraphicsMagick? MyResizer.bash?
![Page 73: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/73.jpg)
• Uploaded files are stored locally
• Resize local file on-the-fly
• configurable pool size to limit concurrent resizing
• Sync upload to cloud storage
• 2 hop problem vs complexity
• Fixed local storage, purge LRU (zero maintenance)
• Spin up new fresh servers anytime…
Attache File Server
![Page 74: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/74.jpg)
• When requested file does not exist locally
• fetch from cloud storage & write locally
• resume operations
Attache File Server
![Page 75: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/75.jpg)
• Remove obsolete file is “best effort”
• If photo delete failed, do you Error 500 or stop the world?
• OCDs can schedule rake task remove dangling files?
Attache File Server
![Page 76: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/76.jpg)
• Caching in production
• Browser → CDN → Varnish → Disk → Cloud
Attache File Server
![Page 78: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/78.jpg)
https://github.com/choonkeat/attache_api
ATTACHE_URL=http://localhost:9292 ATTACHE_SECRET_KEY=topsecretrake
Compatibility Check
![Page 79: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/79.jpg)
tus.io
• Open protocol for resumable uploads built on HTTP
• Perfect for mobile apps
• Rack middleware implemented in choonkeat/attache#10
![Page 80: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/80.jpg)
Responsive Images with Client Hints
http://blog.imgix.com/2015/10/13/next-generation-responsive-images-with-client.html
![Page 82: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/82.jpg)
Dragonfly & refile
![Page 83: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/83.jpg)
Dragonfly & refile
• tldr: we can fuss over implementation, but it is mostly about architecture
![Page 84: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/84.jpg)
Dragonfly & refile• Rack middleware in your Rails app by default
• performing on-the-fly image resize 😱
• Rack standalone end point
• Dragonfly.app - upload, download, delete
• Refile::App - upload, download, delete
• Downloads are unthrottled
![Page 85: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/85.jpg)
Dragonfly & refile• BEFORE: Rails integrate with AWS
• AFTER: Rails integrate with AWS + Rack app
• Maintain identical AWS config in both apps
• Rails app couldn’t shed the “concern”
• Multiple images still require multiple models
![Page 86: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/86.jpg)
refile
class Post < ActiveRecord::Base has_many :images, dependent: :destroy accepts_attachments_for :images, attachment: :fileend
class Image < ActiveRecord::Base belongs_to :post attachment :fileend
![Page 87: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/87.jpg)
refile
class Post < ActiveRecord::Base has_many :images, dependent: :destroy accepts_attachments_for :images, attachment: :fileend
class Image < ActiveRecord::Base belongs_to :post attachment :fileend
“Note it must be possible to persist images given only the associated post and a file. There must not be any other validations or constraints which prevent images from being saved”
i.e. Must be pure dummy wrapper; no validations here
![Page 88: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/88.jpg)
refile download
https://github.com/refile/refile/blob/master/lib/refile/app.rb
get "/:token/:backend/:processor/:id/:filename" do halt 404 unless download_allowed? stream_file processor.call(file)end
![Page 89: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/89.jpg)
refile download
https://github.com/refile/refile/blob/master/lib/refile/app.rb
get "/:token/:backend/:processor/:id/:filename" do halt 404 unless download_allowed? stream_file processor.call(file)end
How many ImageMagick can you run in parallel?
has_many :images?
![Page 90: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/90.jpg)
refile upload
https://github.com/refile/refile/blob/master/lib/refile/app.rb
post "/:backend" do halt 404 unless upload_allowed? tempfile = request.params.fetch("file").fetch(:tempfile) file = backend.upload(tempfile) content_type :json { id: file.id }.to_jsonend
def file file = backend.get(params[:id]) unless file.exists? log_error("Could not find attachment by id: #{params[:id]}") halt 404 end file.downloadend
![Page 91: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/91.jpg)
post "/:backend" do halt 404 unless upload_allowed? tempfile = request.params.fetch("file").fetch(:tempfile) file = backend.upload(tempfile) content_type :json { id: file.id }.to_jsonend
def file file = backend.get(params[:id]) unless file.exists? log_error("Could not find attachment by id: #{params[:id]}") halt 404 end file.downloadend
refile upload
https://github.com/refile/refile/blob/master/lib/refile/app.rb
2 hops problem when uploading and downloading
• 3mb file becomes 6mb each way
• #create and #show becomes 12mb process
• has_many :images?
![Page 92: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/92.jpg)
refile• CarrierWave-styled named processors (e.g. :fill, :thumb) vs
passing through syntax to underlying ImageMagick
• personally prefer leveraging off existing knowledge
• instead of manually configured syntax sugar
• “2 hop problem” however provide higher consistency when running multiple refile servers
• upload-here-download-there problem
• (considering to perform 2-hop upload instead of async)
![Page 93: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/93.jpg)
refile• Can upload to S3 direct and/or upload to refile
• Redundancy interesting, but prefer less concern in Rails app
• Concept of cache and store to manage “dangling file problem” is worth considering
• Download urls are signed-only (this practice should be adopted)
• can partly counter motivation to abuse “dangling file problem” (aka free file hosting, whee!)
![Page 94: File Upload 2015](https://reader030.fdocuments.us/reader030/viewer/2022021502/58efb47b1a28abb2318b45b9/html5/thumbnails/94.jpg)
Questions?Attache server https://github.com/choonkeat/attache
Gem for Rails https://github.com/choonkeat/attache_rails
Demo https://attache-demo.herokuapp.com/