(originally posted at:
http://casey0.com/archive/2006/October/How_to_serve_the_rails_public_directory_out_of_S3.html)
Even if your Apache rewrite rules serve up your rails /public directory without hitting the mongrel (or whatever) servers, serving up all the javascript, stylesheet, and image files use up apache threads and bandwidth to your web application box.
Ruby on Rails has a great configuration option, config.action_controller.asset_host, that allows you to push all your assets onto a different machine, but for most projects, it’s hard to justify renting another server at first.
Enter Amazon’s S3 service. You pay $0.15 per GB per month of storage used, and $0.20 per GB of data transferred. So when you’re developing, you’ll be paying pennies a month, and when you get huge, you’re fast and happy.
With S3, you can make it so each time a user loads a page, only a single request hits your server, which is pretty remarkable. Everything else stays between the client and Amazon’s data centers, which are designed for speed and reliability.
Here’s how you can do it.
Sign up for S3 and get your keys
Pretty straightforward: http://aws.amazon.com/s3
Get S3.rb and put it in your rails /lib directory
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=135&categoryID=47
Technically all you need is some way of creating a bucket and sending files to rails, but if you’re writing a ruby on rails app, you’re probably pretty comfortable in ruby.
Choose your hostname for the static assets
Personal preference, for instance: assets.example.com or static.example.com.
If you are unable to create a CNAME in your domain (get a real DNS provider), you will need to choose something.s3.amazonaws.com
instead.
I’ll use assets.example.com for the rest of the steps.
Create your S3 bucket
You need to make an S3 bucket with your asset hostname as the key. I just used script/console, something like this:
>> require ‘S3’
>> AWS_ACCESS_KEY = ‘’
>> AWS_SECRET_ACCESS_KEY = ‘’
>> BUCKET_NAME = ‘assets.example.com’
>> conn = S3::AWSAuthConnection.new(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, false)
>> conn.create_bucket(BUCKET_NAME)
If you’re using whatever.s3.amazonaws.com, the bucket name will be just the whatever, and needs to be all lowercase.
I like to just leave that connection open with BUCKET_NAME defined for ease of debugging. S3.rb has operations to list your buckets, see what’s in your bucket, etc. Remember to look at the response body if there are any errors.
Point your host at S3
What you need to do is create a CNAME for assets.example.com that points to s3.amazonaws.com. This allows us to use S3’s Virtual Hosting of Buckets:
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/VirtualHosting.html
Skip this step if you’re using something.s3.amazonaws.com.
Upload a test file
You can skip this step if you like, but if you haven’t used S3 before, it might be worth putting something up there and making sure it all works.
Choose an image in your public directory, and go back to the script/console where you’ve already created conn.
Pretending you chose public/images/menu.png, try this:
>> datafile = File.open(‘public/images/menu.png’)
>> conn.put(BUCKET_NAME, ‘images/menu.png’, datafile.read,
{ “Content-Type” => ‘image/png’, “Content-Length” => File.size(‘public/images/menu.png’).to_s,
“Content-Disposition”=> “inline;filename=menu.png”,
“x-amz-acl” => “public-read” })
If that worked, you should be able to now go to
http://assets.example.com/images/menu.png and see the image.
Create an easy uploading script
I wanted something easy that I could use to send new and updated files to S3, so I created script/s3commit. There are probably better (and
more ruby on railsy) ways to set this kind of thing up.
My script/s3commit looks something like this:
#!/usr/bin/env ruby
require File.dirname(__FILE__) + ‘/../config/boot’
PUBLICDIR = File.expand_path(“public”, RAILS_ROOT)
require ‘mime/types’
require ‘S3’
AWS_ACCESS_KEY_ID = ‘’
AWS_SECRET_ACCESS_KEY = ‘’
BUCKET_NAME = ‘assets.example.com’
MIME::Type.new(‘application/x-javascript’) do |t|
t.extensions = ‘js’
MIME::Types.add(t)
end
def upload_asset(path)
if File.directory?(path)
# go recursive
Dir.foreach(path) {|file|
if /[\.].*$/.match(file)
upload_asset(“#{ path }/#{ file }”)
end
}
else
# it’s a file, check for validity and upload it
if /#{ PUBLICDIR }\/(.+[~])$/.match(path) && File.readable?(path)
key = Regexp.last_match1
mime = MIME::Types.type_for(key).to_s
if mime.length == 0
mime = ‘text/plain’
end
puts “uploading #{ path } as #{ key } mime #{ mime }”
datafile = File.open(path)
conn = S3::AWSAuthConnection.new(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, false)
conn.put(BUCKET_NAME, key, datafile.read,
{ “Content-Type” => mime, “Content-Length” => File.size(path).to_s,
“Content-Disposition”=> “inline;filename=#{File.basename(key).gsub(/\s/, ’_’)}”,
“x-amz-acl” => “public-read” })
end
end
end
if ARGV.length > 0
# some files were selected
for arg in ARGV
upload_asset(File.expand_path(arg))
end
else
# no files were specified, try to upload everything public
upload_asset(PUBLICDIR)
end
Running script/s3commit by itself will re-upload everything reasonable in public to S3. Running it on one or more files or directories will upload only those, recursively.
If something like this would be better done with a rake or capistrano action or anything, I’d love to hear it.
This could be tied into SVN, but don’t forget that you probably want it to happen as a sort of “update hook”, not a commit hook. Otherwise
if you commit (for example) new javascript and HTML that interacts, your users may see the new javascript before you get a chance to update production with the new HTML.
Upload all your public files
However you want to get the files in there, go ahead and push them now. If you use that s3commit script, just run it. For a path of the
form RAILS_ROOT/public/dir/file, the S3 key should be just dir/file. More subdirectories are fine.
Test a few by going to
http://assets.example.com/stylesheets/example.css, etc.
Start sending back S3 URLs in your rails application
Rails makes this really easy. Just add (or uncomment) the following line in your config/environments/production.rb:
config.action_controller.asset_host = “http://assets.example.com”
And restart mongrel (or whichever you use).
Obviously you can also put it in development.rb if you want to test it there first, but eventually you probably want development running locally so you can change files without uploading them each time.
Now load some pages, and you’ll see that all of your image_tag, image_path, javascript and stylesheet asset tag helpers are putting
http://assets.example.com before each asset.
That’s it! Make sure it works, you’re done.
Of course, there are a few other little things.
Find where you were lazy with assets
Watch your Apache logs to see if any public/ files are being served up by your host. You may find a few places where somebody didn’t use an
AssetTagHelper.
Use s3commit in the future
Just think of it as equivalent to db/migrate. Develop on your dev box, and when you update your production environment, watch the
list. Action in db/migrate means you have to run rake migrate, action in public means you have to run s3commit.
In case of trouble
Your public directory will be up-to-date because that’s where you’re developing. If you keep your Apache config set up for static files, it’s really easy to switch back.
So if S3 goes down or anything bad happens (the credit card you gave Amazon gets maxed out…), serving the assets locally is a one line change in your environments/production.rb config file.
Have a ServerAlias in your Apache config for the asset host’s name too, so that you can also change the CNAME (or use a hosts file) for emergency debugging.
Watch out for swfs and java applets
Just make sure that if you put any flash or java files into S3, they know that all controller actions, etc will be on a different host and now require an absolute URL. Cross-domain security issues will apply, so you may wish to keep those out of s3 (there isn’t a flash url helper in AssetTagHelper anyway).
Questions? Improvements?
Feel free to write me, casey@jamglue.com
—-
anyone knows how configure multi asset host without coping buckets?? (having all of assets hosts pointing to a unique bucket) i’m happy to receive ideas to jsilva at jcode.cl