How to Sync, Put or Pull from Linux to Amazon S3

I'm now using Amazon S3 as my web server for this site.

I'm uploading the files to it either from my Arch Linux Computer, or from my MacBook Pro

To do that, I'm using s3tools. It is a set of tools to be used from the command line either on Linux or Mac.

To install it on the Arch Linux box was as easy as:

pacman -S s3cmd

Now, you have some sub-commands with s3cmd, but the first think you should do is to configure it, so run:

s3cmd --configure

You should not run the above command as root

It will ask you for your Amazon S3 Access_Key and Secret_Key. If you don't know how to find them, here is a good tutorial.

Once you have configured s3cmd you can use:

s3cmd put, s3cmd pull. But my favorite is s3cmd sync. It works a lot like rsync, and is the perfect tool for syncing your local PC with Amazon S3 buckets.

The command I use is:

s3cmd sync --delete-removed /home/user/my_blog/_site/ s3://www.garron.me/ --add-header "Cache-Control: max-age=86400"

The option –add-header "Cache-Control: max-age=86400" is very important, and it is telling, that all your files should have a max age of 24 hours in cache servers, before they should be expired. You can adjust the number of seconds to your needs. To me 24 hours is a good time, for my content to be in cache (I'm using Amazon CloudFront).

Remember you can also install s3cmd on a Mac running Mac OS X