Brendan Nee

Automatic wordpress backups to Amazon S3

October 25th, 2007

I use wordpress for many sites, and its really important for me to be able to automatically perform nightly backups of not just the database, but all files including uploads, themes, plugins and the basic wordpress files. I need the ability to revert any file back to a previous version.

I found a variety of solutions to backing up files using S3, but none that exactly met my needs, so I decided to post my work here.

Note that these instructions assume your web server is Unix, Linux or Mac OS X based. It also assumes you have a basic understanding of FTP, SSH, firefox extensions and compressed files.

1. Get the s3-bash code library

The s3 BASH library is a collection of scripts that can get, put or delete files from Amazon s3. You can grab them from Google Code.

2. Unpack and upload these scripts to your server.
They should probably live outside of your webroot. Depending on how your server is set up you might put them in:

/home/USER/s3

Where USER is your username.

3. Give s3-put the proper permissions
We need to be able to execute s3-put so chmod it to 744.

chmod 744 /home/USER/s3/s3-put

4. Sign up for an Amazon S3 Account
You’ll need an Amazon S3 account, so sign up and then get your access key ID and secret access key on the AWS access identifiers page.

5. Put your secret access key in a text file and upload it to the server
You could call it s3.key and put it in the same folder you uploaded the s3-bash scripts to. Chmod it to 640.

chmod 640 /home/USER/s3/s3.key

6. Create a folder for the backups
Make a folder on your webserver for backups, perhaps:

/home/USER/backups

Where USER is your username.

7. Create a bucket on Amazon S3 for your backups
The easiest way I found to do this is to use the firefox extension S3 Firefox Organizer. Add the extension, restart firefox and then in Firefox go to “Tools” > “S3 Organizer”. Click “Manage Accounts” and enter your access key ID and secret access key and then login. Create a bucket (directory) for your backups. I created a one called bnee.

8. Write the backup script
This is the tricky part. Here is the script I came up with. The [[Double Bracketed]] items are places where you’ll have to enter information specific to your account (see below).

#!/bin/bash
USER=[[USER]]
mysqldump –add-drop-table -u [[DBUSER]] -p[[DBPASSWORD]] -A >/home/$USER/backups/mysql.sql
rm /home/$USER/backups/backup.tgz
tar -czvf /home/$USER/backups/backup.tgz [[PATHTOWEBROOT]] /home/$USER/backups/mysql.sql
NOW=$(date +_%b_%d_%y)
/home/$USER/s3/s3-put -k [[S3ACCESSKEYID]] -s /home/$USER/s3/s3.key -T /home/$USER/backups/backup.tgz -v /[[S3BUCKET]]/backup$NOW.tgz

[[USER]] – Because the username of my web server is used a lot in the paths, this is a variable
[[DBUSER]] – The username of the mysql database. You can get this from the wp-config.php file of your wordpress installation.
[[DBPASSWORD]] – The password for your mysql database. You can get this from the wp-config.php file of your wordpress installation.
[[PATHTOWEBROOT]] – The path to the webroot. Its likely something like /home/USER/public_html
[[S3ACCESSKEYID]] – The access key ID for Amazon S3 (see step 3).
[[S3BUCKET]] – The name of the bucket (directory) you created in step 6.

You might have to change the various paths depending on the way your web server is set up and the folder names you chose in previous steps.

Paste all of this into a text file and name it something like backup.sh. Upload it to your web server, outside of the webroot.

/home/USER/backup.sh

9. Test the script
Verify that the script is working by running it.

sh /home/USER/backup.sh

If everything is working ok, you’ll see a lot of output as each file of your website is added to the compressed backup and then uploaded to Amazon S3. It may take a minute or so to run, but when it is done you can verify that the script worked by using the S3 Firefox plugin to check that the file ended up where you expected it to.

10. Automate it
Add a cron job to run it every night

0 0 * * * sh /home/USER/backup.sh

If everything is configured correctly, this should perform a full backup nightly at midnight.

Note for subversion users: if you’d like to exclude all of the .svn folders and files from your backup, just use

--exclude=".svn"

before the [[PATHTOWEBROOT]] in step 8 above.

  • http://elihorne.com Eli

    Wow. I went googling for a tutorial on this and guess where I ended up.

    Thanks for the guide! Seriously handy.

  • http://elihorne.com Eli

    for the record: works like a charm.

  • http://www.bnee.com Brendan Nee

    Glad to know it was useful and that this post shows up on google. If you have modifications or suggestions let me know.

  • http://elihorne.com Eli

    The only mods I had to make were specific to my hosting environment. That, and I added in a couple variables so I could use a single reference point to backup all my sites. All in all, exactly what I was looking for.

  • kato

    backing up the whole database every night doesn’t seem very efficient. isn’t there any way to just back up the changes or otherwise establish some kind of version control?

  • http://www.bnee.com Brendan Nee

    There probably is a way to backup just the changes to the database, but it was beyond the scope of what I was trying to do. From a storage space issue, the fact that this script backs up all of the files each day (regardless of if they’ve changed) might be more of an issue. Also, it could be designed to delete backups after a certain time period. If anyone tinkers with the code and has suggestions or solutions, please let me know.

  • Nnyan

    Now if I can find something like this that can make incremental backups.

  • Pingback: de » Amazon s3 backup for web server files and SQL using bash

  • Duivesteyn

    I’ve modified this to better suit CPanel based sites with sql support at http://duivesteyn.net/2008/amazon-s3-backup-for-webserver-public_html-sql-bash/

    hope it helps someone

  • Pingback: links for 2008-07-23 : Digirati Dad

  • http://www.johndesu.com John Dyer

    Thanks so much for this post! Works great :)

  • Pingback: I have moved. And wordpress 2.7. : John Desu

  • Pingback:   Amazon S3 by Fishcakes Blog

  • Pingback: Automated, Nightly, Offsite backup for wordpress (or anything else) thanks to Amazon S3 | blinktag inc

  • http://www.TheBigQuestions.com/blog Steven E Landsburg

    Thank you a thousandfold. This was amazingly clear and helpful.

  • Pingback: Hacer respaldo automatizado de WordPress usando Amazon S3

  • http://www.coolshite.net Bruce Moyle

    You sir are awesome.

    Thanks for the tutorial, it has made my life a lot easier and my websites a hell of a lot more safe. My only question I would propose onto of this already excellent howto is incremental .tgz files. If are running a website like me with lots of big media assets, you don’t want to be uploading 20+ gigs each time.

    Thoughts how you could put this together?

    Thanks

    Bruce