20 Aug 2008

Backing up Subversion with DreamHost

6 Comments Programming

DreamHost have been offering Subversion repositories with their hosting packages for the last couple of months, and if you're a developer, a central repository can be a useful thing to have.

Although DreamHost have a simple form for setting up a new repository and they make regular backups incase a hard drive fails, I still have a lingering fear of losing all of my source code. Luckily, DreamHost also offer SSH access and allow you to setup cron jobs to backup your data yourself. This guide provides instructions for creating a shell script to backup, compress, email and restore your SVN repository.

The tools you'll need to export a repository are svnadmin, tar, split, mutt – an email client, and optionally crontab.

  1. The first thing you need to do is create a shell script to do the backing up. This can either be done from Windows, then uploaded to your home directory, or from a terminal using nano or vi.

    # Clean up any failed attempts and create a temporary directory for the backup
    rm -rf $HOME/svnbackup/
    mkdir $HOME/svnbackup
    cd $HOME/svnbackup
    # dump the repository
    svnadmin dump $HOME/svn/$REPO > $REPO.dump
    tar -cf - $REPO.dump | gzip > svn-$REPO$(date +\%Y\%m\%d).tar.gz
    split --bytes=15m svn-$REPO$(date +\%Y\%m\%d).tar.gz svn-$REPO$(date +\%Y\%m\%d).tar.gz.
    ls svn-$REPO$(date +\%Y\%m\%d).tar.gz.* | mutt -s "Backup - SVN $REPO ($(date +\%Y\%m\%d)) - Summary" -n -F /dev/null $BACKUPADDR
    for file in `ls svn-$REPO$(date +\%Y\%m\%d).tar.gz.*`
    echo "" | mutt -a $file -s "Backup - SVN $REPO ($(date +\%Y\%m\%d)) - $file" -n -F /dev/null $BACKUPADDR
    # remove the temporary directory
    rm -rf $HOME/svnbackup/

    The text at the start of the file needs to be changed to reflect the name of the repository to be backed up and the address to which to email the backup.

  2. Save the file as svnbackup.sh. I'll assume that the majority of users aren't linux gurus, so I'll leave it up to you to create the file and upload it.

    This script will first dump the whole repository, including all revisions and branches. The file created by the svnadmin tool can be in excess of 300Mb, depending on the size of your project. Emailing it as it is would use a ton of your bandwidth, so it'ts first compressed using tar.

    I use Google Hosted for my email, which is set to filter any emails to backup@mydomain.com into a separate folder to keep things tidy. The problem with this is that Google currently impose a 20Mb attachment size limit, so if you send a 90Mb attachment the email will just get bounced back to the sender. To get around this I use the split utility and a for loop to split the compressed repository and email each file separately. If you use the script as it is, you'll also get a message with the names of all of the created files, so if one times out while sending you'll know there’s one missing.

    Summary email with files

  3. Upload the file to your home directory (the default directory when you connect via FTP with the default account), then open an SSH session to continue. The best tool for this is Putty.

  4. When you've connected via SSH, you need to make the backup script executable. From Putty, type chmod a+x ./svnbackup.sh.

  5. If everything was set up correctly, you should be able to run the file as needed by connecting via SSH then typing ./svnbackup.

    The script can be modified to backup multiple repositories if necessary.

  6. It's a nice convenience to run backups automatically, and DreamHost also let you run cron jobs so there's no reason not to. Linux includes a task scheduler called crontab, which you can use to schedule the script to run automatically. From an SSH terminal, type crontab -e to edit your crontab.

  7. The syntax for crontab is minute, hour, day, month, dayofweek, so on a new line type 37 9 1 * * $HOME/svnbackup.sh to run the script at 9:37 on the first day of every month.

Backing up an entire Subversion repository creates large files that use up your bandwidth allowance, so use the script sparingly. Of course having a compressed dump of a repository isn't much good if you can't restore it in the event of a problem. once you have the files archived somewhere, if the repository gets corrupted or the SVN server dies, you can rejoin the files and use the svnadmin tool to load the backup into an existing repository.

  1. To restore an SVN backup, you first need to rejoin the files and unzip them. From the command line, type cat svn-myrepository.tar.gz.aa svn-myrepository.tar.gz.bb > svn-myrepository.tar.gz adding all of the files delivered to the address you set in the script.

  2. Unzip the archive with tar xvzf svn-myrepository.tar.gz.

  3. Import the backup into an existing repository (which should be empty) using svnadmin load /path/to/repository < /path/to/svn-file.dump. The process takes a while but you should have an exact copy of your original repository when it's done.

Tags: , , ,
written by
The author didn‘t add any Information to his profile yet.

6 Responses to “Backing up Subversion with DreamHost”

  1. Reply Davide says:

    Thank you for sharing this tutorial.
    However, I’ve run into a problem: looks like sendmail won’t send 20Mb files.
    This is the mail I get

    * Dumped revision 0.
    * Dumped revision 1.
    * Dumped revision 2.
    postdrop: warning: uid=0: File too large
    sendmail: fatal: root(0): message file too big
    Error sending message, child exited 75 (Deferred.).
    Could not send the message.

    I am an absolute beginner with mailing systems, but I think the problem must be in some config files (I’m using gmail too.. the problem can’t be in the recipient)

    Where can I change those settings? I know a lot of programs actually co-operate to send the mail.. but not which or how.

  2. Reply Echilon says:

    The split command should take care of that. Is the script splitting the files first?

  3. Reply Dhruv says:

    Hey, Are you still using this script to backup SVN repository? I mean splitting files makes backup and restore complex. Is there any way so that we can directly copy the file to Google Drive?

  4. Reply Dhruv says:

    Hey, I don’t mean that. What I am trying to say is that instead of sending split SVN backup files to email, can we upload a SVN backup file to google drive? Is there any facility in the linux which can do that.

Leave a Reply