Setting up svnsync-ed (mirrored) SVN repositories on Ubuntu (part 2 of 2)

In: Open Source|Operating Systems|Web development

31 Mar 2007

Note: if you haven’t already, you may want to read part 1 of this article first.

Phew, and that was all the work you needed to do to migrate your Subversion repository to another server. We’ve barely touched that other server that we wanted to use for mirroring the repository!

“Bootstrapping” your mirror SVN server for svnsync

This mirror SVN server also needs Subversion 1.4.x installed, so go ahead and do the (almost) same things we’ve done in part 1 to get Subversion installed. You should be able to use the .deb package generated by checkinstall on your main Subversion server to install Subversion 1.4.x on the mirror SVN server. Just scp it to the mirror SVN server and install it (dpkg -i subversion-1.4.3.deb).

With Subversion installed, create a new Subversion repository (using svnadmin create, remember?), but don’t load the repository dump like we did on the main SVN server in part 1 (of course, since we are going to be mirroring the repository!).

Setting up svnsync to mirror your repository

First, create a SVN user for svnsync to use – let’s call this the ‘svnsync user’. The easiest (and best) way to do this is edit the svnserve.conf and passwd files:

conf/svnserve.conf


# Uncomment this line.
password-db = passwd

conf/passwd


svnsync = secret

This gives read and write access to the ‘svnsync user’. The svnsync program will authenticate with our repositories as this user via the svn:// protocol (i.e. via svnserve).

Next, we need to create a pre-revprop-change hook for the destination repository. The svnsync documentation has a detailed explanation. Create a hooks/pre-revprop-change file under your destination repository’s directory.


#!/bin/sh
USER="$3"

if [ "$USER" = "svnsync" ]; then exit 0; fi

echo "Only the svnsync user can change revprops" >&2
exit 1

Make it executable, and then initialize the sync:


chmod +x hooks/pre-revprop-change
svnsync init file:///var/svn/repositories/destination_repos svn://source.host/source_repos

Don’t worry, this only sets up the sync – there’s no actual data copying yet. Syncing your repository data may take a long time if you have a big source repository, so I suggest using nohup to run the code overnight (or something), or at least saving the output in a log. Either way, the command to start the sync is:

svnsync sync --username svnsync file:///var/svn/repositories/testsync/

You should start seeing svnsync committing in changes from your source repository. Instant gratification (well, almost)! Should your svnsync process get aborted or killed, you can remove the hanging lock by running:

svn propdel svn:sync-lock --revprop -r 0

Setting up ‘on-the-fly’ syncing

So now you have your source and destination repositories synced, but what happens when you start committing changes to your source repository? Nothing! That’s because svnsync is merely a passive syncing tool (meaning you have to run it to sync, instead of it knowing when to sync automatically).

There are two ways you can setup ‘real-time’ syncing:

  1. Use cron (or a similar scheduler) on the destination repository server. Add something like this to your crontab:

    * * * * * /usr/local/bin/svnsync --non-interactive sync svn://source.host/source_repos

    This basically runs svnsync on your destination repository server every minute to pull down any changes to your source repository.

  2. Add a post-commit hook to the source repository. I found this svnsync entry by Paul Querna that has a sample post-commit hook. If I recall correctly I tried it but it didn’t work for me, so I settled on using cron to sync up my repositories.

Things that I skipped

There’re some things that I skipped over while writing this, mainly to do with SVN authentication.

  • If you’re accessing your repository via the svn+ssh:// protocol, you’ve to manage the (group) permissions of the repository files in the filesystem appropriately (basically the repository should be group writable by your users). chmod and chown are your friends, as is NIS (or something similar) to manage your users. I use these steps to create a new SVN repository that gets access via the svn+ssh:// protocol:

    
    sudo mkdir /var/svn/repositories/funky_project
    mkdir /tmp/funky_project
    mkdir /tmp/funky_project/trunk
    mkdir /tmp/funky_project/branches
    mkdir /tmp/funky_project/tags
    sudo svnadmin create /var/svn/repositories/funky_project
    sudo svn import /tmp/YourProjectNameHere file:////var/svn/repositories/funky_project -m "Initial import."
    rm -rf /tmp/funky_project
    sudo chown -R www-data:www-data /var/svn/repositories/funky_project
    sudo chmod -R g+w /var/svn/repositories/funky_project
        

    As you can see, my SVN users are part of the www-data group, and the repository directory is made group-writable.

  • The svn:// protocol has authentication configuration files in the conf/ directory of your repository. The SVN book has a section explaining how to configure authentication for svnserve.
  • Apache httpd can be used to expose your SVN repositories via the WebDAV protocol. This allows for the very commonly seen http:// repository URLs (especially for Open Source projects). Configuration is a little more involved and you would probably have to install Apache from source as well. The SVN book has the details.

Wrapping up

I hope someone found this entry useful – I know I could have used one when I was setting up Subversion and svnsync.

7 Responses to Setting up svnsync-ed (mirrored) SVN repositories on Ubuntu (part 2 of 2)

Avatar

gnuhoi

May 18th, 2007 at 1pm

This article is very wordy. It is not an improvement on the barebones svnsync.txt that already exists. i thought you were going to explain file permission stuff, but it fell flat

Avatar

Setting up svnsync-ed (mirrored) SVN repositories on Ubuntu (part 1 of 2) - redemption in a blog

July 23rd, 2007 at 9pm

[...] This is a 2-part journal on setting up migrating and upgrading a Subversion repository, and then using svnsync to mirror the newly created repository. (Part 2) [...]

Avatar

Confluence: ITS Development Space

April 8th, 2008 at 1am

Subversion…

Location of SVN repositories Subversion is running through apache on charlie.mannlib.cornell.edu. \\ You can browse the repositories at…

Avatar

Confluence: IT

October 31st, 2008 at 6am

Subversion…

Reference using svnsync…

Avatar

Confluence: IT

October 31st, 2008 at 7am

Subversion (ext)…

Reference Bookmarks using svnsync…

Avatar

Successfully Running svnsync through svn+ssh from post-commit and post-revprop-change Hooks | Eus' Technical Blog

December 10th, 2010 at 6am

[...] set up a subversion mirror and kept it in sync with svnsync through svn+ssh protocol as described here, I would like to have the sync happen automatically everytime there was a commit. Oh, the blog [...]

Avatar

Subversion Backup mit svnsync « Ralph's Blog

April 8th, 2011 at 7pm