While performing a normal update from Subversion recently, I received an error stating that an error had occurred during the download, and the update stopped. This article shows how I diagnosed the problem and got svn back on track.
The error I was getting from the svn client (in this case, Tortoise SVN) was:
svn: REPORT request failed on '/repos/myproject/!svn/vcc/default'
svn: REPORT of '/repos/myproject/!svn/vcc/default': 200 OK
To ensure that this was not due to a problem in the copy of source I had, I used another PC to do a full checkout of the code. The error occurred in the same location. I then decided to look at the server logs and see if I could get any better idea of what was going wrong. Since I use svn with apache, all I needed to do was look at the apache erorr log, where I saw the following:
A failure occurred while driving the update report editor [5000, #200002]
Can't read length line in file '/var/www/svn/myproject/db/revprops/733'
I opened the above file, and found that it was corrupted to the point of being unreadable; it appeared to be binary data.
I first tried to manually fix the properties file (revprops/733), but that had no effect other than changing the error message to something very arcane, but still reporting problems with rev 733.
svn: The REPORT request returned invalid XML in the response: XML parse error at line 79099: no element found (/repos/ASIOne_Prototype/!svn/vcc/default)
I do full backups of the subversion repositories every night using svnadmin dump. Since I wasn’t able to fix the repository manually, I had to turn to the full backup, but ran into a problem: The backup was also suffering from this same problem. Since it couldn’t retrieve rev 733 from the repository, it just stopped at revision 732.
Fortunately, I do a backup of each revision just after it’s committed. I do this with a svnadmin dump —incremental command in each repository’s pre-commit hook. I looked at the backup file for rev 733, and it appeared to be fine; it was completely readable since the corruption occurred some time after rev 733 had been committed.
I moved the original repository, and created a new myproject repository, then loaded the full backup (which went only to rev 732) using the svnadmin load utility. After the full backup had finished, I wrote a script to call svnadmin load on the per-revision backup files starting at revision 733, and going all the way to 763 (the most current rev at the time).
Once the script completed, I exported a revision I knew I had fully backed up elsewhere, created a sha1 hash file for the directory containing the source, and then compared the hashes against the backup of those files. There were no checksum mismatches or missing files.
Ensure you have backups. It’s very easy to do. To create a nightly full backup, create a script similar to the following:
mv SVN_Full.dump SVN_Full`date +%Y%m%d`.dump
svnadmin dump /var/www/svn/myproject > SVN_Full.dump
Add an entry to your crontab to run this once a day, preferably late at night or early in the morning.
Here’s what my per-revision backup script looks like:
NEW_REVISION=$1
svnadmin dump /var/www/svn/myproject -r $NEW_REVISION:$NEW_REVISION --incremental > SVN_$NEW_REVISION.dump
It takes a single parameter, the revision number of the commit just completed. Note the use of the —incremental switch. That will make svnadmin dump backup only those revisions shown in the range. Since we’re backing up only one revision, the start and end revision numbers are the same. This script is invoked in the post-commit hook with the following code:
/bin/sh /home/backup/scripts/backup_svn_rev.sh "$REV"
