SVN repository backup strategies - windows

What methods are available for backing up repositories in a Windows environment?

You could use something like (Linux):
svnadmin dump repositorypath | gzip > backupname.svn.gz
Since Windows does not support GZip it is just:
svnadmin dump repositorypath > backupname.svn

We use svnadmin hotcopy, e.g.:
svnadmin hotcopy C:\svn\repo D:\backups\svn\repo
As per the book:
You can run this command at any time and make a safe copy of the repository, regardless of whether other processes are using the repository.
You can of course ZIP (preferably 7-Zip) the backup copy. IMHO It's the most straightforward of the backup options: in case of disaster there's little to do other than unzip it back into position.

There's a hotbackup.py script available on the Subversion web site that's quite handy for automating backups.
http://svn.apache.org/repos/asf/subversion/trunk/tools/backup/hot-backup.py.in

Here is a Perl script that will:
Backup the repo
Copy it to another server via SCP
Retrieve the backup
Create a test repository from the backup
Do a test checkout
Email you with any errors (via cron)
The script:
my $svn_repo = "/var/svn";
my $bkup_dir = "/home/backup_user/backups";
my $bkup_file = "my_backup-";
my $tmp_dir = "/home/backup_user/tmp";
my $bkup_svr = "my.backup.com";
my $bkup_svr_login = "backup";
$bkup_file = $bkup_file . `date +%Y%m%d-%H%M`;
chomp $bkup_file;
my $youngest = `svnlook youngest $svn_repo`;
chomp $youngest;
my $dump_command = "svnadmin -q dump $svn_repo > $bkup_dir/$bkup_file ";
print "\nDumping Subversion repo $svn_repo to $bkup_file...\n";
print `$dump_command`;
print "Backing up through revision $youngest... \n";
print "\nCompressing dump file...\n";
print `gzip -9 $bkup_dir/$bkup_file\n`;
chomp $bkup_file;
my $zipped_file = $bkup_dir . "/" . $bkup_file . ".gz";
print "\nCreated $zipped_file\n";
print `scp $zipped_file $bkup_svr_login\#$bkup_svr:/home/backup/`;
print "\n$bkup_file.gz transfered to $bkup_svr\n";
#Test Backup
print "\n---------------------------------------\n";
print "Testing Backup";
print "\n---------------------------------------\n";
print "Downloading $bkup_file.gz from $bkup_svr\n";
print `scp $bkup_svr_login\#$bkup_svr:/home/backup/$bkup_file.gz $tmp_dir/`;
print "Unzipping $bkup_file.gz\n";
print `gunzip $tmp_dir/$bkup_file.gz`;
print "Creating test repository\n";
print `svnadmin create $tmp_dir/test_repo`;
print "Loading repository\n";
print `svnadmin -q load $tmp_dir/test_repo < $tmp_dir/$bkup_file`;
print "Checking out repository\n";
print `svn -q co file://$tmp_dir/test_repo $tmp_dir/test_checkout`;
print "Cleaning up\n";
print `rm -f $tmp_dir/$bkup_file`;
print `rm -rf $tmp_dir/test_checkout`;
print `rm -rf $tmp_dir/test_repo`;
Script source and more details about the rational for this type of backup.

I use svnsync, which sets up a remote server as a mirror/slave. We had a server go down two weeks ago, and I was able to switch the slave into primary position quite easily (only had to reset the UUID on the slave repository to the original).
Another benefit is that the sync can be run by a middle-man, rather than as a task on either server. I've had a client to two VPNs sync a repository between them.

svnadmin hotcopy
svnadmin hotcopy REPOS_PATH NEW_REPOS_PATH
This subcommand makes a full “hot” backup of your repository, including all hooks, configuration files, and, of course, database files.

svnbackup over at Google Code, a .NET console application.

You can create a repository backup (dump) with svnadmin dump.
You can then import it in using svnadmin load.
Detailed reference in the SVNBook:
"Repository data migration using svnadmin"

Basically it's safe to copy the repository folder if the svn server is stopped. (source: https://groups.google.com/forum/?fromgroups#!topic/visualsvn/i_55khUBrys%5B1-25%5D )
So if you're allowed to stop the server, do it and just copy the repository, either with some script or a backup tool. Cobian Backup fits here nicely as it can stop and start services automatically, and it can do incremental backups so you're only backing up parts of repository that have changed recently (useful if the repository is large and you're backing up to remote location).
Example:
Install Cobian Backup
Add a backup task:
Set source to repository folder (e.g. C:\Repositories\),
Add pre-backup event "STOP_SERVICE" VisualSVN,
Add post-backup event, "START_SERVICE" VisualSVN,
Set other options as needed. We've set up incremental backups including removal of old ones, backup schedule, destination, compression incl. archive splitting etc.
Profit!

there are 2 main methods to backup a svn server, first is hotcopy that will create a copy of your repository files, the main problem with this approach is that it saves data about the underlying file system, so you may have some difficulties trying to repostore this kind of backup in another svn server kind or another machine.
there is another type of backup called dump, this backup wont save any information of the underlying file system and its potable to any kind of SVN server based in tigiris.org subversion.
about the backup tool you can use the svnadmin tool(it is able to do hotcopy and dump) from the command prompt, this console resides in the same directory where your svn server lives or you can google for svn backup tools.
my recommendation is that you do both kinds of backups and get them out of the office to your email acount, amazon s3 service, ftp, or azure services, that way you will have a securityy backup without having to host the svn server somewhere out of your office.

Here a GUI Windows tool for make a dump of local and remote subversion repositories:
https://falsinsoft-software.blogspot.com/p/svn-backup-tool.html
The tool description says:
This simply tool allow to make a dump backup of a local and remote subversion repository. The software work in the same way of the "svnadmin" but is not a GUI frontend over it. Instead use directly the subversion libraries for allow to create dump in standalone mode without any other additional tool.
Hope this help...

I like to just copy the entire repo directory to my backup location. That way, if something happens, you can just copy the directory back and be ready to go immediately.
Just make sure to preserve permissions, if needed. Usually, this is only a concern on Linux machines.

For hosted repositories you can since svn version 1.7 use svnrdump, which is analogous to svnadmin dump for local repositories. This article provides a nice walk-through, which essentially boils down to:
svnrdump dump /URL/to/remote/repository > myRepository.dump
After you have downloaded the dump file you can import it locally
svnadmin load /path/to/local/repository < myRepository.dump
or upload it to the host of your choice.

If you are using the FSFS repository format (the default), then you can copy the repository itself to make a backup. With the older BerkleyDB system, the repository is not platform independent and you would generally want to use svnadmin dump.
The svnbook documentation topic for backup recommends the svnadmin hotcopy command, as it will take care of issues like files in use and such.

#echo off
set hour=%time:~0,2%
if "%hour:~0,1%"==" " set hour=0%time:~1,1%
set folder=%date:~6,4%%date:~3,2%%date:~0,2%%hour%%time:~3,2%
echo Performing Backup
md "\\HOME\Development\Backups\SubVersion\%folder%"
svnadmin dump "C:\Users\Yakyb\Desktop\MainRepositary\Jake" | "C:\Program Files\7-Zip\7z.exe" a "\\HOME\Development\Backups\SubVersion\%folder%\Jake.7z" -sibackupname.svn
This is the Batch File i have running that performs my Backups

For the daily and full backup solution just use the SVN backup scripts here.

I have compiled the steps I followed for the purpose of taking a backup of the remote SVN
repository of my project.
install svk (http://svk.bestpractical.com/view/SVKWin32)
install svn (http://sourceforge.net/projects/win32svn/files/1.6.16/Setup-Subversion-1.6.16.msi/download)
svk mirror //local <remote repository URL>
svk sync //local
This takes time and says that it is fetching the logs from repository. It creates a set of files inside C:\Documents and Settings\nverma\.svk\local.
To update this local repository with the latest set of changes from the remote one, just run the previous command from time to time.
Now you can play with your local repository (/home/user/.svk/local in this example) as if it were a normal SVN repository!
The only problem with this approach is that the local repository is created with a revision increments by the actual revision in the remote repository. As someone wrote:
The svk miror command generates a commit in the just created repository. So all the commits created by the subsequent sync will have revision numbers incremented by one as compared to the remote public repository.
But, this was OK for me as I only wanted some backup of the remote repository time to time, nothing else.
Verification:
To verify, use the SVN client with the local repository like this:
svn checkout "file:///C:/Documents and Settings\nverma/.svk/local/" <local-dir-path-to-checkout-onto>
This command then goes to checkout the latest revision from the local repository. At the end it says Checked out revision N. This N was one more than the actual revision found in the remote repository (due to the problem mentioned above).
To verify that svk also brought all the history, the SVN checkout was run with various older revisions using -r with 2, 10, 50 etc. Then the files in <local-dir-path-to-checkout-onto> were confirmed to be from that revision.
At the end, zip the directory C:/Documents and Settings\nverma/.svk/local/ and store the zip somewhere. Keep doing this regularly.

as others have said, hot-backup.py from the Subversion team has some nice features over just plain svnadmin hotcopy
I run a scheduled task on a python script that spiders for all my repositories on the machine, and uses hotbackup to keep several days worth of hotcopies (paranoid of corruption) and an svnadmin svndump on a remote machine. Restoration is really easy from that - so far.

1.1 Create Dump from SVN (Subversion) repository
svnadmin dump /path/to/reponame > /path/to/reponame.dump
Real example
svnadmin dump /var/www/svn/testrepo > /backups/testrepo.dump
1.2 Gzip Created Dump
gzip -9 /path/to/reponame.dump
Real example
gzip -9 /backups/testrepo.dump
1.3 SVN Dump and Gzip Dump with One-liner
svnadmin dump /path/to/reponame | gzip -9 > /path/to/reponame.dump.gz
Real example
svnadmin dump /var/www/svn/testrepo |Â gzip -9 > /backups/testrepo.dump.gz
How to Backup (dump) and Restore (load) SVN (Subversion) repository on Linux.
Ref: svn subversion backup andrestore

Related

Teamcity upgrade from 9.16 to 10

I want to upgrade Teamcity 9.16 to 10. I want to proceed with manual back-up and then restore it . I am using external database - mysql . i want to upgrade database as well. How should i proceed with this?
TeamCity documentation states that
Backups created with TeamCity 6.0+ can be restored using the same or
more recent TeamCity versions
so you should be able to create a backup in TC9 and then restore it in TC10.
The simplest way to create a backup is to navigate to the Administration | Backup section in the server UI to specify some parameters and run the backup, as described here.
The other options are
backup via the maintainDB command-line tool — it is basically the same option, as backup via the UI
manual backup
which are described on the corresponding page of the TC documentation.
Restoring data from backup is performed using the maintainDB tool, basically the steps for your case are:
install new TeamCity (but do not start the server)
create a new empty Data Directory
create and configure an empty database
configure a temporary database.properties file
place the database drivers into the lib/jdbc in new data directory
use the maintainDB utility located in the <TeamCity Home>/bin to run restore command:
maintainDB.[cmd|sh] restore -A <absolute path to the Data Directory> -F <path to the TeamCity backup file> -T <absolute path to the database.properties file>
If the process completes successfully, copy over
/system/artifacts from the old directory
More details could be found on the corresponding page.

Can lftp execute a command on the downloaded files (as part of the mirroring process)?

This may be asking too much from an already very powerful tool, but is there a chance that lftp mirror can execute a command during the mirroring process (from remote directory to the local machine)?
Specific example: lftp is asked to mirror a remote directory with xml files into a local folder and as soon as each file is downloaded/updated, it converts the file to JSON format using xml2json.
I can think of a solution that relies on monitoring the local copy of the mirrored folder for changes via find and then executing xml2json on the new/updated files, but perhaps there is a simpler way?
You can use xfer:verify and xfer:verify-command settings to run a local command on every transferred file.

Bonobo Git Server: how do you open complete repository in windows explorer/backup?

I recently installed Bonobo git server on my workstation. It will work as a git server for me and my colleagues. Using a script, the repositories will be stored on a separate server usinng a .bat file to copy the files regularly. The problem is I don't know where to find the repo files. I selected D:/git_repo as destination for bonobo repositories, but when i go there in explorer, there are only a few files and not the sources (the folders are alos only a few hundreads kbs big).
Do you have any ideea where the complete files are located?
If it was me, instead of using a scheduled script to copy the repository files to a separate server, I would on the destination/backup server have a scheduled task that performed a 'git pull' from your server.
Did you changed the Repository directory in the Bonobo UI settings ?
If you did, try to restart the IIS service.
If none helps, thenWhat exactly are you copying ? If your repositories are initialy exist, either by a script you execute or by the Bonobo UI, they can be created directly to your D:\Git_Repos as you specify

backup collabnet subversion edge to another hard disk

I've installed Collabnet Subversion Edge, and would like to make sure I have it backed up properly. I would like NOT to use the CloudBackup service offered.
I've went to the administration interface for collabnet (localhost:3343) and went to Repositories > Backup Schedule. There, one can choose between 3 different 'Type of Job':
Cloud Services Backup
Full Dump Backup
Hotcopy Backup
Neither lets you choose where to copy the backup. I've tried looking up how this works, but documentation seems to be lacking a lot.
What is the best way to backup such a repository? Shall I just keep a copy of the entire collabnet folder (c:\csvn)?
The Subversion Edge admin UI lets you specify the folder for backups. It defaults to a folder inside the normal data folder, but you can specify a different value. So, for example, if you have a D:\ drive that you want the backups to go on you can just specify that folder in the settings and the backups will go to that folder.
It does need to be a physically accessible hard drive though.
See the Backup Directory configuration item in this screenshot:
https://ctf.open.collab.net/sf/projects/svnedge/screenshots/screens/config/config.png
You can use Windows Server Backup to backup Subversion repositories. It allows you to shedule backups
to a network share, dedicated backup volume, writeable media. For example, wbadmin command-line tool allows you to safely backup your repositories. This simple command performs one-time copy backup of C:\foo\bar to X:\ volume:
wbadmin start backup –backupTarget:x: -include:c\foo\bar -vsscopy
(To install Windows Server Backup, run ocsetup WindowsServerBackup in elevated command-prompt).
You can setup backup in different ways:
wbadmin command-line tool,
PowerShell cmdlets, good for automation and customization of backup actions,
Windows Server Backup wizard (control panel, actually) MMC snap-in.
It's not required to stop server's service when you run the backup because FSFS repository backend is always in consistent state.
Here are general tips about recovering Subversion repository from a backup:
Recover repository backup to an empty directory to make sure that restored repository files won't mix with files of the broken one. After repository if recovered, you can delete broken repository and then replace it with the recovered one.
Stop-start cycle your Subversion server after recovering repository from a backup.
If your clients get errors after repository recover, run svnadmin recover against it. The command finishes instantly and makes repository accessible again.
If you have access to the repository directories then you should be able to use hotcopy directly and specify where the backups go.
It's enough to take a periodical backup of just csvn/data directory where all your repositories and configuration files are stored.
Visit this link for backup (and upgrade) options. The contents in the link is added below. Hope it helps.
Manual Upgrade/Reinstallation Steps
Subversion Edge includes an integrated mechanism for installing updates. This is the preferred way to do an upgrade as it handles whatever steps are needed to perform the upgrade and can be done remotely from your web browser. However, there are scenarios where you might want or need to do an upgrade manually, for example your Subversion Edge server might not be able to access the Internet to pull down the updates or maybe one or more critical installation files have become corrupted and you need to reinstall using the same version. Here are the steps for performing a manual upgrade or reinstallation:
Windows
If your existing Subversion Edge installation was installed using the installer from Subversion Edge 2.0.0 or later, then all you need to do to upgrade is download the latest installer and run it. This will uninstall the current version and install the new version (which is how the Windows Installer (.msi) process works for upgrades).
If you are not sure what version you installed with, you can always safely use this approach:
Stop the existing services and uninstall the current version from the
Windows Control Panel. This will leave behind your C:\csvn folder and
any files in it that have been modified since the original install.
Delete everything in the C:\csvn folder EXCEPT the data folder. So
you should be left with just the C:\csvn\data folder.
Install the new version. The installer will pick up the existing data folder and when the services start it will basically just be an upgrade to the new
version.
WARNING: Take note of this reported bug and backup the svn_access_file first:
artf7081 - Using Windows installer for updates can overwrite the svn_access_file
Linux/Solaris
To upgrade a Linux/Solaris installation, this is the safest way to do it:
Stop the servers $ bin/csvn stop $ bin/csvn-httpd stop
Rename the csvn folder $ mv csvn csvn-old
Untar the new release as a non-root user
Move the data folder back into the new release
$ mv csvn-old/data csvn
Important! Copy "dist" configuration files to data folder
$ cp -f csvn/dist/*.dist csvn/data/conf
Start the servers
$ bin/csvn start
$ bin/csvn-httpd start

SVN Post Commit Hook Batch Windows

I have a Windows Server running Visual SVN Server to store our repositories.
Also on that server is our test copy.
I'm trying to set up a simple SVN post-commit hook so it updates that test copy automatically every time I commit something
In Visual SVN in the post-commit hooks I've set up like this
"C:\Program Files (x86)\VisualSVN Server\bin\updatescripts.bat" D:\inetpub\TESTCOPY
Then that batch file has a simple update like this
PATH=%PATH%;"C:\Program Files (x86)\VisualSVN Server\bin\"
svn update %1
If I run the batch file in the server by double clicking on it or from command line works fine.
When committing something from my laptop it freezes and doesn't give me any error and locks the test copy so then I need to go in and run a clean up.
Visual SVN service is running as network service and this aacount has full access to the bin folder and the test copy on the server.
Any idea how to set up a simple svn update post commit hook anyone?
Thanks
Fede
I had a similar problem and it turned out to be that SVN likes paths to use forward slashes instead of backslashes.
Try this:
set MYPATH=%1
:: Transform backslashes to forward slashes
set MYPATH=%MYPATH:^\=/%
svn update %MYPATH%
You're running the svn update command. Exactly what working copy are you trying to update?
The parameter being passed is the Repository's path. This points not to a working directory, but to the directory that contains the Subversion master repository. This is the same directory where your post commit hook is stored.
Subversion hooks do not have access to the user's working directory, so you can't manipulate the user's files. Hook scripts usually should be using svnlook and not svn. By doing this, you prevent yourself from getting into any sort of trouble.
It is possible to update a Subversion working copy on the server, if you know the location:
PATH=%PATH%;"C:\Program Files (x86)\VisualSVN Server\bin\"
set SVN_WORK_DIR=C:\SVN\workdir"
svn update %SVN_WORK_DIR%
However, I wouldn't recommend this because it ties up Subversion. The user who did the commit would have to wait until the update is complete before Subversion returns the control of the prompt back to the user.
After trying a million different things this worked for me...
I put this in my post-commit hook
"C:\Program Files\VisualSVN Server\bin\svn.exe" update "C:\my path\"
where my path is the path to the working copy to be updated
Also I had to change the service to run as local system
That is really not the way you want to do that. What you should do is use something like Jenkins to watch your repository. Jenkins can watch your repository, and it when it changes, update your test copy, kick-off builds, run automated tests, etc.

Resources