I am thinking about a script/program that can run in background, and attempt to backup or synchronize a given filesystem path to a mirror location (probably located on an external/separate storage device).
This should apply to Windows but it could as well be used under Linux.
Differential/incremental backups are a bonus.
Windows System State backups are a bonus too.
Keeping the origin free of meta-data is essential. (unlike version control)
Searching by file or activity date could be interesting (like version control)
Backup repositories should be easy to browse and take little space.
Deleted files should be available for recovery for a period of time.
Windows Backup is tedious and bloated and limited.
Tar-gzipping is not accessible.
User interaction during backup should be nonexistent.
Amanda is the ultimate full-featured open-source backup solution, and there's a (relatively) new Zmanda Windows Client.
Duplicity is free and creates encrypted, incremental, compressed offsite backups. It's a linux app, but you could run it in cygwin or a small virtual machine.
I've written a perl script that runs it via a cronjob to backup several very big directories over DSL and it works great.
Check out AJCBackup. Does an excellent job at a good price.
Acronis True Image is great. It's not free but the Home edition is pretty cheap for what it does and it works reliably. Does image- and file- based backups, scheduling, instant backup of chosen folders accessible from explorer context menu, incremental/differential backups, can mount the backup files as Windows volumes and browse them, copy files out etc. It has saved my ass a few times already.
Related
My FreeNAS server is slowly dying and before that happens i need to migrate all data in the NAS to a windows server.
The FreeNAS has ZFS Snapshots and i need to restore data from a few days ago to the Windows server.
I have done some research and i can't think of the best way to do this. (i am not linux/Zfs savvy)
So the things i need to do is,
Restore ZFS Snaptshot from a few days ago to a windows Server
I mounted a windows share to the Freenas using mount_smbfs //username:password#server.name/share_name share_name/
I can copy and create files on that share just fine. So I was wondering if it was possible to restore an entire data set from an snapshot to the windows share.
Any help, tips is much appreciated.
Note. I could easily copy all data on a freenas volume to the windows share, but what makes it complicated for me, is restoring data from a snapshot without overwriting the current data on the volume and moving that data to the windows share.
You have two sensible possibilities:
Access the ZFS dataset (shared over SMB) from your Windows Server, then right-click on it in Explorer and choose "Previous Versions". You will get (after a short time depending on the number of snapshots) a list of all snapshots with their dates. You can then either explore them and copy some files over, or you can choose to copy all to another location (e. g. your new share).
Mount the Windows share on FreeNAS like you did, then go to <pool>/<filesystem>/.zfs/snapshot/ (path completion on the shell might be turned off for the .zfs directory, so type it in manually). There you'll find all your snapshots (like you would have on Windows' Previous Versions) and you can copy some or all files over to the new directory.
I would suggest the first way, because you have the GUI and cannot do any harm to the FreeNAS system this way.
On the other hand, have you thought about the possibility of rescuing the system? You did not specify why it's dying, but things like hard drives or mainboards can be swapped quite easily without requiring setting up everything anew. Maybe this would help you more than moving the data off to another, unconfigured system?
We need to distribute lots of small jpg files to offline systems. Right now, we send it as a 7zip (or plain zip) which is 800MB (230K files) and use 7zip to unzip it. It is taking about an hour to unzip on fairly large 4 core processors.
Is there a way on windows7 (or win server 2008) to create and unpack a package of files of this size in a more reasonable time frame?
(I will entertain even far out answers such as: put this all in a single CloudDB database as binary blobs and then ship the archive to the target machine, or create a VM, or a virtual disk image - but I will need some pointers to tips on doing that sort of stuff).
So then here's your far out answer: ;)
The problem probably doesn't lie in computing power. The filesystem and/or harddisk are the bottleneck most likely.
For Win7 (and afaik Server2008 as well) you could use a Virtual Hard Disk instead of zipping it. Win7 has native support for VHD-files and can emulate the content as a drive or subfolder via Disk Management. So there would be no need to unzip the files.
I had the same problem, and solved it. The issue is likely the Windows Attachment Service, which subjects downloaded or attached zip files to additional scrutiny for security reasons.
To bypass this:
Right-click the file
Choose Properties
Check Unblock
For more info, see: Why is WinZip slow?
I spoke to some colleagues, and they might have an easier solution. Since the size is under 4GB, and I want READ-ONLY access, I can create an ISO image, and then mount it on win7 or win2008server, using this Microsoft utility:
This utility enables users of Windows XP, Windows Vista, and Windows 7 to mount ISO disk image files as virtual CD-ROM drives.
I'm not familier with linux and debian system I work most with windows computers. But one of my clients uses debian linux web server and I need to upgrade the server's raid array.
Before I do anything with the server I would need a full system backup. I search through the internet for solution and also this site, but I haven't found acceptable answer.
I would need something like LVM snapshot, but I don't want to convert everything to LVM partition just for a backup. I found the DD to make bit by bit copy of the hardrive, but I should unmount the drive for it and I don't too much service offline. The reconfiguration of the raid will be enough offline. I found solution like TAR the necessary files and send through SSH, but it isn't a full system backup. I do backup every month form the files and settings.
I need a solution that makes an easy restorable image file of the server for emergency case. If the raid configuration fails I will need SOS restoration of the full system to the old config.
You should use rsync.
It's not a snapshot, but if you don't want to use LVM, it's a start.
I'm working on a windows platform and want to be able to auto sync my files one way 'on change' to my virtual windows or linux web server - also need to be able to filter file types. i can connect to the remote machine via network drives.
i'm ideally looking for a free, easy to set up solution - a commercial product that does what I need is called ViceVersa but its a little overkill and costs :)
Thanks
Josh
I'd use rsync - simple, easy to setup, and provides the filters you need. Also very low on bandwidth after the first pass.
Here is a link explaining how to get it working in Windows
Whilst rsync doesn't allow 'on-change' auto-syncing, it is very fast when it scans a sync'ed directory (even very large ones), so you could schedule a frequent sync to overcome this.
Edit: You could combine it with a program like this, to trigger an rsync on folder contents change. Cheaper than viceversa
For other users, its worth mentioning lsyncd, it will auto sync on changes between two machines (by default deferring to rsync). Will only work on Linux though, but if thats not a problem it works great.
It also seems that Sparkleshare has finally released some working code (Dropbox clone). Havent tried it myself but does cross-platform synching and you can setup your own server.
I admit this is not strictly a programming question, although I do use my WHS as a source repository server for home projects, and I'm guessing many other coders here do as well.
Does anyone have a recommendation for a good backup solution for the non-fileshare portion of Windows Home Server? All the WHS backups I've seen handle the fileshares, but none of the system files or other administrative stuff on the box.
Thanks,
Andy
Windows Home Server is designed to not need a backup of the OS. If your system drive fails, install a new drive, and then boot the WHS OS setup disc and install the OS. It will find the data on the other drives and recreate all the shared folders. You do need to do some configuring once it is back up but that is pretty small compared to not having to back it up.
One good solution for backing up the home server itself is to attach an external drive, say via USB 2.0 or eSATA. For this to work, though, you need the supporting software like Norton Ghost or something similar installed on your WHS server.
Windows Home Server Power Pack 1 (aka WHS PP1) added a feature to perform backups of the WHS shared folders to an external drive -- as you mention, this feature is only intended to do the data side and not the OS.
If you have an HP MediaSmart server, you could try the method mentioned in Quick & Easy Windows Home Server Backup and Restore. The author said it worked for him, but of course, caveat emptor. This technique has you creating a disk-image for your backup, and using that to restore from in the Recovery Disk / Restore disk process.
If you want a faster way to recover your OS and you do not have a Media Smart server, you can also check out these instructions on how to use a USB flash drive for installing WHS, and merge in the instructions found above for restoring a disk image vis-à-vis the OS Recovery disk process.
WHS OS backup solved by running two copies of WHS each on its own computer in a virtual machine with each WHS backing up the other (running in a VM makes the WHS a file thus able to be backed up and restored by WHS).
iDrive is Great and free under 2 gigs