Windows Incremental/Differential Backup Software

I might have asked around this topic before, but I now have a 3TB USB3 drive and USB3 'card in my machine and an urgent desire to backup. Windows' own backup utility is free, but is a mirror backup; that is, every single backup (for me) is of the order of 1.1TB when compressed, and takes hours and hours.

I want a much, much faster and more space efficient piece of software, with the ability to return individual files, folders and whole drives. What I'm envisioning is a huge initial backup, and then incremental/differential alterations that take of the order of minutes rather than years. If I have to I'll even pay for the damned thing.

Recommendations?

 

I'd recommend Macrium Reflect.
I've been using it for a few years and I find it great, plus the support staff are very responsive and even provide custom scripts on occasion to suit user's needs (as the program supports VBScript and batch files for automation of tasks).

I'm not certain what the situation is with regards to a free version, but they do have something for download there. I bought a licensed copy in a special offer a while back.

 

Acronis.

Can do all of the above and much more. Fabulous piece of software.

I still do complete back ups though, despite having around 1.5TB as the server and the mirror server are on 24/7 and I schedule it to back up overnight, but incremental back ups are what you need if your desktop machine is backing up lots of files.

Your C Drive, if you back this up should be fast enough to do a full back up though (unless you're backing up to a USB 2.0 drive).

 

Acronis as well. If you don't mind putting in a bit more effort for free software though SyncBack and the many rSync derivatives can do the same with a bit of intervention.

 

I use http://synchronicity.sourceforge.net/help.html but this is not for os back up but files and folders. If you want os backup try http://www.todo-backup.com/products/home/free-backup-software.htm both are free.

 

Thanks for the suggestions - remembering Pookey's evangelical zeal, I'm giving Acronis a try. Everything except my Movie partition is 388GB; the first backup of those took hours and hours (a 'Chain' backup on 'High' compression) and was 297GB when finished. I just asked it to back up again, and it's now predicting a 3.3hr backup.

I think this 3.3hr is another full backup. Does anyone know how long an incremental/differential takes, or how I trigger incremental/differential interim backups as part of the chain process?

 

If you have set Acronis to do a "Version Chain", the first is a full backup with the rest of them incremental until the sixth (I think) backup at which stage a full backup is run again.

Have to turned off verification? this takes nearly as long as the backup itself.

 

I'm in the middle of verifying the first backup - better safe than sorry, I figured - and have configured it to verify every month or something. Once that's done I'll see about trying to trigger an incremental backup - having not changed anything I was under the impression that there would not be much to backup and therefore it would not take very long...?

 

An incremental shouldn't take long, mine don't.

I have read in the Acronis support forums that some people are finding that automated validations (i.e. that run after a backup) are taking forever whilst manually run validations are quick (i.e. about the same time as a backup).

There is also a bug with the latest version that gives an erroneous error message at login: http://forum.acronis.com/forum/21859.

 

Uhm, an unverified backup is as useless as no backup at all.
And compression just aggravates things unless you do nothing but completes, with incrementals you shouldn't need bother.

 

The verification completed, so I'll check up on that later. I'm setting the Video backup now, which is incremental, weekly and max compression (anything to cut down the size!). I've set the other one to run its chain backups twice a week, so hopefully I'll see the first incremental non-video backup later this week and we'll see how long that and verification takes.

 

OK, I can give definitive results. The download of Portal 2 (some 11GB) and assorted alterations added up to a 15GB incremental backup - which took at least two hours.

I mean, that's hardly exceptional over a USB3 connection. Bizarre. Any tips for optimization?

 

I will stick my neck out and suggest rsync. It's a free (as in speech) tool that's industry-standard (I use it at work to backup ~350GB of of data).

It's incremental (only syncs changes) and you can choose your GUI (cwRsync, DeltaCopy) or you can use the CLI client (comes bundled with Cygwin).

If you ever move to a home fileserver set up, you can adapt the scripts with next to no trouble, and you can sync across multiple O/Ses. You can also do more advanced stuff like rotation, symlinking and network stuff like checksums, compression, transfer shell (SSH). You can also rsync between local disks (including USB).

Guides:
http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp
http://www.itefix.no/i2/node/10650
http://www.gaztronics.net/rsync.php

Any questions, just ask

 

Is that with verification on or off? Is it max compression? Lots of small files or fewer large ones?

My weekly backup (lots of photos and music files, along with all documents etc) ends up at around 125GB, using normal compression and it takes just over an hour to backup to my NAS via Gbit. I only verify once a month.

 

Verification off, normal compression, dunno about the files (lots of small ones, I'm guessing). I've also set mine to verify once a month - which for the Video disk (~900GB) will take almost a day, I reckon. I'll see if the following ones go any faster.

Cheers everyone for the advice.

 

I can actually see the throughput I am getting by the console on my NAS - I get around 85mbps during a backup.

It is a shame you can't do the same on your USB 3 connection - it sounds like it is quite slow. How can you measure the throughput during a backup?

 

An excellent question to which I don't have an answer. There's no 'detail' pop-out window telling me exactly what file is copying at the moment and the transfer speed.

 

I back up 120GB each night, and it takes 18 minutes over gigabit ethernet, so 15GB over USB 3 should be a fraction of that time. I suspect the USB 3.0 is not running as it should.

ctrl+shift+esc... then select performance, then resource monitor.. select disk to see disk activity of the disk being backed up, that's a rough indication.

I have stopped verifying. Every time I've restored from an acronis back up it's been absolutely fine. I also keep multiple copies. If you're doing incremental however, I'd leave it on, as they should be fast enough to not make an issue.

 

Well, I just ran a backup for the second time today just to see how long it takes. This is an incremental backup on my C:/Documents/Images/Music/Games partitions. The total data stored on those partitions is around 390GB; the increment at present is 580MB.

I started at 22:29. It's 23:15 now and I'll let you know when it finishes.

Also, the Time remaining estimate (currently reading 17 hours 44 minutes, as it has for the last thirty minutes) over the progress bar on True Image Home doesn't work, at all, and the progress bar doesn't seem to be working either.

Resource Monitor details TrueImageHomeService.exe's Read (B/sec) at between 205-1850 and Write (B/sec) at between 0-275. The I/O priority is Normal and the response time between 8-12ms. An in-progress screenshot (1680x1050 by download).


Resource Monitor Screenshot by Pure_Silver, on Flickr

Update: finished at 23:34, so 1hr 5m to backup an 8.89GB increment. This is wrong. There's no way that 8GB of stuff has changed between this afternoon and this evening. WTF?

Can anyone tell me what range the various values should be between?

 

I cant view it bigger... it says it is a private photo. You need to make it public on Flickr.


I've just started a back up of my D: drive to test now... it began precisely at 10:10. My D: drive is 230GB, and this is a complete back up.

Here's a screenie just after start.



I used the network traffic to judge speed as I back up to my RAID server over ethernet. It was going at 80MB/sec at start... it gradually gets faster as it goes along. This is with compression set too.

Here's a screenie of the disk usage screen.



Speed already increasing to over 90MB/sec.. progress bar in Acronis working well.

I'll edit this post when complete to evaluate final transfer time and total time taken.

I reckon the time is being taken evaluating the drive to establish what the increment is, not the actual back up of 8GB, which should take about 5 minutes.

EDIT

Finished at 10:52. Total time 42 minutes to back up 230GB with compression. This was over ethernet with a 100MB/sec limit. Highest recorded transfer rate was 98MB/sec. Backing up to a local drive should be faster still.




EDIT

Just ran an incremental. Initial back up obviously the same as the one above, but the incrementals took less than one minute... obviously only a few MB backed up.