Storage Advice on future plan for my File Server

I have a storage conundrum; I’m trying to figure out an upgrade path and future proof plan for my home file server. I’ll start by outlining what I currently have and what it does, before looking at the options as I see them. Any and all comments/feedback is most welcome.

Currently I’m running a WHS 1. The Hardware is as follows,

AMD Athlon II x4 620
4 Gigs of DDR2
ECS Black Motherboard (onboard video, onboard gig LAN (unused), 5 sata ports)
ST labs PCIe 4 port Sata Card
ST Labs PCIe 2 port Sata Card
Intel PCIe GIGe network card
460 Watt Cooler master PSU

Hard Drives
1x 320 gig WD OS Drive
3x Samsung F1 1tb Drives
5x Samsung F2 EG 1.5tb Drives
1x Seagate 1tb Drive

For a total of ~11 TB of storage (approx 1tb free)

The Server is one of two (flatmates is other) on a UPS, and network wise is plugged into a Managed 24 port Linksys GIG switch, which in turn is connected to a Linksys router/modem. Entire house is wired with cat 5e.

Server functions

The Server does a lot of stuff in the household, including but not limited to,

Encoding media, (from either DVD or videocams etc)
Transcoding, (for my poor underpowered netbook)
Downloading,
Serves files to the HTPC upstairs,
Serves files to potentially 3-5 other computers,
Storage location for TV being recorded.

The files it stores are mostly video files, however also stored is software, Photos, and Documents (receipts/serials/business docs). The documents and photos themselves are backed up to a portable drive regularly which is kept in a different part of the house. All shares on the server set to Duplication ON.

The problem,

1: Hard Drive thrashing - Due to the way WHS places files on the drives, any new media generally gets stuck on the same drive. Thus downloading, encoding and streaming tend to cluster **** the one drive that happened to have been added in last.

2: Cost effectiveness of Storage – As the capacity of the machine has been increasing its been getting more and more expensive to add in more hard drives, from a power usage , Case size, sata ports available, and accessories cost points of view.

3: General Speed – Even when optimally setup I find the performance losses in NTFS when drives get so full cause large drops in copy speeds to and from the server. A tangible problem (we often host home LAN parties and the server gets a good going over at those) in some situations that makes it all but impossible to use it for other functions.

What I’m wanting,

1: Better IO performance when multiple people are streaming HD 720/1080p content, + music, + recording TV. Ideally 4 of those. i.e. 4 clients watching movies or 2 watching movies 1 listening to music and 1 recording TV etc.
2: Better value with additional storage purchases.
3: To potentially migrate from 2k3 based os. Just feels a bit unpolished and bitsy, esp. in WHS install (lots of hacks etc)

Possible options in my eyes,

1: Do nothing major, keep adding in hdds and balance them to push data around? (not practical as number of available hdd ports is stretched already and mobo has no more PCIe slots)

2: Wait for WHS 2 and any potential fixes to the problems EDIT just read some reviews of beta. 10 hdd limits is a showstopper.

3: Hardware raid under 2k8 R2. Buy a real raid card. And Raid 6 the 1.5tb drives with an eye towards expanding that array to 8 drives total. (9tb usable).

4: Software raid Under Linux: Raid 6 again with the 1.5tb’s.

5: ZFS raid2? Don’t know much about this option.

Pro’s and Con’s,

My current mindset is leaning towards option number 3. Options 1 and 2 don’t appeal to me in any major way, as they don’t really solve the all the problems.

My current intention would be to drop the 320 gig drive. Boot the OS off either a 2x 1tb Raid 1 or a 4x 1tb Raid 10. Partition the array into something for the OS, and then a scratch drive for downloads, encoding in progress, TV recordings, and a Storage drive for the small files. I.e. Photos, Documents, Maybe even music. Raid 6 the 1.5’s and grab or 2 more initially. Use these for storage and serving of the media, Movies, TV and any Software. Max capacity of 9 TB is just less than double what I currently have so that’s plenty of room for growth. And can always add in another controller and array later. Acquiring a legal copy of 2k8 R2, is not a problem for me.

Moving the downloads off the Raid 6 pool to the Raid 10, allows for the Raid 6 to focus on serving. I can just schedule stuff to be copied across during the night time etc.

Option 4 while it does appeal to me in terms of its price, my last encounter with Linux raid was rather unsatisfying. I had 5x 400 gig drives in raid 5. And it worked ok, despite being quite slow, however the problem was me. I’m not a Linux user, even the most basic things requires me to Google it, and trawl forums. I’m not sure I’ve enough time to jump into that. Esp. while intrusting all my data into it.

The same goes for ZFS Raidz2, with its advantages being really appealing, and quite neat in the geeky way. Lack of understanding of the OS in general is the big problem
So running with the Idea of option 3, what controller should I be looking at?

I’m based in New Zealand, so this sort of thing is uncommon and generally moderately expensive. Options for raid cards that are reasonably affordable and available include Adaptec 4805, a 5805? and of course anything I could potentially import. The 5805 is a Supermicro version which as far as I know means that its just got 256mb cache rather than 512.

Any and all thoughts/comments would be appreciated.

 

Yay another kiwi!

I appreciate your problem. I have 4 of the EG 1.5TB in RAID 0 (A mate has more or less the same data as me so we are backups for each other) off the X58 chipset. The performance is very good but obviously not as good as a proper hardware-based RAID. Unfortunately I can't recommend any good RAID cards.

I'd go for option 3. The improved RAID controller should improve performance a lot and is more scalable (since it has more ports). A hardware RAID will perform better than all those other options.

Windows Server 2008 R2 is a very nice OS. I run it as my desktop OS and recently had it running headless for three weeks non-stop (Uni holidays so left my PC locked in my cupboard at my hall of residence with a long ethernet cable haha). Remote Desktop is easy to set up and works very well.

One thing to note for Server, make sure your drivers are compatible. Theoretically anything that works on Windows 7 will work on Server 2008 R2. However I have had installers get partway through, determine that the OS is not the Windows 7 or Vista they were expecting and quit. Most hardware shouldn't be a problem (especially RAID cards since they are server orientated) though.

Of course you maybe already familiar with Server, I don't know. Good luck!

 

Option 3 without a doubt.

 

This.

Spend money on a card that has online expansion and migration AND cache if you are REALLY hammering the **** out of it. A chunk of cache will save your disks somewhat, but cost you more.

 

I'd go with option 3 too.

 

Thanks a bunch for the advice guys. Any suggestions on brands of Hard Raid cards? Would an Adaptec 4805 Work fine? or do i really need to get one of the newer more grunty 5805's or another brands equivalent