Some help needed for linux based home file server

Dear all,

Bit-Tech's article on "Build you own server" has encouraged me to, well, build my own server. So I made one with the following spec:

Pentium D 2.4Ghz
512Mb DDR2
2 x 250GB SATA HDD
Next Mobo (forgot full spec)

The server is to be used as:
- File server for my home (music/video)
- Stream server for my game console
- Web server (possibly)
- FTP server
- Backup server

However, some question left unanswered by the article:
1. Is it possible for me to monitor the power usage of my server? I don't want it to be as power hungry as my normal PC. Is there a script or apps that could manage power usage during idle period? and can I monitor the stat?

2. Any recommendation on backup solution? I've read about RAID but not quite get it. All I need is a daily backup (performed at night or something) of my files (pictures, docs, movies) so in case something wrong happen with the main drive, I have a backup

3. Recommendation on the HDD. Since I'll be using 2 x 250GB, should I partitioned the first HDD to install linux, or should I just use the whole HDD? I usually partitioned my HDD since there is possibility that I change OS, that way, I dun have to format my data partition, just the OS partition.

That's all for now. Thanks in advance.

 

1) I know of no way of monitoring actually wattage you'd need something on the plug. There are plenty of hardware monitors, conky, gkrellm are nice and visual, top or lm-sensors if your more cli inclined. As for idle times, disks can be made to spin down and linux is fully compatible with amd and intels speed stepping functions on their cpus. But tbh those Pd's are hungry buggers i'd be inclined to sell it and get something much lighter for what your looking to do.
2) rsync is pretty good for backups
3) I'd always partition the disk it makes things much easier in the future. 5-10 gig for the os 10-15 gig for the /home partition and the rest however you'd like it.

 

You can't 'guess' the power usage from software. However, ACPId will be able to throttle the system, and disable unused parts of it. Cpufreqd or Powernow will be able to scale your CPU too. Google your type of processor and you'll find configs/tutorials for it.

Backup on the same system is very insecure (think about a PSU that goes down the drain, and takes the system with it). However, backuppc is all you want/need, it's even in Synaptic.

How I do it? A cheap and small HD for the OS, and a (soft)RAID array of big ones for the data. If the system goes, you have your data, if the data goes (FS corruption) you can still use the system to check out what's going on.

 

1.) there's a tool called powertop -> http://www.lesswatts.org/projects/powertop/ that was quite useful to me

2.a) either cron + rsync
or 2.b) software raid

3.a) 1gb swap, 25gb linux (including logs and everything else), 224gb data.

2nd disk will be backup only? then i'd go for the software raid.

 

I wouldn't bother with raid. If you setup raid 0(stripped raid) your data will not be safe as if one disk drops then the other will have bits and pieces of data that when combined with disk one would make up a whole, so in essence becoming useless. If you go for raid 1(mirrored) then yes all the data will be backed up but if for any reason the disc fails due to software corruption then that corruption will be passed onto the mirrored disk in the setup as well so both discs could potentially be knackered.

I would say go with the 1gb swap partition, 25gb linux partition(even though this is very generous) and just have a data partition. Then setup a seperate partition on the second drive which is for backups only. Set the system up to backup the folders you want, pictures/movies/music etc at a certain time of night to the new partiton on the second drive.

Thats just my opinion but I don't really like raid 0/1. My server at the moment runs clark connect. This monitors all the power useage, statistics and such for you in a web interface. Also acts as a firewall for your network if you want it to. Have fun setting up the server though

 

RAID is not backup... RAID prevents data loss due to HW failure, backup prevents data loss in general... And RAID 0, isn't even RAID (no redundancy) and has no place in a server environment (look into RAID 5 for that)!

While that setup is viable, if your system gets fried, all data is lost... Backup on the same machine == bad idea.

 

Well seeing as how it is a home server I doubt that a spare machine specifically setup for backups is an option here.Yes, I know that raid 5 is the best option as you have the data spread across 3+ drives and have the parity allowing you to rebuild if a drive fails. I never said that raid was backup. It is all personal preference when it comes to raid and system setups. The bottom is how I have mine setup in a HOME environment and it works pretty well. If you want it backed up on a seperate entinty but don't want to build a new system, buy a external caddy and be done with it.

 

whoa! that's heaps of information! thanks guys. let's see:

I was about to install xubuntu (heard its lighter than ubuntu cuz of xface?) but again, i was confused with the partitioning. I'm used to windows partition but has no idea about linux. How should I partition my HDD? I start with a 250gb SATA. Should I:
- 19GB primary mount point /
- 230GB logical mount point ??
- 1GB swap
I got confused about primary/logical/mount point (/, /home, etc) please help.

as for the backup purpose, I was planning to add another 250gb to the system purely for backup. So I thought whatever I put on HDD1 will be scheduled to be copied (by backup apps) to HDD2, so when somethin happen to HDD1, I got a backup on HDD2. And yes, since this is a home file server, this is THE backup server for other computers. Having a spare machine for backup only is not an option i've got. This server meant to serve the other pc while backing up whatever data I put in it.

Not quite get it with the RAID. I guess I'll go for the backup software solution instead. Which from the mentioned is newbie-friendly? I just need em to be able to sync and backup my folders of choice.

 

You are mixing primary and logical partitions with mounts. You mount a partition (block device) somwhere (mountpoint).

I'd partition as following:

Code:

/boot hda1 250MB
Swap  hda2 2*RAM
/     hda3 10GB
/home hda4 rest

All being primary partitions.

Are all those folders on the server? Or are there remote systems involved? If the latter go for backuppc. If just local you can do it too (it has a great pooling filesystem), but a simple script that copys&tars the files is sufficient.

 

Is that much really necessary? I have 19 kernels and their configs in my /boot (as well as GRUB) and it doesn’t top 70MiB.

 

Debian kernels are quite big, certainly when they come with an initrd image. It goes up to 10-15MB for the kernel, initrd, config, system map...

On my Gentoo system the /boot partition is only 25MB but my kernels are 1/10th the size of the Debian ones

Of course, it can be smaller if wanted... but 250MB isn't really too much on a 250GB disk...

 

However you run the risk of going over the first 1024 cylinders, not a problem on his newer box (i doubt) but on older machines keeping /boot nice and small means you can still bot the box.

 

the idea of a raid(1) is to clone every bit thats written to a harddisk automatically to another disk. works as hardware or software implementation. you can easily set up a software raid in linux and don't have to care any more about copying data from disk1 to disk2 plus it will give you (at least theoretically) twice the performance when reading files. the advantage is that you won't feel the backup-process (as you would when you use some kind of backup software)

the other easy way to mirror your files from disk a to disk b is with rsync and cron.. when you're done installing your distribution of choice you'll find a directory called /etc/ with a file called crontab inside. in there you can create a rules that tell the cron-daemon to execute certain jobs in certain intervals. a line would look like this:

00 */2 * * * root /usr/local/bin/mail_poll

it would mean to execute "/usr/local/bin/mail_poll" as user "root" every 2 hours.
you could use the same technique to sync files from one disk to another

00 */2 * * * root rsync /data /backup

assuming /data is where your files are and backup is where your second hd is mounted

you might have to investigate a little deeper but thats more or less all that you'll need

 

However, the correct way to edit a crontab is with the crontab command (crontab -e)

 

I dont think on a modern distro a seperate /boot partition is needed.

On my home server which also has a 250gig hard drive

/ 10gig
/swap 1gig
/home all the emaining space.

The simplest was for a new linux guy to get involved with servers is to install a base server install and then install Webmin. Then you can install all your services from firefox.

 

I disagree... having a separate boot partition ensures 99% of the time that if you mess up the system you can easily fix it. But like all things in Linux, you have a free choice...

Ah good old Webmin... While it seems easy and dandy, it's about the biggest security risk you can have... And also, 99% of the fun of setting up a server are the things you learn from doing it hands down. You'll thank god that you did it yourself when Webmin acts up and you have to do it on your own.

 

some of the terms and CLI seems overwhelming for me. I'm new to linux, but thanks anyway, i'll try to follow up as much as I could.

/boot hda1 250MB
Swap hda2 2*RAM
/ hda3 10GB
/home hda4 rest

with this setup, does it means I've got 10gb to install apps? and then the rest of the disc for data?

yes, I meant to put all the other datas (photos/music/pics) in the server, which then will be accessed by other pcs (LAN only) on daily basis. The server serves another purpose of backin up all those files to the 2nd HDD (which installed in the same server as well)

tried to install xubuntu 8.04 last night but the screen freeze during partitioning process (it gets to 5% and that's it). isn't the 8.04 xubuntu stable yet? or I just did something wrong? anyway, will try out again tonight.

 

sounds like a dodgy disk, did you check the md5sum of the iso you downloaded?

 

not yet, totally forgot about it. i thought since i could burn the iso, its ok. it loads everything n all. but nvm, i'll just check the md5sum. thx

 

Also, do you know the SATA controller your board uses? There may be a bug somewhere.