1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Open Source I want to take my NAS to the next level

Discussion in 'Software' started by Cleggmeister, 7 Jan 2016.

Page 1 of 3
  1. Cleggmeister

    Cleggmeister Of reasonable knowledge...

    Joined:
    12 Oct 2009
    Posts:
    1,140
    Likes Received:
    22
    Greetings folks,

    Some 'strategic' advice please... For the past three years I've been running openmediavault on my N40L. It has served me well (pun intended) however I'd like to incorporate cloud access to my files and other cool stuff (as much for learning as for function).

    OMV relies on plugins to add these extra functions, such as dlna media streaming, print serving, tormenting, etc. All the plugins work nicely and are simple to install and administer. Which is kinda my problem. They're limited and I'd like to learn and engage more.

    I am very familiar with Synology, QNAP, etc and don't want to go down these routes as they're even more prescriptive and 'hands off'.

    Instead I have a notion to install Ubuntu server (or similar) on my N40L and really learn how to get under the bonnet. My question is, I guess, will this still allow me to serve music to my Linn HiFi, films to my Openelec Rpi2, pictured to my telly, torrents, etc, etc, etc. in a robust, straightforward and reliable way. And will I then be able to access my media remotely with something like OwnCloud or whatever? And will this scratch my itch with regards having more control and more choice and more education/engagement with the whole process?

    Any advice or thoughts most welcome!

    Many thanks in advance.

    Cleggy.
     
    Cleggmeister, 7 Jan 2016
    #1
  2. deathtaker27

    deathtaker27 Modder

    Joined:
    17 Apr 2010
    Posts:
    2,238
    Likes Received:
    186
    Sounds like a fun project, I'm about to do something similar as well so hopefully I can give you some advice, (My experiance is more with CENTOS but they are all very similar)

    Questions:
    1. How are you currently serving your media at home, is it just DNLA or is it a plex server?
    2. What other services are you running including backups

    I would install your prefered choice of linux distrubution, then install Samba for your file shares, a LAMP stack (Linux, apache, mysql/mariadb, php) and then install your owncloud instance, and any others you want.

    I find a lot of good tutorials on digitalocean for linux and once i know more I will be able to advice on the media streaming and other services,

    DT
     
    deathtaker27, 7 Jan 2016
    #2
  3. Gareth Halfacree

    Gareth Halfacree WIIGII! Lover of bit-tech Administrator Super Moderator Moderator

    Joined:
    4 Dec 2007
    Posts:
    17,132
    Likes Received:
    6,728
    I do pretty much exactly what you want to do, on an N54L running Ubuntu Server.

    File Sharing: SSH, 'cos all my clients are Linux (aside from one OS X machine.)
    Media Sharing: MiniDLNA (UPnP-compatible streaming server, works with everything.)
    Backups: CrashPlan.
    Proxies: Privoxy (strips advertising) and Polipo (caches), for the mobile clients.
    Remote access: SSH again.
    File Synchronisation: Syncthing.
    DHCP: ISC DHCP server (pooled with a second instance running on a Raspberry Pi.)

    Data is stored on a two-drive 2TB Btrfs mirror, compressed, and backed up to a single 6TB drive plus CrashPlan's cloud servers. The OS and web cache are stored on the 250GB drive that came free with the server.

    Works for me. It'd be easy to install Samba if you wanted to easily share files with Windows clients (which I don't need to do), and if you want to use OwnCloud you can install that pretty easily too. Unlike deathtaker, tho', I wouldn't recommend installing a LAMP stack unless you actually *need* a LAMP stack (which OwnCloud does, so if you want to use OwnCloud ignore this advice 'cos step one of installing OwnCloud is to set up a web server) - basically, do you want to serve web pages? If not, don't install a LAMP stack: it just bogs down the server and increases the attack surface for no good reason.
     
    Gareth Halfacree, 8 Jan 2016
    #3
  4. marlowdrummer

    marlowdrummer Minimodder

    Joined:
    24 Nov 2008
    Posts:
    173
    Likes Received:
    6
    Hi,
    I am doing similar to Gareth, but a lot simpler :)
    I have my N40 running CentOS7 with ZFS and 3x2Tb drives (not good as it breaks the ZFS on every update...).
    Have Plex server and it backs up my NAS box via R-Sync (PC's back up to NAS which is RAID 1).
    Files shared out via Samba for tablets/laptops to access or play music/video.
    Also stuck a 4 port GB NIC in and set NIC teaming/failover.
    Been solid. Only the ZFS has been an issue.

    Go for it. I learned a shed load when I set mine up and it was fun.
    All the best
    S.
     
    marlowdrummer, 8 Jan 2016
    #4
  5. nimbu

    nimbu Multimodder

    Joined:
    28 Nov 2002
    Posts:
    2,596
    Likes Received:
    283
    nimbu, 8 Jan 2016
    #5
  6. David

    David μoʍ ɼouმ qᴉq λon ƨbԍuq ϝʁλᴉuმ ϝo ʁԍɑq ϝμᴉƨ

    Joined:
    7 Apr 2009
    Posts:
    17,460
    Likes Received:
    5,870
    Isn't there an OwnCloud plugin for OMV? I could've sworn I saw it on my plugin list.
     
    Last edited: 8 Jan 2016
    David, 8 Jan 2016
    #6
  7. phuzz

    phuzz This is a title

    Joined:
    28 May 2004
    Posts:
    1,712
    Likes Received:
    27
    Open Media Vault is Debian based, so everything that works on OMV will work on Ubuntu as well with a bit more work.
    It might be worth setting up a VM and playing around with Ubuntu on that before you take the plunge and reinstall your server. VirtualBox is free and runs on most OSs and is pretty straight forward to use.
     
    phuzz, 8 Jan 2016
    #7
  8. Cleggmeister

    Cleggmeister Of reasonable knowledge...

    Joined:
    12 Oct 2009
    Posts:
    1,140
    Likes Received:
    22
    Cheers chaps, really good food for thought. I'll give this some time when I'm back home. There is a own cloud plugin for OMV but its a nightmare to get working. I've tried and failed many times!

    Thanks again, will report back next weekend.
     
    Cleggmeister, 9 Jan 2016
    #8
  9. Cleggmeister

    Cleggmeister Of reasonable knowledge...

    Joined:
    12 Oct 2009
    Posts:
    1,140
    Likes Received:
    22
    A little bit rude of me for not replying to DT's post; here goes...

    For music streaming I use Asset Upnp. It's a CLI installation rather than an OMV plugin but its the best dlna media server (music only) that I've seen. I don't use Plex as, for music, it doesn't offer any more than asset, minim, Twonky et al. I did use minidlna for a few years but its not compatible with some features of my latest upnp control point (Linn Kazoo)

    Video serving just uses SMB to the RPI.

    Services running include transmission and cups. Backups I do manually by just copying and pasting everything to large USB drives every few months. Hardly cutting edge I know, but this started life as a music NAS and hasn't really changed much. :)

     
    Cleggmeister, 10 Jan 2016
    #9
  10. Pliqu3011

    Pliqu3011 all flowers in time bend towards the sun

    Joined:
    8 Aug 2009
    Posts:
    2,736
    Likes Received:
    257
    My little server runs FreeNAS with the ZFS file system (which in retrospect probably was overkill for my needs). Runs Plex, minidlna, ownCloud, CIFS shares, a small Minecraft server, and some other smaller stuff. Remote access with SSH and IPMI is super nice.
    Setting up the OS and some of the plugins was a lot of work (you better love using a CLI!), but it really was a learning experience for me.

    I don't know if other operating systems have this too but especially FreeNAS/FreeBSD's "jails" are a really nice feature for tinkering. They're basically "mini-OSs" separate from the main OS with their own hard drive structure, IP address and everything, in which you install and run your applications. So even in case you completely mess up, you can just delete the jail and start over again, without the chance of ruining something else.
    I always use this when upgrading my plugins, or the time I changed OwnCloud's DBMS from the standard SQLite to MySQL. Normally I'd be anxious of breaking everything, but here I just create a new jail, install the application, couple it to (a part of) my data pool and run it alongside the old jail for a while to see if runs as it should. If it does, I delete the old jail, if it doesn't I can just delete the new jail and decide I didn't really need the upgrade anyway because the old version worked just fine. ;) (or try again of course)

    I haven't used any other server OSs so I can't compare, but if choice and control is what you're looking for FreeNAS is maybe also worth looking into.
     
    Last edited: 10 Jan 2016
    Pliqu3011, 10 Jan 2016
    #10
  11. Cleggmeister

    Cleggmeister Of reasonable knowledge...

    Joined:
    12 Oct 2009
    Posts:
    1,140
    Likes Received:
    22
    OK, back home after a lovely short break at Centre Parcs. Downloading Ubuntu Server as I type :)

    I've decided on US simply because of it's "blank slate". I looked at Amahi and a couple of other things however as well as being functional this project is meant to be fun and educational for me too.

    So, first question. My N40L has four drive bays, all populated with drives of varying sizes/specs (Music, Movies, Backup and Data respectively). I currently run OMV (the operating system) from a USB pen drive. Am I OK to run Ubuntu Server from a pen drive in a similar manner? I know you hear horror stories about them failing after so many write cycles however I've been running from USB pen drives for nearly four years and never missed a beat...

    No worries if not, but I will need to hack the N40L to accommodate another SATA drive, which in turn will delay things a while...
     
    Cleggmeister, 11 Jan 2016
    #11
  12. Cleggmeister

    Cleggmeister Of reasonable knowledge...

    Joined:
    12 Oct 2009
    Posts:
    1,140
    Likes Received:
    22

    Many thanks for the pointer. I did look at FN years ago but the system requirements at the time were bonkers. That said it does seem comparable to OMV hence I'll swerve just now and go straight for the full blown server OS. How's that for enthusiasm over ability?! :)
     
    Cleggmeister, 11 Jan 2016
    #12
  13. Cleggmeister

    Cleggmeister Of reasonable knowledge...

    Joined:
    12 Oct 2009
    Posts:
    1,140
    Likes Received:
    22
    Hmmmm, just caught up with the whole LAMP thingy. OK, here's where I'm at...

    I'd really like to access my files remotely, hence my penchant for OwnCloud. However I also want my server to be secure, low footprint, low energy, low stress, etc...

    Is there an alternative that'll give me remote access without a heavyweight infrastructure running underneath?
     
    Cleggmeister, 11 Jan 2016
    #13
  14. phuzz

    phuzz This is a title

    Joined:
    28 May 2004
    Posts:
    1,712
    Likes Received:
    27
    If you've been using the same USB stuck as an OS drive, maybe it's time to treat yourself to a new one for a new OS, they're pretty cheap now (plus you can roll back to the old OS just by using the old stick).
    As for remote access to your files, there's a few options; there's FTP of course, or BitTorrent sync, it depends a bit on what you need. Owncloud isn't too heavy weight though I think.
     
    phuzz, 13 Jan 2016
    #14
  15. Cleggmeister

    Cleggmeister Of reasonable knowledge...

    Joined:
    12 Oct 2009
    Posts:
    1,140
    Likes Received:
    22
    That sounds like a very good plan. I've ordered a 32GB stick which will be plenty for Ubuntu Server (which I think I've decided upon).

    Since all 4 of my data drives are configured with EXT4 should I simply be able to mount these under the new OS and see the same folder and files within? I have them all backed up but it will be preferable to just mount them for use, rather than formatting, partitioning then restoring thw data (around 7TB) from a backup.

    Many thanks.

    Cleggy
     
    Cleggmeister, 14 Jan 2016
    #15
  16. Gareth Halfacree

    Gareth Halfacree WIIGII! Lover of bit-tech Administrator Super Moderator Moderator

    Joined:
    4 Dec 2007
    Posts:
    17,132
    Likes Received:
    6,728
    Yup: Ubuntu will mount ext4 with no problem (as will any other *NIX). The only thing to be very, very careful with if you're planning to use a USB stick as your OS drive is killing it through excessive writes. USB flash drives are *not* built to the same standard as SSDs: I killed a Kingston 4GB once simply by using it as /var, not even for swap. Now when you plug it in, it reckons it's a 4MB drive...

    Yes, there are NAS operating systems (like FreeNAS) which run fine on a USB drive; these are typically tweaked in a variety of ways to make that work, such as disabling swap and sticking various things in RAM instead of on-disk. You'll need to do those tweaks yourself manually if you're installing Ubuntu Server.

    Oh, and I've just made a few changes to my own server: installed arpwatch and set up zram. The former means I get an email every time there's a new MAC address on my network, which is handy when I'm testing hardware 'cos the instant I fire up a new box I get an email telling me what its IP address is - and it also gives me confidence that a neighbour hasn't piggybacked onto my Wi-Fi or Powerline networks. The latter takes half my physical RAM and creates compressed RAM drives, which are then used as swap - meaning when the server runs out of RAM and tries to swap stuff out, instead of writing to disk it quickly compresses it and keeps it in RAM. If it runs out of compressed RAM, too, then naturally stuff is written to disk. It's not that useful on a system with lots of RAM, but on my 4GB server it helps a fair bit when things like CrashPlan start chewing up resources (and cheers for that, Java...)
     
    Gareth Halfacree, 15 Jan 2016
    #16
  17. phuzz

    phuzz This is a title

    Joined:
    28 May 2004
    Posts:
    1,712
    Likes Received:
    27
    Wait, so to avoid the problem of you machine running out of RAM and using swap, you've reduced the amount of RAM?!
    I'm not saying it wouldn't work, it just seems a bit counter-intuitive.
     
    phuzz, 15 Jan 2016
    #17
  18. Gareth Halfacree

    Gareth Halfacree WIIGII! Lover of bit-tech Administrator Super Moderator Moderator

    Joined:
    4 Dec 2007
    Posts:
    17,132
    Likes Received:
    6,728
    Yup!

    Incidentally, it's basically exactly what Apple has done with El Capitan: introduced a chunk of compressed swap in RAM. Except, of course, when zram was launched nobody noticed but when Apple nicked it it was THE BEST THING EVAR.

    It works nicely: the data is still in RAM, but compressed with - on average - a 2:1 ratio. So, I've traded a little latency and CPU time for having the ability to shove around 6GB of working data in my 4GB of RAM. Add to that the fact that only the older, haven't used it in a while data gets compressed, and you've got a neat little system - with the added bonus that the hard drive doesn't spin up as much any more. Oh, and it also means that old stuff shoved in my /tmp directory (mounted in RAM as tmpfs) also gets compressed without me having to do anything. Lovely!

    Bloody easy to do, too: apt-get install zram-config. Done. It's active and working, using half your RAM as compressed swap split into as many chunks as you have CPU cores (so it uses a multi-core processor efficiently, running multiple compression/decompression threads). You can temporarily disable it with service zram-config stop, or get rid entirely with apt-get remove zram-config. Bosh.

    EDIT: Just realised I forgot to mention something important: the zram devices only take up space when there's something in them. They don't take up 2GB of RAM unless there's 4GB-ish of data that already needs to be swapped out. (Well, they *do* take up *some* space when empty - about 1 per cent of the defined size.)
     
    Last edited: 16 Jan 2016
    Gareth Halfacree, 15 Jan 2016
    #18
  19. theshadow2001

    theshadow2001 [DELETE] means [DELETE]

    Joined:
    3 May 2012
    Posts:
    5,284
    Likes Received:
    183
    Ah, so basically you've enabled compression on half of your RAM. That's another one of your computing things that took me a minute to get.
     
    theshadow2001, 16 Jan 2016
    #19
  20. Cleggmeister

    Cleggmeister Of reasonable knowledge...

    Joined:
    12 Oct 2009
    Posts:
    1,140
    Likes Received:
    22
    Thanks chaps, informative and thoughtful as always here! :)

    Just looked at the cost of a 32GB SSD on Amazon. C£25 for a decent drive; wow, these have come down a bit!

    Is 32GB enough for a Ubuntu Server install, allowing for apps, cache, etc?

    Many thanks.

    Cleggy.
     
    Cleggmeister, 17 Jan 2016
    #20
Page 1 of 3

Share This Page