1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Storage Homelab Project 2024

Discussion in 'Hardware' started by meandmymouth, 6 Mar 2024.

  1. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    Howdy folks

    I wasn't sure exactly where this thread should site but I've ended up with Storage in Hardware. It will cover lots more and I'll be looking for advice on a lot more but here it is for now.

    My homelab has never been anything more than a dodgy work-in-progress. The hardware is split across 2 rooms and in a constant state of change. I want to fix that.

    We are also looking to move house this year (been house hunting for about a year already) and so I want to overhaul it so that when that does eventually happen, it's easy to move and is plug and play at the new house.

    I have 2 high level goals:
    - everything running in one rack where possible (should be everything except modem)
    - spend as little as possible in the process

    In the next post I'll detail everything I have currently and my full requirements, then I can dive in to the endless knowledge of the bit-tech hive mind to guide me.
     
    Byron C likes this.
  2. RedFlames

    RedFlames ...is not a Belgian football team

    Joined:
    23 Apr 2009
    Posts:
    15,686
    Likes Received:
    3,163
    Isn't that every homelab ever?
     
  3. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    Well yes, so I'd like to remove the 'dodgy' part of it.
     
  4. David

    David μoʍ ɼouმ qᴉq λon ƨbԍuq ϝʁλᴉuმ ϝo ʁԍɑq ϝμᴉƨ

    Joined:
    7 Apr 2009
    Posts:
    17,743
    Likes Received:
    6,275
    IKEA lack tables and some acrylic sheeting. :thumb:
     
  5. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    The lack table is one of the dodgy parts I'd like to remove.
     
    David likes this.
  6. David

    David μoʍ ɼouმ qᴉq λon ƨbԍuq ϝʁλᴉuმ ϝo ʁԍɑq ϝμᴉƨ

    Joined:
    7 Apr 2009
    Posts:
    17,743
    Likes Received:
    6,275
    I was focusing on your second goal :lol:
     
    meandmymouth likes this.
  7. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    The current homelab

    As previously mentioned I have everything split between two rooms.
    In the living room I have the following:
    • Modem (FTTP)
    • Opnsense router - 4x2.5GbE mini PC
    • Asus GS-AX3000 in AP mode for Wifi
    • 2 x minisforum mini PCs (UM 560 XT and UM450, both with 64GB RAM) - Proxmox
    • Pi 4 running home assistant
    • 8 port smart managed switch from Netgear
    • Hive hub
    • UPS
    • Nvidia Shield - included as it's my current plex server

    I have 2 cat5e cables that run on the outside of the house up to the spare room/office where I have the rest of the hardware, which is as follows:
    • TrueNAS server (v3 Xeon, Asrock board, 32GB ECC, 4x8TB HDD, 2x1TB SSD, 4x1GbE NIC)
    • Synology DS923+ (4x6TB HDDs and 1 SSD cache - picture below has old DS416play that I though was dead).
    • Pi 4 clusters with PoE switch
    • 16 port Netgear smart switch
    All stored in a lack rack and with the rest of my tech pile - desktop, work laptop, various hard drives etc.


    There has also been a couple of new additions this week (I can't help myself apparently) which are a TP-LINK Omada EAP610. This will support the use of multiple networks/VLANs

    Minisforum UN100L mini PC - N100 CPU intended to replace Plex and use Jellyfin for media, also replace the pi4 for running home assistant.


    The Plan
    Ultimately I want to make it all less janky (particularly the living room set up) and have the confidence to run my own services and ditch some cloud services (Nextcloud replacing Goolge being the main one and being able to offer this to family members as well).

    What I'm currently thinking is consolidating a little and moving everything to the office with the exception of the modem - that has to stay put in the living room.
    I think the UPS will squeeze in behind my truesnas server and could then serve truenas, synology, a single switch, pi cluster and the 3 mini PCs.

    A proper rack would be nice as would a decent 24 port switch with 2.5GbE and 10Gb (Ethernet or SFP+). In theory I could do that with this rack frame and this ubiquiti switch. I could probably do with a better UPS as well...

    I did have grand plans to upgrade truenas server and through in some GPU horsepower for some home AI workloads (ollama as well as voice and video recognition in home assistant) but I don't think that is sensible.

    I could go in to a lot more detail but it's already a pretty long post.

    So, thoughts?
     
    Byron C likes this.
  8. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    Here's my thoughts so far (been on my mind a lot this morning):

    Get a small network cabinet (6U probably) to place on top of the lack rack. In here can go the ubiquiti switch that I'm considering, the router and the 3 mini PCs (two of which are 2.5GbE, hence wanting a switch with 2.5GbE). I may also add the pi cluster in there as well. I'd then only need to get a couple of 1U shelves for those.
    The truenas host and synology NAS can stay put 'in' the lack rack. I would like to upgrade my truenas host so it can be the full storage backend for some VMs running in proxmox on the mini PCs instead of just mapping NFS/SMB shares in the VMs as I do currently. That's a rabbit hole I don't need to go down yet.

    You might be wondering why I don't just go for a proper rack and my thinking there is in a future house move. It will be a lot easier if the rack essentially splits in 2 and I wouldn't even have to remove anything from the network cabinet.

    This feels like the best way forward for a 'phase 1' of the homelab project.
     
    Byron C and IanW like this.
  9. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    After thinking about this for the last few days I have come to the conclusion that I am over-thinking it. I can consolidate and make everything a bit less sketchy just by buying another UPS and tidying and rearrange everything a little bit. The truly terrible diagram below shows what I think will work.


    Main changes are as follows:
    Moving the Mini PCs (1 and 2) up to the spare room and putting them in the lack rack.
    Setting up the new mini PC (N100 PC) in the living room
    Adding a UPS to the rack in the space room

    From a hardware POV nothing else is needed. The 16 and 8 port switches I have are both smart managed so will support VLANs so that I can segregate my network properly. They are only 1GbE switches so I'll just have to live with that for now (boo hoo, poor me). Back ups of VMs and replication between truenas and synology will happen overnight so it can saturate the network then.

    This would all still be literally lift and shift-able when moving house and just means I have a network wall in the living room (similar to as it is now) instead of a network cabinet on the rack. I have an ikea pegboard that can replace the plank of wood to make it a little nicer.

    As I write this I'm thinking all add a couple small upgrades - 32GB ECC to the Synology box and start the replacement of the 6TB drives in it, which can be done over time. Other than that, this all feels very sensible now and tbh I don't like it. :grin:
     
    Byron C likes this.
  10. saspro

    saspro IT monkey

    Joined:
    23 Apr 2009
    Posts:
    9,622
    Likes Received:
    415
    Question. Is this homelab or homeProd?

    I'd ditch the unmanaged switch. I've found these cause more issues when you're using vlan's than they fix.
     
  11. wyx087

    wyx087 Multimodder

    Joined:
    15 Aug 2007
    Posts:
    12,084
    Likes Received:
    753
    Good diagram, what software did you use?

    Also, am I reading it correctly? There's 4 cables from TrueNAS to the 24 port switch?
     
  12. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    Thanks! I used draw.io - free to use and you can run it locally. This was thrown together pretty quick as well.

    As for the cables, you are correct. 3 for data and 1 for the motherboard BMC. I have some apps running so I'll assign one interface to those and the LAGG the other 2 for the SMB/NFS shares. That's the idea at lease.
     
    wyx087 likes this.
  13. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    Over the weekend I manage to install a new UPS in the spare room so now both 'racks' are covered in power loss events. I also moved hardware to match up with the diagram I posted. I haven't configured everything, mainly in the living room, but all the hardware is at least in the room it's supposed to be now.

    Next is the fun bit where it could all go terribly wrong - network configuration.
     
  14. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    Little update and change to the plan. I've managed to pick up 3 16TB EXOS drives (refurbs, but 0 hours) and since my old Synology actually still works with a new power supply this provides an opportunity for an offsite backup.

    So new DS923+ will have the 3x 16TB drives, Synology Hybrid Raid so I can either increase size or redundancy of the volume in the future with an extra disk.

    TrueNAS server to get an extra 8TB drive so that will be 5x8TB in RAIDZ1 to match the capacity of the Synology above. Each will replicate data from the other and will be running their own set of services.

    Old DS416Play will get back the 4x6TB drives in either 3 disk RAID 1 with a hot spare or 4 disk RAID 2 (a couple of the disks have about 50k hours and the other 2 have 25k so mindful of 2 failing close together). Doesn't need as much storage as I won't be backing up TBs of media.

    This will then give me my 3-2-1 backup (well, more like 3-1-1 since I'm using spinning rust for everything) and enough storage flexibility to get various family members backing up there stuff as well. I think the Synology software will be ideal for that. All the REALLY important stuff will also be backed up to backblaze or something like that.

    Organising and configuring all of this will also encourage me to continue the clean up of old back ups and and lots of other stuff that I don't need, likely clearing lots of space in the interim.

    I do wonder if the truenas host is overkill, however (i.e. is it needed at all?). That has got me wondering...
    I also wonder if I should keep all 3 mini PCs in their current configuration. Between the newer N100 mini PC, the new Synology and TrueNAS system that is more than enough to run anything I want between them, with the exception of local AI... let's not go too far down that rabbit hole just yet.
     
    Byron C, andrew8200m and IanW like this.
  15. andrew8200m

    andrew8200m Multimodder

    Joined:
    4 May 2009
    Posts:
    2,672
    Likes Received:
    321
    I see you found the EXOS recerts 16tbs for around £180 on Amazon! Banging value with the warranty from Seagate too!

    You planning on mini racking?
     
  16. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    Not sure what you mean by mini racking?

    The drives are even cheaper if you choose Amazon EU as the seller, closer to £160 each.
     
  17. andrew8200m

    andrew8200m Multimodder

    Joined:
    4 May 2009
    Posts:
    2,672
    Likes Received:
    321
    Wheter you plan on installing in a small 4-9U rack or softs, a little like the one Ive picked up or a few others on here have shown off recently.

    Yea, I had a quick look again and saw the Amazon EU shipped product. Absolute bargains to be fair. I ended up picking up 4 of them to use and to replace my 3x WD red 8TBs I had. this didnt work as I now have 88TB raw storage, why not eh! I know people screw their face up at recerts HOWEVER, after selling over 600,000 HDDs over the years, I can say with absolute certainty that the recerts are more reliable than a standalone standard drives and rarely if ever see an RMA.
     
  18. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    This is very interesting and comforting information on these drives.
    As for racking, just sticking with the Ikea lack rack that I've built, as shown in one of the photos above.
     
  19. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    I'm still toying with the idea of replacing the motherboard and CPU in my truenas host and using my main PC instead (3950X). I'd then run my main PC as a VM with the 1080 passed through. I think I would have to swap to proxmox and virtualise truenas as well but that's no biggy. The main thing stopping me is finding an appropriate mATX motherboard that will support ECC (albeit maybe not full ECC with the 3950X).
     
  20. meandmymouth

    meandmymouth Multimodder

    Joined:
    15 Sep 2009
    Posts:
    4,344
    Likes Received:
    361
    After a bit of ebay hunting with a voucher the TrueNAS host is getting an upgrade.
    Ryzen 5 Pro 5650G and Asrock B550M Pro 4 on their way.
    I'm going to rebuild the whole rig I think and it'll fully look like this:
    5650G (6 core/12 thread)
    64GB RAM (maybe ECC, maybe not - DS923 has it so less worried on this machine, 128GB in the future)
    5x8TB HDDs RAIDZ1 with cold spare
    3x1TB SATA SSDs RAIDZ1 (these and the HDDs will fill out the hot swap bays)
    2xSATA SSDs for boot drives (had this before and both failed at the same time - couldn't believe it).
    8 port SATA card (PCIex1 - to purchase)
    4 port NIC (PCIex4)

    This leaves two m.2 slots on the motherboard and a full PCIe x16 slot free. I could add NVMe drives in the future for either another pool or SLOG for the other pools. Keeping the x16 slot free in the hope that one day I can afford some sort of RTX 4000 card. That would be pretty beastly in a relatively small package that is the Silverstone CS381.

    Between that and the DS923+ the mini PCs I have are almost redundant...
     
    Byron C likes this.

Share This Page