Howdy folks I wasn't sure exactly where this thread should site but I've ended up with Storage in Hardware. It will cover lots more and I'll be looking for advice on a lot more but here it is for now. My homelab has never been anything more than a dodgy work-in-progress. The hardware is split across 2 rooms and in a constant state of change. I want to fix that. We are also looking to move house this year (been house hunting for about a year already) and so I want to overhaul it so that when that does eventually happen, it's easy to move and is plug and play at the new house. I have 2 high level goals: - everything running in one rack where possible (should be everything except modem) - spend as little as possible in the process In the next post I'll detail everything I have currently and my full requirements, then I can dive in to the endless knowledge of the bit-tech hive mind to guide me.
The current homelab As previously mentioned I have everything split between two rooms. In the living room I have the following: Modem (FTTP) Opnsense router - 4x2.5GbE mini PC Asus GS-AX3000 in AP mode for Wifi 2 x minisforum mini PCs (UM 560 XT and UM450, both with 64GB RAM) - Proxmox Pi 4 running home assistant 8 port smart managed switch from Netgear Hive hub UPS Nvidia Shield - included as it's my current plex server PXL_20240104_145817465 by meandmymouth posted 7 Mar 2024 at 09:02 I have 2 cat5e cables that run on the outside of the house up to the spare room/office where I have the rest of the hardware, which is as follows: TrueNAS server (v3 Xeon, Asrock board, 32GB ECC, 4x8TB HDD, 2x1TB SSD, 4x1GbE NIC) Synology DS923+ (4x6TB HDDs and 1 SSD cache - picture below has old DS416play that I though was dead). Pi 4 clusters with PoE switch 16 port Netgear smart switch All stored in a lack rack and with the rest of my tech pile - desktop, work laptop, various hard drives etc. PXL_20240104_145252434 by meandmymouth posted 7 Mar 2024 at 09:02 There has also been a couple of new additions this week (I can't help myself apparently) which are a TP-LINK Omada EAP610. This will support the use of multiple networks/VLANs Minisforum UN100L mini PC - N100 CPU intended to replace Plex and use Jellyfin for media, also replace the pi4 for running home assistant. The Plan Ultimately I want to make it all less janky (particularly the living room set up) and have the confidence to run my own services and ditch some cloud services (Nextcloud replacing Goolge being the main one and being able to offer this to family members as well). What I'm currently thinking is consolidating a little and moving everything to the office with the exception of the modem - that has to stay put in the living room. I think the UPS will squeeze in behind my truesnas server and could then serve truenas, synology, a single switch, pi cluster and the 3 mini PCs. A proper rack would be nice as would a decent 24 port switch with 2.5GbE and 10Gb (Ethernet or SFP+). In theory I could do that with this rack frame and this ubiquiti switch. I could probably do with a better UPS as well... I did have grand plans to upgrade truenas server and through in some GPU horsepower for some home AI workloads (ollama as well as voice and video recognition in home assistant) but I don't think that is sensible. I could go in to a lot more detail but it's already a pretty long post. So, thoughts?
Here's my thoughts so far (been on my mind a lot this morning): Get a small network cabinet (6U probably) to place on top of the lack rack. In here can go the ubiquiti switch that I'm considering, the router and the 3 mini PCs (two of which are 2.5GbE, hence wanting a switch with 2.5GbE). I may also add the pi cluster in there as well. I'd then only need to get a couple of 1U shelves for those. The truenas host and synology NAS can stay put 'in' the lack rack. I would like to upgrade my truenas host so it can be the full storage backend for some VMs running in proxmox on the mini PCs instead of just mapping NFS/SMB shares in the VMs as I do currently. That's a rabbit hole I don't need to go down yet. You might be wondering why I don't just go for a proper rack and my thinking there is in a future house move. It will be a lot easier if the rack essentially splits in 2 and I wouldn't even have to remove anything from the network cabinet. This feels like the best way forward for a 'phase 1' of the homelab project.
After thinking about this for the last few days I have come to the conclusion that I am over-thinking it. I can consolidate and make everything a bit less sketchy just by buying another UPS and tidying and rearrange everything a little bit. The truly terrible diagram below shows what I think will work. Home-network.drawio by meandmymouth posted 14 Mar 2024 at 09:34 Main changes are as follows: Moving the Mini PCs (1 and 2) up to the spare room and putting them in the lack rack. Setting up the new mini PC (N100 PC) in the living room Adding a UPS to the rack in the space room From a hardware POV nothing else is needed. The 16 and 8 port switches I have are both smart managed so will support VLANs so that I can segregate my network properly. They are only 1GbE switches so I'll just have to live with that for now (boo hoo, poor me). Back ups of VMs and replication between truenas and synology will happen overnight so it can saturate the network then. This would all still be literally lift and shift-able when moving house and just means I have a network wall in the living room (similar to as it is now) instead of a network cabinet on the rack. I have an ikea pegboard that can replace the plank of wood to make it a little nicer. As I write this I'm thinking all add a couple small upgrades - 32GB ECC to the Synology box and start the replacement of the 6TB drives in it, which can be done over time. Other than that, this all feels very sensible now and tbh I don't like it.
Question. Is this homelab or homeProd? I'd ditch the unmanaged switch. I've found these cause more issues when you're using vlan's than they fix.
Good diagram, what software did you use? Also, am I reading it correctly? There's 4 cables from TrueNAS to the 24 port switch?
Thanks! I used draw.io - free to use and you can run it locally. This was thrown together pretty quick as well. As for the cables, you are correct. 3 for data and 1 for the motherboard BMC. I have some apps running so I'll assign one interface to those and the LAGG the other 2 for the SMB/NFS shares. That's the idea at lease.
Over the weekend I manage to install a new UPS in the spare room so now both 'racks' are covered in power loss events. I also moved hardware to match up with the diagram I posted. I haven't configured everything, mainly in the living room, but all the hardware is at least in the room it's supposed to be now. Next is the fun bit where it could all go terribly wrong - network configuration.
Little update and change to the plan. I've managed to pick up 3 16TB EXOS drives (refurbs, but 0 hours) and since my old Synology actually still works with a new power supply this provides an opportunity for an offsite backup. So new DS923+ will have the 3x 16TB drives, Synology Hybrid Raid so I can either increase size or redundancy of the volume in the future with an extra disk. TrueNAS server to get an extra 8TB drive so that will be 5x8TB in RAIDZ1 to match the capacity of the Synology above. Each will replicate data from the other and will be running their own set of services. Old DS416Play will get back the 4x6TB drives in either 3 disk RAID 1 with a hot spare or 4 disk RAID 2 (a couple of the disks have about 50k hours and the other 2 have 25k so mindful of 2 failing close together). Doesn't need as much storage as I won't be backing up TBs of media. This will then give me my 3-2-1 backup (well, more like 3-1-1 since I'm using spinning rust for everything) and enough storage flexibility to get various family members backing up there stuff as well. I think the Synology software will be ideal for that. All the REALLY important stuff will also be backed up to backblaze or something like that. Organising and configuring all of this will also encourage me to continue the clean up of old back ups and and lots of other stuff that I don't need, likely clearing lots of space in the interim. I do wonder if the truenas host is overkill, however (i.e. is it needed at all?). That has got me wondering... I also wonder if I should keep all 3 mini PCs in their current configuration. Between the newer N100 mini PC, the new Synology and TrueNAS system that is more than enough to run anything I want between them, with the exception of local AI... let's not go too far down that rabbit hole just yet.
I see you found the EXOS recerts 16tbs for around £180 on Amazon! Banging value with the warranty from Seagate too! You planning on mini racking?
Not sure what you mean by mini racking? The drives are even cheaper if you choose Amazon EU as the seller, closer to £160 each.
Wheter you plan on installing in a small 4-9U rack or softs, a little like the one Ive picked up or a few others on here have shown off recently. Yea, I had a quick look again and saw the Amazon EU shipped product. Absolute bargains to be fair. I ended up picking up 4 of them to use and to replace my 3x WD red 8TBs I had. this didnt work as I now have 88TB raw storage, why not eh! I know people screw their face up at recerts HOWEVER, after selling over 600,000 HDDs over the years, I can say with absolute certainty that the recerts are more reliable than a standalone standard drives and rarely if ever see an RMA.
This is very interesting and comforting information on these drives. As for racking, just sticking with the Ikea lack rack that I've built, as shown in one of the photos above.
I'm still toying with the idea of replacing the motherboard and CPU in my truenas host and using my main PC instead (3950X). I'd then run my main PC as a VM with the 1080 passed through. I think I would have to swap to proxmox and virtualise truenas as well but that's no biggy. The main thing stopping me is finding an appropriate mATX motherboard that will support ECC (albeit maybe not full ECC with the 3950X).
