Multi CPU rig (for bigadv-16)

You guys have such nice hardware, +rep to you all

 

And to think that 40kppd got me into the top 20, for a short while !!
Well done you guys, keep it up

 

Christopher - amazing, wish I could afford to make a rig similar to that

Did you purchase the bits new and could we have some photos and an idea of cost please.

 

Here's a parts list for those who are interested. I've included some comments about why I chose them; my main desire was to get a quiet and energy efficient machine. In retrospect there may be some better decisions I could have made.
About prices - I'm not going to put up exact numbers because I don't want to know the total of how much I've spent! The purchasing was spread over several months, and I made good use of 'daily deals', Google Shopping and eBay. There were some items I bought which didn't get into the final build, and will be going onto eBay when I find the time. I like to think I've spent significantly less than DocJonz, but then he has the security of support and warranties (I hope).

OS - Ubuntu 10.10 - FREE
Folding application 6.34 - FREE
Installation using Musky's instructions, and including The Kraken and Langouste (for higher actual PPD).

HHD - A spare Seagate I had. 80GB is plenty of room for a dual boot Ubuntu/Windows system.
PSU - Corsair AX 750 (new). I wanted a high efficiency, modular PSU with two 8 pin EPS connectors and thought this would be about right for running at 50% load. It is very quiet. For a while I was folding on only 2 CPUs, and the PSU was so cool that the fan would stop spinning. A heart-stopping moment the first time it happened, until I went and read the specs more carefully.
Board - Supermicro H8QGi+-F (new). This is the single most expensive part of the system. There aren't very many quad socket, G34 boards around, and this had a best-looking layout. This is the entry-level board of the family and doesn't have energy consuming controllers I wouldn't need, such as SAS. I'm hopeful that AMD won't change their sockets for a few years (unlike some other companies) so that the board will have a long and useful life.
UPS - was in my first plan, in case of power cuts, but uses a surprisingly high amount of power, even on standby. This was a case of cost-of-definitely-running-all-the-time vs. very-unlikely-loss-of-points, so the idea was dropped.

Now things get a bit pricier because you need 4 of everything....
Heat sink and fan - Noctua NH-U9DO A3 (new). There aren't many coolers that fit G34 out-of-the-box and I didn't want to do a mod on such an important component. Surprisingly hard to find to purchase. I got one 12cm model to try, but it was just too big and heavy for comfort. The heatsinks come with two fans each, but the moment I'm only using one. Eventually I hope use the 'noise reducing' resistors, but I need to get the machine cased and measure the temperatures first.

Memory - A difficult choice (new, but on eBay). I don't need expensive server-grade memory. I wondered about using only two sticks per socket (to save energy) and did some experiments, which confirmed that running with four sticks is significantly faster. Given the non-linear Quick Return Bonus formula, I calculated that four sticks per socket would give better PPD/W even though the total wattage would be higher than with two sticks.
Most people seem to use G.SKILL Ripjaws X Series DDR3 1333 Desktop Memory but I discovered Samsung 1.35v 30nm "Green" memory and thinking that smaller process and lower voltage means less power, I went for these. The smallest size was 2GB, so I ended up with 32GB of which only 5GB is used. I'm not going to buy a set of Ripjaws to compare PPD/W but if someone was prepared to lend me a set it would be an interesting comparison.

CPU - AMD 6166HE (eBay). Actually the easiest component to choose. 6100 series Opterons are more energy efficient for the same clock speed than 6200 series (just like the desktop Bulldozer CPUs). The 6166s are the fastest twelve-cores in the low power bracket, and at the time were selling on eBay for reasonable prices. I wouldn't have gone ahead with this build if I had to buy at new, UK prices. This review was influential when I was considering the build, and I have not been disappointed.

Still to do - put in a case, use [H] BIOS to overclock CPUs and play with memory timings, measure temperatures and adjust fans to suit. Photos

 

Thanks for that Christopher, really illuminating.

I would like to build a multi-socket rig, maybe I could buy the components a few at a time so I don't realise the total cost.

 

That is really useful info. I've been thinking about a multi CPU rig just for folding - it gets the competitive juices flowing doesn't it! I thought about building one over time but don't have the patience to do it. I would be looking at the growing pile of parts itching to build it!

 

Thanks for the breakdown Chris. This will form the basis of the shopping list next year.

+rep

 

As long as you can get a board, CPU and cooler you can get started straight away and just add further CPU's as you get the cash.

Yeah its the expensive bit first, but that's the way it is for everything.

 

Can you run the multi opted on boards with only one CPU in then?

 

One of the features of this board is that it will run with 1, 2, or 4 sockets occupied.

 

That's really interesting Chris, +rep for all the info. I'll do a bit more research on this.

One of the main issue seems to be finding a case that a SWTX mobo will fit into!

 

Hi Chris

Can you tell us if you got the CPUs from a UK seller or from the US and were they new or used? When I look on Ebay most of the UK dealers are selling Xeons and most of the G34 Opterons are in the US

 

Used, from USA

 

thats an awesome rig Chris...

looking forward to some photos

 

I've seen this...

http://www.xtremesystems.org/forums...ma-AMD-Opteron-Quad-Core-Socket-F-3000-series

Anyone had any experience? I'm only up to page 10, and it seems like they have a lot of problems... or is that just me?

 

Just started on the G34 4P route. Looking at a Xigmatek Elysium as it seems to be the biggest tower case for sensible money.

I have no idea whether a H8QGi +- F will fit though

 

i have this rig running now

 

Looked it up and the case only goes to xl atx

XL-ATX

The name "XL-ATX" has been used by at least two companies in different ways. In September 2009,EVGA Corporation had already released a 13.5" wide by 10.3" deep "XL-ATX" motherboard as itsEVGA X58 Classified 4-Way SLI.[5] In Q2/2010, Gigabyte launched another XL-ATX Mainboard with model number GA-X58A-UD9.



The motherboard may not fit.

 

4P cases

Hi One_Box

I do not know of any full tower cases that support the 4P boards as standard. If the motherboard will physically fit in the case then you can add extra standoffs to support the top edge of the board You need to look for a full tower case that supports E-ATX motherboards (these are the same width at 13") and has room for an extra power supply in the top position.

Cooler Master HAF 932 has been done, Xigmatek Elysium has been done, Enermax Fulmo GT should work. NZXT Switch 810 is not big enough. There may be other options. The 4P G34 boards are 16" by 13" The I/O area and expansion slots fit the normal ATX standard but the board has an extra 4" at the top.

Got any parts yet? How many cores are you going for with this machine?

 

Thanks for your advice.
I think I'll go for the Elysium as it is bigger than the HAF 932 and I prefer the looks.
My budget is limited so i've sourced 4 X 6128SE chips (which will give me 32 threads in total) 16Gig memory and passive heatsinks.
I intend to buy a Corsair AX 750 PSU, the Xigmatek case and H8QGi+-F mobo.
It will be interesting to see if it outperforms my 3930K