bit-tech.net

DocJonz · 3rd May 2012, 07:21

It's time I put together a substantial multi CPU machine to move into the bigadv-16 era!

I was toying with dual Xeon E5 2670's, but it seems the general route is quad G34 Opterons, mainly on cost grounds, is that right? The former I can cope with, but the latter is out of my area of knowledge - if I've trawled my threads correctly, some of you guys have gone down this route. Therefore, I'm after some advice on such a build if possible ....

phoenicis · 3rd May 2012, 12:07

Hi Jon,

A lot depends on your budget and asthetic requirements. I ended up with some used mid range chips (6174s) off ebay, GSkill 1333 CL7 memory and no case. The m/b is one of the Supermicros for which the boys at [H] have created an OC BIOS.

The whole thing came to almost £3K although I did buy 64GB of memory for BOINC when you'd get by with 16 x 1GB sticks for folding. There are cost variations on most of the parts that will provide a delta to the speed or effort required. For example, a good deal on 6128s (octo cores) can be around £100 each whereas 6174s (dodeca cores) are typically closer to £400 each after duty etc.

As you can see, built for speed rather than looks

Bob

DocJonz · 3rd May 2012, 17:53

Bob - that's fantastic

I'll send you a pm .......

Christopher N. Lew · 4th May 2012, 00:06

I eventually went down this route to do something a little bit different, but if you can put together a dual Xeon system, the quad G34 Opteron build will be no trouble. It seems that like Bulldozer CPUs, the Opteron 6200 series give no more PPD than the equivalent 6100s, but use significantly more power. I decided to concentrate on a low power machine, which I don't have completely running yet*, but my two cents worth is ...

+1 Supermicro board with the [H] bios. The memory on those boards runs quicker if all four slots per CPU are populated, but the total memory doesn't have to be very high for folding. I got some low profile, low voltage Samsung. I have in mind that the memory timings can be adjusted with the [H] bios, which can yield a few more PPD.

The glut of cheap Opteron 6100s on eBay seems to be over, so it principally comes down to how much you want to spend and how long are you prepared to wait. I may have a spare set of 8-cores soon, let me know if you're interested.

{*The problem is that one of my 12 cores CPUs isn't recognised. I've still got to discover if its the CPU or the board that is faulty. So I'll be running at reduced capacity for the Chimps Challenge, and RMA later}

DocJonz · 4th May 2012, 06:32

Chris - it looks like I will be going down the Supermicro quad socket route from what I've read/heard, and I think your post has helped confirm this. Cheers.

coolamasta · 9th May 2012, 14:56

Good stuff Jon, I actually have a suitable 4U HP case that will accommodate one of these Supermicro boards as I was planning to go the same route myself after packing in GPU folding but I just don't have the cash to get the project started at the moment

I'm keeping the case though as its rare as rocking and I might just do it next year.... I'm not even sure what form factor these boards are lol.

Oh and I think you should do a built thread Doc

DocJonz · 10th May 2012, 18:34

Quote:

Originally Posted by coolamasta

Oh and I think you should do a built thread Doc

For the first foray down this route, the build log will be very, very short indeed - in fact, I can summarise it here; contact an IT solutions company, spec a rig, place an order, take reciept of new machine - job done

RedFlames · 10th May 2012, 18:55

would love to build [and have] a rig like that... wouldn't love the leccy bill so much though... but it'd give me something to do...

good luck with your build/purchase

DocJonz · 10th May 2012, 19:02

Quote:

Originally Posted by RedFlames

would love to build [and have] a rig like that... wouldn't love the leccy bill so much though... but it'd give me something to do...

good luck with your build/purchase

I had it mind to do this about a year and a half ago, but didn't have the funds at the time ....

This is all about leccy - actually reducing it drammatically, as it will enable me to forgo all the GPU Folding (this rig should use less juice than one of my GPU Folding rigs).

Scorpuk · 10th May 2012, 19:07

which case coolamasta?

Sent from my GT-I9100 using Tapatalk 2

DocJonz · 30th May 2012, 19:57

Well, after a few teething troubles, my first venture into the multiple CPU socket world is in business - quad Opteron 6134's (see progress in the sig)

TaRkA DaHl · 30th May 2012, 21:43

Initial outlay if you don't mind?

And power drawn at the plug if you know would be very helpful

DocJonz · 30th May 2012, 21:51

Quote:

Originally Posted by TaRkA DaHl

Initial outlay if you don't mind?

And power drawn at the plug if you know would be very helpful

1. You don't want to ask .... !!
2. 590W at 100% load

Scorpuk · 31st May 2012, 05:22

Any piccy's? :-)

fuus · 31st May 2012, 08:03

Quote:

Originally Posted by DocJonz

1. You don't want to ask .... !!

Oh, we do

Tattysnuc · 31st May 2012, 12:57

Quote:

Originally Posted by fuus

Oh, we do

seconded. Shame you haven;t done a build and/or log to follow. I've found a similar topic article over on Overclockers and that's really whet my appetite to do something similar - for my SQL server/Folding project.

DocJonz · 31st May 2012, 21:05

Quote:

Originally Posted by Tattysnuc

seconded. Shame you haven;t done a build and/or log to follow. I've found a similar topic article over on Overclockers and that's really whet my appetite to do something similar - for my SQL server/Folding project.

As mentioned earlier in the thread, there was no build log .....

Quote:

Originally Posted by DocJonz

For the first foray down this route, the build log will be very, very short indeed - in fact, I can summarise it here; contact an IT solutions company, spec a rig, place an order, take reciept of new machine - job done

As for piccy's ... it's not very exciting to look at (it's got nothing on phoenicis's stirling effort!), as it's basically a very large (rack-mountable) tower just like this;

fuus · 31st May 2012, 21:09

*bugs for prices*

On a side note, I actually quite like that case..

DocJonz · 31st May 2012, 21:29

Quote:

Originally Posted by fuus

*bugs for prices*

No specifics, but let me just say that the VAT alone would have bought a decent desktop PC

Christopher N. Lew · 1st Jun 2012, 18:59

Got my 48 cores folding (Opteron 6166 at 1.8GHz), and here are some numbers -

P6901: TPF 08:08, PPD ~254000, power 394W
P6903: TPF 17:10, PPD 371182, power 395W
P6904: TPF 24:53, PPD 349300, power 396W

Hopefully a little overclock will take me past the 1000 PPD/W mark.

3rd May 2012, 07:21	#1
DocJonz Another CPC refugee ..... Join Date: Apr 2009 Location: The North Posts: 619	Multi CPU rig (for bigadv-16) It's time I put together a substantial multi CPU machine to move into the bigadv-16 era! I was toying with dual Xeon E5 2670's, but it seems the general route is quad G34 Opterons, mainly on cost grounds, is that right? The former I can cope with, but the latter is out of my area of knowledge - if I've trawled my threads correctly, some of you guys have gone down this route. Therefore, I'm after some advice on such a build if possible .... __________________ Folding Farm Stats (HFM): http://www.docjonz.net/summary.html

3rd May 2012, 12:07	#2
phoenicis Retired Chimp Join Date: Apr 2009 Location: UK Posts: 490	Hi Jon, A lot depends on your budget and asthetic requirements. I ended up with some used mid range chips (6174s) off ebay, GSkill 1333 CL7 memory and no case. The m/b is one of the Supermicros for which the boys at [H] have created an OC BIOS. The whole thing came to almost £3K although I did buy 64GB of memory for BOINC when you'd get by with 16 x 1GB sticks for folding. There are cost variations on most of the parts that will provide a delta to the speed or effort required. For example, a good deal on 6128s (octo cores) can be around £100 each whereas 6174s (dodeca cores) are typically closer to £400 each after duty etc. As you can see, built for speed rather than looks Bob __________________

3rd May 2012, 17:53	#3
DocJonz Another CPC refugee ..... Join Date: Apr 2009 Location: The North Posts: 619	Bob - that's fantastic I'll send you a pm ....... __________________ Folding Farm Stats (HFM): http://www.docjonz.net/summary.html

4th May 2012, 00:06	#4
Christopher N. Lew Folding in memory of my father Join Date: Apr 2009 Posts: 1,325	I eventually went down this route to do something a little bit different, but if you can put together a dual Xeon system, the quad G34 Opteron build will be no trouble. It seems that like Bulldozer CPUs, the Opteron 6200 series give no more PPD than the equivalent 6100s, but use significantly more power. I decided to concentrate on a low power machine, which I don't have completely running yet, but my two cents worth is ... +1 Supermicro board with the [H] bios. The memory on those boards runs quicker if all four slots per CPU are populated, but the total memory doesn't have to be very high for folding. I got some low profile, low voltage Samsung. I have in mind that the memory timings can be adjusted with the [H] bios, which can yield a few more PPD. The glut of cheap Opteron 6100s on eBay seems to be over, so it principally comes down to how much you want to spend and how long are you prepared to wait. I may have a spare set of 8-cores soon, let me know if you're interested. {The problem is that one of my 12 cores CPUs isn't recognised. I've still got to discover if its the CPU or the board that is faulty. So I'll be running at reduced capacity for the Chimps Challenge, and RMA later} __________________

4th May 2012, 06:32	#5
DocJonz Another CPC refugee ..... Join Date: Apr 2009 Location: The North Posts: 619	Chris - it looks like I will be going down the Supermicro quad socket route from what I've read/heard, and I think your post has helped confirm this. Cheers. __________________ Folding Farm Stats (HFM): http://www.docjonz.net/summary.html

bit-tech.net

Site Links

Our Other Sites

9th May 2012, 14:56	#6
coolamasta Folding@Home CC Captain 2010/11/12 Join Date: Apr 2009 Location: Sunny Brizzol! Posts: 2,339	Good stuff Jon, I actually have a suitable 4U HP case that will accommodate one of these Supermicro boards as I was planning to go the same route myself after packing in GPU folding but I just don't have the cash to get the project started at the moment I'm keeping the case though as its rare as rocking and I might just do it next year.... I'm not even sure what form factor these boards are lol. Oh and I think you should do a built thread Doc __________________ My Main Rigs:~ H2O'd i7 3770K @ 4.5 - 16GB - H2O'd GTX 670 \| Asus Z9PE-W8, EP-Xeon 2670 @ 3.2 - 16GB \| ~~~ My Current Dual EP-Xeon 2670 Project ~~~ \| ~~~ My Mitsubishi Evo Track Car Project ~~~ \|

10th May 2012, 18:55	#8
RedFlames Chief Gorilla - CC 2013 Join Date: Apr 2009 Location: The northern wastes of Geordieland Posts: 2,053	would love to build [and have] a rig like that... wouldn't love the leccy bill so much though... but it'd give me something to do... good luck with your build/purchase __________________ I have the heart of a child... in a jar... on my desk... Steam :: Raptr :: XBox :: Twitter

10th May 2012, 19:07	#10
Scorpuk Supermodder Join Date: Jan 2012 Location: North Ayrshire, Scotland Posts: 364	which case coolamasta? Sent from my GT-I9100 using Tapatalk 2 __________________ Folding Summary Desktop: Corsair Carbide 400R; Asus Sabertooth X79; Intel i7-3930K; Hydro H100; EVGA GTX 580 772MHz; Corsair Vengeance 16GB DDR3 1600MHz; Corsair HX850W; Samsung 256GB 840 Pro; 2TB HDD; Windows 7 Ult. 64bit.

30th May 2012, 19:57	#11
DocJonz Another CPC refugee ..... Join Date: Apr 2009 Location: The North Posts: 619	Well, after a few teething troubles, my first venture into the multiple CPU socket world is in business - quad Opteron 6134's (see progress in the sig) __________________ Folding Farm Stats (HFM): http://www.docjonz.net/summary.html

30th May 2012, 21:43	#12
TaRkA DaHl Ultramodder Join Date: Mar 2011 Posts: 1,368	Initial outlay if you don't mind? And power drawn at the plug if you know would be very helpful

31st May 2012, 05:22	#14
Scorpuk Supermodder Join Date: Jan 2012 Location: North Ayrshire, Scotland Posts: 364	Any piccy's? :-) __________________ Folding Summary Desktop: Corsair Carbide 400R; Asus Sabertooth X79; Intel i7-3930K; Hydro H100; EVGA GTX 580 772MHz; Corsair Vengeance 16GB DDR3 1600MHz; Corsair HX850W; Samsung 256GB 840 Pro; 2TB HDD; Windows 7 Ult. 64bit.

31st May 2012, 21:09	#18
fuus Confessed Rep Addict Join Date: Dec 2010 Location: Shropshire Posts: 1,873	bugs for prices On a side note, I actually quite like that case.. __________________ :: i5 3570k :: EVGA GTX 670 :: Z77X D3H :: 8GB RAM :: 800W PSU :: Filco with Browns :: 23' 1920x1080 :: 19' 1440x900 :: Microlab FC360 :: GiffGaff

1st Jun 2012, 18:59	#20
Christopher N. Lew Folding in memory of my father Join Date: Apr 2009 Posts: 1,325	Got my 48 cores folding (Opteron 6166 at 1.8GHz), and here are some numbers - P6901: TPF 08:08, PPD ~254000, power 394W P6903: TPF 17:10, PPD 371182, power 395W P6904: TPF 24:53, PPD 349300, power 396W Hopefully a little overclock will take me past the 1000 PPD/W mark. __________________ Last edited by Christopher N. Lew; 4th Jun 2012 at 14:11. Reason: Added P6904 figures