Coolamasta's SandyBridge-EP (Xeon) build! - Some Pics Up!

[Ben Lamb]

Originally Posted by coolamasta

Cool cheers for the info Ben

I am actually having 2nd thoughts about the Asus board now and considering the SuperMicro X9DAi, reason being is that a lot of people have had issues with the Asus board and certain model Nvidia graphics cards not working and also it does not suppose Win 8 or Server 2012, if you try to install either the machines hangs with "ACPI_BIOS_ERROR".

Asus is being totally rubbish with helping people get the graphics card problem sorted, one person has been waiting over 2 months for a BIOS that will fix it and still waiting now, also they announced that they are currently trying to develop Windows 8 compatibility BUT it will NOT SUPPORT Server 2012 as its a workstation board and to go and buy there Z9PE-D16 server board if you want 2012 compatibility, how crap is that!

Really making me think twice, I know right now im not going to be running powerful graphics cards or Server 2012 but I would like to have the option further down the line which is why im thinking about the SuperMicro X9DAi, also the X9DAi has on-board graphics, double the RAM slots and a Mini-SAS port.... Hmmmm.....

Yes the asus board does have an issue with certain nvidia graphics boards it causes the machine to blue screen at the windows7 installation hardware detection element. This caused me much frustration but the fix is to install win7 up to the point it has copy and expanded all the files and it wants to restart, after it restarts turn it off and remove the nvidia card. Then switch it back on and let it continue installing without the gpu. Leave it for a while to let it finish then switch off and put card back in. Then it will boot into windows and you must install the latest nvidia driver off the web and all is good the board just doesnt like the nvidia driver that comes with win7 its a ballache but works.

I have looked at the X9DAi myself but you have to remember that asus boards are the best if running ES chips. I wouldnt even be that confident that the X9DAi would boot up with your chips. If the chips are unmarked like yours it means that they were used internaly for testing by intel technicians/partners and that means they are early chips that are not widely distributed. The ES chips that are marked up are distributed more readily to testers/manufacturers, they are later chips more likely to have compatable microcode still in a motherboards bios.

The supermicro is not as well made as the asus with crap cooling on the vrms and chipset basicaly dont bother and get the asus as it is a great board trust me and you realy dont want to run anything else with ES chips as they are a lottery and asus gives you the very best chance of winning.

P.S. the asus also has onboard graphics you just have to buy a vga adapter cable/bracket off them to use it which is compltetely ridiculous at this price point.

[coolamasta]

Great thanks very much for the info Ben, some great points there, back to the Asus board I think it is then lol, will start looking around to see who has them in stock...

+Rep to you as well for the help, much appreciated

[kirk46]

coola...

are you keeping the sr-2 and x xeons folding as well?

[coolamasta]

Originally Posted by kirk46

coola...

are you keeping the sr-2 and x xeons folding as well?

I would like too mate but depends on how much power this new rig draws, once its up and running and I can compare PPD/WATT output I will make a choice, I'm spending money I don't really have at the moment so I may well sell the SR2 rig off but I would like to keep it as it is a beast

[kirk46]

ok bud

if you do sell i maybe interested in the xeons... at the right price

[Slowlemon]

That`s interesting about the issue with Nvidia Graphics cards. I had a similar problem installing Ubuntu on my system, the GT610 would not work with the drivers on the CD. I had to fit an older card for the OS install and then after updating the Nvidia drivers I swapped back to the GT610 and everything was fine.

Almost all the Ebay sellers of LGA2011 Xeon ES chips say that their chips have been tested working in the Z9PE-D8, most recommend using this board.

[coolamasta]

Yeah I noticed that as well Slowlemon, Asus board now ordered, should have by Friday so over weekend I can start looking at the build, I have 2 Titan Fenrir coolers which I am going to see if I can get to fit but if its not happening I will have to buy some early next week, would really like to use the Titans though as they are great coolers.

Otherwise I just need a case to fit it in, im not going to just buy one as this is all about building a high-spec rig with as tight a budget as I can! I have 3-4 cases sat about so I will be shoe-horning this Asus board into one of them, time to get the dremel out!

If this rig works well I may consider a second one and sell off my SR2 beast, better not get ahead of myself just yet though, still a fair way to go!

[Ben Lamb]

Originally Posted by coolamasta

Yeah I noticed that as well Slowlemon, Asus board now ordered, should have by Friday so over weekend I can start looking at the build, I have 2 Titan Fenrir coolers which I am going to see if I can get to fit but if its not happening I will have to buy some early next week, would really like to use the Titans though as they are great coolers.

Otherwise I just need a case to fit it in, im not going to just buy one as this is all about building a high-spec rig with as tight a budget as I can! I have 3-4 cases sat about so I will be shoe-horning this Asus board into one of them, time to get the dremel out!

If this rig works well I may consider a second one and sell off my SR2 beast, better not get ahead of myself just yet though, still a fair way to go!

Been looking at these X9DRL-iF only £310 trying to find out if they work with ES chips from a source in china. Its a tight squeeze but these will fit in an atx case !

[Slowlemon]

Hi Coolamasta

The Titan Fenrir coolers look useful. You could drill out the mounting holes in the hold down plate and cut some M4 screws to length . Be very careful with this, there is no through hole in the motherboard, if the screws are too long they could screw into the board and destroy it!

Alternatively get these. They will fit the board but I don`t know if the length or thread is compatible with the Titan fittings.

[coolamasta]

ITS ALLLLLIIIIIIIVVVVVVEEEEEEEEEEEEEEEEE

The Asus board arrived today, I just chucked a couple sticks of RAM in and managed to get the Titan's mounted, found some perfect M4 bolts and with a single washer on top meant there was enough thread to hold them down but also not enough for them to hit the board

It posted first time, went into the BIOS and had a poke about, it has BIOS 2.14.1219 installed at the moment, am going to go to the Asus Website in a minute and check what other versions there are, I guess you Ben and Slowlemon are using the latest one?

I'm uber happy although for some reason in the BIOS in the monitor section its saying both CPU's are in the high 50's which seems a bit weird?!

Have to go out now but next stage is to stick some more memory in, update BIOS to latest and get an OS installed and start having a play around

[Slowlemon]

That`s excellent news coolamasta .
That`s three dual E5-Xeon machines on the team, I am liking this a lot .
Keep us posted.

[coolamasta]

Right guys heres some pics, not had much chance to touch it this weekend but I've made a thread in the project section HERE where I will be putting most of the build pics but I will put some up in here also:

I've currently got Win 7 x64 installed to do some testing with, all 32 threads are showing up, the RAM is running at 1600mhz and I'm getting 2.4ghz with Turbo on, for the 1st 30 seconds I can hit 2.5ghz but it drops back to 2.4ghz after, seems like the same issue Ben and Slowlemon are getting, need to do some research on it I think...

I ran a couple of A4 folding projects yesterday on the Windows W7 client and im seeing 72,000ppd using 230w from the wall, once I've finished playing and testing I will be installing Linux 12.04 x64 and the Linux W7 client

I have one issue at the moment though and that's on CPU 2, a couple of the cores are 20 degrees higher than the rest of them and I don't know why, going to re-apply the TIM and see if that makes a difference in a bit...

Anyway heres some pics!

[One_Box]

Coming on really nicely Coola

[Ben Lamb]

Nice pics - you are even running the same msi 5450 as me - good choice for low power consumption. The cores will only run full whack for 32s as that is the longest term the chip will allow you to exceed the max tdp. Turbo is controled by hardware and firmware on the chip and there is no way round it believe me I have tried ever since nehalem came out. Dont worry about the cores running hotter again nothing you can do and it isnt your tim. Its just the ES chips, most B0s run like this due to buggy temp sensors on the die, some show 1 degree on one core and 60 on the next so dont worry. When you get it finished and linux running may be best running the older 6 series client for folding as the kraken isnt working so great on v7 I am having trouble with DLB not engaging as autorestart wont work on v7 and isnt included in the latest kraken so experimentation needed here for us e5 guys.

[coolamasta]

Cheers guys Also thanks Ben for the info on the temp sensors, I hope its a bug as most cores hover around low 40's while under load but these 2 cores are low 60's!

I was off work today so this morning I installed Ubuntu 12.04 x64 via WUBI so I could have quick dual boot with Win 7 on my test HDD, I installed version 6.34 of the FAH client along with latest Kraken and left it folding a bigadv, just checked on HFMmon and on a 8101 project I am doing 21.05 minutes a frame making around 178,000 PPD while only using 238w from the wall!

I am soooooo impressed with the performance of this rig I cant even begin to tell you!

I am going to have a play with the BCLK after this project has completed and dive in with a 104mhz and see how it runs, at 104 I should see just under 2.5ghz (24x104 = 2496mhz) will let you know how it goes!

Otherwise I need to find a case to fit this bad boy in, am thinking about carving up an old Antec 300 as I would like to try and keep it a pretty small footprint, more to come anyway...

[Christopher N. Lew]

Anyone fancy a quad socket Xeon-5 machine? These are on eBay -

Intel Xeon E5 4620 ES Sandy Bridge-EX 2.2GHz 20MB 8C 16MB C0 4 Way

The board would cost another £800

[coolamasta]

A 4-way SandyBridge EP would be EPIC, 64 threads of WIN lol

Maybe for my next project!...................

[coolamasta]

Small update - rig is still sat on my bench running, not had much time to do any more work on it but I did bump the BCLK up to 104

Result is CPU's run at 2495mhz with Turbo on (24x104) and im getting 20m 41s time frames on a 8101 which = 184k PPD-ish Oh and its pulling 252w from the wall

Have shut down my SR-2 Hex core Xeon rig for the moment as it needs a good service, water blocks needs a clean out as my flow rate has dropped, clean water needs flushing through and then some fresh put in, will leave the EP-Xeon build running on the bench for now and then look at getting it in a case etc next week when the SR-2 is running again

[coolamasta]

It wasn't 100% stable at 104 BCLK and reset itself every 30 or so hours so I've dropped it to 103 BCLK and its been up for 4 days solid now without problems.

Is there a voltage I can tweak guys to help it become stable with a 104 BCLK? lol this is funny trying to get it stable with 1mhz more

[Slowlemon]

Hi Coolamasta

I have lost two 8101s as early unit end so I think my setup is not 100% stable at 104MHz. For now I have backed it down to 102MHz. You can adjust Vcore and VSA in the BIOS, I have played around with the voltages a bit but I have not done any proper testing. More work is needed.