A few questions regarding my new multi GPU rig

Discussion in 'bit-tech Folding Team' started by Unicorn, 23 Apr 2010.

  1. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    *Oh no, not this guy again!* :p

    I'm getting ever closer to building the multi GPU folding rig of my dreams, but it's been so long since I dreamed up this monster that I have forgotten some of the fundamentals that I learned from the likes of JackOfAll, coolamasta & saspro back then regarding multiple GPUs in the same board. What I need to know right now is:

    - Will having two reduced speed PCI-E slots (8x) reduce the PPD that multi GPU cards will be able to produce?

    - What is the maximum number of GPU clients that you can run under windows with the current client version? i.e. will I be able to fully utilize 3x GTX 295's if I go for that option?

    - On a side note (and this is really only aimed at those who currently run 295's under water) what are the chances that a 120x360mm HWlabs BIGT radiator would keep two or possibly three 295's cool enough to fold?

    Thanks in advance. I may change the title of this thread to "Uni's GPU2 Monster discussion" if I have any more questions regarding the specifics of the folding clients etc.
     
  2. Ph4ZeD

    Ph4ZeD What's a Dremel?

    Joined:
    22 Jul 2009
    Posts:
    3,806
    Likes Received:
    143
    The lanes available doesn't matter for folding. I was under the impression that 7 or 8 clients was the maximum you could run due to driver limitations.
     
  3. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    All good news to me so far. The board I am using for this rig is going to be tri-SLI capable so I'll be able to have up to six available GPU cores. Whether I'll be able to use them all at once is what I need to know now, so that I can plan which cards to budget for and buy.
     
  4. Ph4ZeD

    Ph4ZeD What's a Dremel?

    Joined:
    22 Jul 2009
    Posts:
    3,806
    Likes Received:
    143
    The reason I'm sure you can run 6+ gpu clients at once is because I remember CustomPC talking about a rig they had for folding - 4x 295 GTXs in one motherboard.
     
  5. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    As far as I can remember (don't have that issue to hand) that system ran Linux. I can do that if necessary, although I'd like to be able to use Windows. Actually the board I'm using is a Tri-SLI board that also has a 4th PCI-E slot for a PhysX graphics card, so I could fold on that as well. That's potentially 7 clients. Drooling yet? ;)
     
  6. Ph4ZeD

    Ph4ZeD What's a Dremel?

    Joined:
    22 Jul 2009
    Posts:
    3,806
    Likes Received:
    143
    Is the rig going to be folding 24/7? I hope so, because over the last month we've stagnated a little bit in overall PPD. Do 295GTXs pull in 13K each?
     
  7. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    It most certainly is. It's something just a little bit special, this project. When it pops up, you'll know what I'm talking about ;)

    295's are capable of producing up to 7K PPD per core as far as I can remember. That's up to 14K per card and potentially 42,000 points per day plus whatever the PhysX card (9800GT) throws into the mix. Like I said, this project is special.
     
  8. Ph4ZeD

    Ph4ZeD What's a Dremel?

    Joined:
    22 Jul 2009
    Posts:
    3,806
    Likes Received:
    143
    Sick ************... like I said we need the points boost! I myself will be pushing my PPD to 12K very soon, got a new base unit with Q6600, 9600GSO and 8800GT which will be going in the loft next the router to fold 24/7.
     
  9. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    I really need some help on the cooling for this at the moment. I've been doing some quick "back of a napkin" calculations on heat dissipation for the BIGT radiators and I really don't think they can handle so much thermal output. I may have to completely rethink this unless I can come up with something good on the cooling :sigh:
     
  10. saspro

    saspro IT monkey

    Joined:
    23 Apr 2009
    Posts:
    9,644
    Likes Received:
    437
    You can run 8 gpu clients.
    You can fold a single card on a x1 connection.

    To do 3 cards you may need 2 rads. I think Clive used 2 360mm rads to cool his triple 295/i7 rig
     
  11. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    Thanks Saspro. So I need to fit another 360 rad in there somewhere :rolleyes:

    Back to the drawing board. I'll figure this out eventually. I don't see it at the moment, but there's a solution to this problem that just needs to be uncovered. I must go and look up Clive's rig again. That's be a good reference point for me to use regarding temps etc.
     
  12. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    So, less than 4 hours since I posted ^that^ and with lots of CAD prototyping and brainstorming (and furious measuring of the case this is all going in!) I have found an easily implementable solution that allows me to have the cooling capacity of 2x 360mm radiators for the cards without actually having 2x 360mm radiators :D And on top of that (literally :hehe:) Iactually do have a 360mm radiator for the CPU and chipset.

    I love it when a plan comes together :thumb:
     
  13. coolamasta

    coolamasta Folding@Home CC Captain 2010/11/12

    Joined:
    26 Apr 2009
    Posts:
    2,618
    Likes Received:
    110
    Yeah get lost stranger :p :p haha just kidding, come on now mate get that beast up and running quick for the CC!! :rock:

    Nope absolutely fine, its been proven that you can fold on a 1x slot all bit it with slightly reduced PPD but anything 4x and above is cheesecake lol

    You can run upto 8 GPU clients, my biggest rig runs 6 GPU clients using 3 x 9800GX2's

    Not a chance mate, my main/gaming rig I have a GTX295 under water with an i7 @ 4.2 with twin 120x240 rads and while its keeping the temps down really well if I added a 2nd GTX295 I would prob be on the limit so you will deffo need to add at least another 120x240 or ideally another 120x360! Oh and the 120x360 you already have, is it single or double thickness? I hope its double! :worried:

    Good luck getting it up and running mate :thumb:
     
  14. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    Okay, thanks for all the replies and help so far guys! Rob, after some more detailled calculations that I did this afternoon and much measuring up etc, I have come up with what I think will be an acceptable solution. I couldn't fit a second 360 into the case (the first one is for the 930 CPU and the chipset) but I can fit the equivalent of another two into the case by using 3x 120x240mm rads in their place. This takes up less space (vertically or length wise I mean) and allows me to fit the equivalent of two additional 120x360mm radiators into the case to cool the three cards. Not only will these be linked together to form one large heat dissipation loop, but they will be completely isolated from the rest of the machine through the use of a high flow air chamber. This means that the heat produced by the card cores is removed from the system immediately and never affects the ambient temp of the other components. Unfortunately, it's just too tight a squeeze to use double thickness 240mm rads. It's tight enough as it is using 30mm thick BIGT Stealth models for the air chamber, but the 120x360mm that will be cooling the CPU and chipset will almost certainly be a double depth Black Ice GTX rad which is almost 60mm deep. That oughtta keep the chipset and overclocked i7 well in check ;)

    The air chamber will basically be like a high airflow wind tunnel. The entire system will have an idle "stealth" mode and an active mode which should both be fairly self explanatory. I think the 3x BIGT's with enough air moving should keep 3x 295's cool enough. I certainly hope so anyway!
     
    Last edited: 23 Apr 2010
  15. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    New question regarding this project - seeing as 295's seem to be rare as the pot of gold at thbe end of the rainbow at present, I may be forced to use 480's instead. Question follows.

     
  16. stonedsurd

    stonedsurd Is a cackling Yuletide Belgian

    Joined:
    11 Mar 2009
    Posts:
    7,856
    Likes Received:
    418
    A quick google shows that the TDP on the 295 is 289W while it's 250W on the 480. You should be fine.
     
  17. Taffy

    Taffy Folding for Margaret 17/3/2011 RIP

    Joined:
    23 Apr 2009
    Posts:
    561
    Likes Received:
    12
  18. oatey4519

    oatey4519 can't be arsed...

    Joined:
    5 May 2009
    Posts:
    437
    Likes Received:
    2
  19. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    Well if I can't get hold of 295's they are going to have to do, but the 295's are the better option for maximum PPD from one rig. If anyone hears of any GTX 295's for sale in the next 2 months, make me the first to know! I'd still love to get hold of at least two to begin with and then maybe keep my ear to the ground and get a third at a later stage.
     
  20. DocJonz

    DocJonz Another CPC refugee .....

    Joined:
    24 Apr 2009
    Posts:
    1,221
    Likes Received:
    111
    Looks like it's time for an nVidia driver update to be ready to play with the up-coming GPU3 changes.
    It's a shame that finding a suitable, stable nVidia driver is a bit like Russian Roulette :wallbash:
    .... unless anyone is already running a stable CUDA 3 driver (even better if it's working for XP32 and GTX 260's :hehe:)??
     

Share This Page