1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Bigadv SMP + Multiple GPU Clients - Questions

Discussion in 'bit-tech Folding Team' started by Unicorn, 26 May 2011.

  1. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    Yesterday I took delivery of a new GTX 260 TOP (216 core edn) from coolamasta for one of my folding rigs, and after testing on it's own in a benching machine last night and all day today, I wanted to get it running alongside another identical card in one of my i7 desktops. I did the usual setup procedure, extending the displays and installing the second GPU client as I have always done, but something is stopping the cards from running properly together.

    When I start the second GPU client, the first card drops from 8K PPD to 4.5K and the second card hasn't broke 4.5K PPD yet. The SMP client also slows down, going from 33 minutes/frame to over 37 min/frame, and dropping the PPD from 33K to less than 23K. I am running them in an i7 920 equipped machine, the CPU runs at 4.0 GHz and there is 6GB of 2000MHz Corsair Dominator memory installed. I am running the bigadv SMP client on that cpu with the -smp 7 flag.

    I know that it's either a lack of CPU resources, incorrectly assigned CPU resources or a shortage of RAM that is causing the slowdown, but I don't know which. If it is the CPU resource assignment, how should I set it for the best results? The cards are running on Arctic Cooling HSF's and are overclocked and can produce 9K on some projects, so I want to get them both running and have the whole system at it's "sweet spot" for maximum production capability.

    Thanks in advance.
     
    Last edited: 26 May 2011
  2. Leroyingo

    Leroyingo What's a Dremel?

    Joined:
    2 May 2011
    Posts:
    178
    Likes Received:
    9
    Hey Unicorn,
    Are the GPUs running at 100%? This is a quote from holzj17 in another discussion
    " go into the nvidia control panel, then manage 3d settings, and change power management to 'prefer maximum performance' " Other folders and I had experienced times where the GPUs would drop drastically in their usage and this setting rectifies that issue.

    On my machine i7-920 and 2 gtx275 i am running the GPU tracker v2, and I don't use the -smp 7 flag, and it doesn't take too much of a performance hit when I run both cards and the smp with bigadv. I think mine goes from 33:35 a frame to 34:10 a frame when I activate my cards.

    Hope that helps:thumb:
     
  3. One_Box

    One_Box Folder of the month April 2012

    Joined:
    11 Jul 2010
    Posts:
    601
    Likes Received:
    11
    When running SMP and GPU clients together I set the SMP client priority to idle and the GPU clients core priority to "slightly higher". In this way the GPUs will only take the resources they need. (In my experience 200 series cards use less resources than Fermi cards so see no need to set -smp 7)
     
  4. Haphestus

    Haphestus ....the folding under dog

    Joined:
    29 Jan 2010
    Posts:
    242
    Likes Received:
    6
    I thought it was only ATI cards that require a CPU core to run?

    If Nvidia does need a core to run then it is only using a finite amount of it as I don't notice any constant activity on my task manager.
     
  5. Tattysnuc

    Tattysnuc Thinking about which mod to do 1st.

    Joined:
    19 Jul 2009
    Posts:
    1,612
    Likes Received:
    57
    I've had the same experience. I now ONLY run BIGADV units on my fermi rig as the SLI cards interfere too much with the CPU folding to the point where the net gain is only a couple of thousand, but the heat output and therefore leccy consumed is massively more....

    My GTX 295 on the other hand doesn't noticeably affect the BIGADV units time...
     
  6. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    Hey guys, thanks for the replies so far. First of all, the full system specs:

    Asus P6T Deluxe
    i7 920 @ 4GHz
    6GB Corsair Dominator GT @ ~1800 MHz
    2x ASUS ENGTX260 TOP (Core 216) on Arctic Cooling Accelero GTX Pro coolers
    Corsair HX850W
    Windows 7 Professional 64
    CPU client version: SMP 6.34 @ -bigadv -smp 7 -verbosity 9
    GPU client versions: GPU2 6.23

    I double checked the "adjust for maximum performance" setting in the nvidia control panel, it wasn't that. I've tried running the GPU clients and cores at a slightly higher priority than the SMP client and I've tried the priorities in a few other configurations as well, I get the same results every single time. Im now starting to think it might be the second PCI-E slot on the board or a lack of RAM that's causing this.

    I should be able to run a pair of 260's along with the SMP client no problem. I've had the SMP client on a CPU and 4 smaller, less powerful cards before. That was on WinXP though, and was a long time ago, and didn't run bigadv.
     
  7. Christopher N. Lew

    Christopher N. Lew Folding in memory of my father

    Joined:
    23 Apr 2009
    Posts:
    1,357
    Likes Received:
    46
    A few thoughts, though as you say this hardware should run together

    Graphics drivers? Run Driver Cleaner and latest CUDA drivers from nVidia
    Clients? May be worth trying GPU3 (6.41). Shouldn't be necessary but who knows.
    Memory? 6GB should be enough. Does it pass Memtest? How much is used by GPU folding without SMP?

    Do you get the same problem if you swap the cards around? Or if you try this board with other (known reliable) less demanding cards?
     
  8. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    Hey CNL, thanks for your suggestions.

    -The graphics driver is the latest from Nvidia, same for CUDA.
    -It was my understanding that GPU3 only ran on Fermi cards, have I got that wrong?
    -Memory has been run through memtest a lot over the past while, because this machine was crashing randomly but it turned out to be a PSU issue, the memory is fine.
    -I get exactly the same problem if I swap the cards round in their sockets, yes.
    -Memory usage on the GPU clients without the SMP running is less than 3000K (less than 3 MB), the Core_11 processes use around 85,000K (almost 100 MB) and the Core_a5 process uses 1,000,000K (Almost 1 GB). The actual SMP client uses less than 1 MB of memory when running.
    -I haven' tried this board with anything else recently. I used to run a GTS250 and a 9800GT in it before I got my hands on either of the 260's and those ran fine with the bigadv.
     
  9. Leroyingo

    Leroyingo What's a Dremel?

    Joined:
    2 May 2011
    Posts:
    178
    Likes Received:
    9
    As far as your memory usage that sounds fine. My A5 is at 1GB and the 2 core 11s are at 74MB and 83MB, obviously depends what units you are running.
    Maybe you could try using one of the other clients for the 260's, like the GPU Tracker v2 (which i use with no issues apart from its essential to shut down the cpu client before the gpu's otherwise there is a good chance of a system crash) or the F@H client V7.

    My system, hardware wise is very similar, so I doubt its a lack of memory, or system performance.
    I just run the different client.
    i7-920 @ 3.8
    Asus P6T deluxe v2
    6gb Corsair 1600mhz
    1x Gainward GTX 275, 1 x Gigabyte GTX 275
    Enermax Revolution 850w
    Windows 7 Ultimate 64bit
    GPU tracker V2 - which runs the smp client 6.34 with -forecasm -smp -bigadv -verbosity 9
    and the GPUs client 6.23 with -forcegpu nvidia_g80 -verbosity 9 -gpu # (0 or 1)

    What flags are you running for the GPU clients?

    So your system should be completely fine spec wise to do what you are doing,unless as you stated theres a problem with the mobo.
     
  10. Christopher N. Lew

    Christopher N. Lew Folding in memory of my father

    Joined:
    23 Apr 2009
    Posts:
    1,357
    Likes Received:
    46
    A Fermi card must run GPU3, but pre-Fermi cards are fine with GPU2 or GPU3. GPU3 can detect what type of card is installed, and act accordingly.

    Might be worth installing Win XP and checking if that shows the same behaviour, which would suggest a hardware problem.

    Or taking the machine out of the case and trying the third PCIe slot.
     
  11. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    I've never used the GPU tracker client before, so I'll give it a go tonight and see what happens. The GPU clients don't run with any flags other than - gpu and the correct machine ID.

    CNL - I didn't fully understand that before now, so thanks for clarifying it for me.

    So after a minor brainwave I've almost sorted it. That's right, I'm an idiot! :p I didn't have the - gpu 0 and - gpu 1 flags set on the clients :hehe: They are still underperforming though, getting only 7K and 5K PPD respectively. They're both capable of over 8K at standard clocks on their own. I think I'm still missing something here...
     
  12. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    I just want to close out this thread by thanking you all for your suggestions and help with this issue. I also want to apologize for being such a fool and not realising that the problem was so obvious and was my own fault! The important thing is that both cards are now folding along with the SMP client, the cards get anywhere from 7200 to 9000 PPD depending on the project they are running, and the SMP client is still producing the maximum amount of points. I actually changed the smp flag to include all 8 cores as advised here, which dropped another minute off the frame times.
     
  13. coolamasta

    coolamasta Folding@Home CC Captain 2010/11/12

    Joined:
    26 Apr 2009
    Posts:
    2,619
    Likes Received:
    110
    LOL, I was about to step up and say its not something silly like the -GPU 0 and 1 flags and also the different working folder for each client is it, hehe easily done mate :D

    Get those shaders up to 701mhz on the 260's, they will HAPPILY run at that all day long as long as you watch the temps (I used to have fans on 100% because they were in a AC'd data centre) :)
     
  14. Unicorn

    Unicorn Uniform November India

    Joined:
    25 Jul 2006
    Posts:
    12,726
    Likes Received:
    456
    Haha, it's even more easily done when you haven't configured a new folding rig in months :hehe:

    Cores are at 710 and stable bud :thumb: I have them running on AC Accelero GTX Pro coolers :)
     

Share This Page