Pete J's Titan X multi GPU review - Pascal versus Maxwell Well, set period passes and a new generation of GPUs have hit. The regular members here may remember when I treated myself to something a little bit extravagant and I thought that would be me sorted for a while, but that upgrade itch, well, I'm sure a lot of you know the feeling. Additionally, four way SLI support has always been hit or miss, though my recent transition to Windows 10 has improved things considerably. So, let’s see - we have the Titan X versus the Titan X. Really, Nvidia? From now on I’m calling them the Maxwell or Pascal Titan - if Nvidia release an improved Pascal down the road, I guess that’ll be the Pascal Black? Now, I was aware of the fact that the four Maxwell Titans I have may actually be more powerful than two Pascal Titans, but I just HAD to know. In the case it turned out to be true, the plan was to put one of the Pascal Titans in my HTPC and sell the other if Nvidia don’t do refunds. This review is dedicated to Teelzebub, formerly Bumsrush, who kindly donated the nice EVGA four way SLI bridge to me a few weeks before passing away. May you be having fun raising hell wherever you’ve ended up. The rig Processor: i7 5930K @ 4.5GHz (1.36V) Motherboard: ASUS RAMPAGE V EXTREME Corsair 8X4GB Vengeance LPX Black @ 3000MHz 15-15-15-36-2T Boot drive: 500GB Samsung 850 EVO (SATA) Scratch drive: 250GB Samsung 850 EVO(SATA) Game/benchmark drive: 2X1TB Samsung 850 EVO in RAID0 (SATA) Storage drive: 2TB WD Caviar Green PSU: SuperFlower '8Pack Edition' 2000W Monitors: Asus PQ321QE (Main), Dell 3007WFP-HC (secondary) Audio: Onboard SupremeX with QPAD QH-1339 Mouse and keyboard: Logitech MK710 and G700s Miscellaneous: Warthog joystick, Saitek Cessna rudder pedals, Logitech G13 OS: Win 10 Professional 64 bit (Build 1511) UPS: APC SMT3000I Apologies for the crap picture - I’m no photographer and my phone was having real issues with the brightness contrast! The CPU overclock is stable enough to run Intel Burn Test for 25 loops on 'very high' and survive 12 hours of large in-place FFTs in Prime95 for twelve hours on twelve threads. Additionally, I had to up the voltage by 0.01 V compared to my last review as I was compressing a Bluray (a 12 hour task) and found it was crashing out just before the end. All SSDs have had 22% left unformatted. I have screen calibration hardware and software (Spyder4Elite), I left them active for testing. For Maxwell, 368.81 drivers were installed. For Pascal, the only ones available at the time of this review were 369.05. Only the drivers and PhysX options were installed, with a clean sweep between. Initial impressions I’m not going to post any close-up shots modelling shots as all the tech sites have pictures and they’ll be far superior compared to my shaky hand and phone camera. The new Pascal Titan looks the part, especially with its backplate. Very impressed. The new SLI bridge is particularly well constructed and feel like it could be fired through a wall! Temperatures, power and noise I’m not going to comment on this as I feel my noise and heat tolerance are high - I’m interested in raw performance and I use closed ear headphones when I play games, meaning that even fans on 100% don’t bother me. Check one of the review sites for further info - I’m only here for the FPS. In regards to power draw, I covered the Maxwell Titans in my previous review - if you want details on Pascal, please find another review as they have access to monitoring equipment I don’t have. Results Assume that for all tests that all in-game settings are at their maximum unless otherwise specified. And I really DO mean maximum: the highest level of antialiasing, supersampling, you name it (and vsync off) - none of this pussyfooting around that most review sites seem to do (maximum settings and 2xMSAA is NOT maximum settings in my humble opinion). Nvidia Control Panel wise, I set performance to maximum rather than adaptive but leave everything else alone. Also remember that the Maxwell Titans are EVGA Superclocked variants, so already have a bit of a speed increase over stock. I chose games which have an in-built benchmarking routine that I own. Annoyingly I forgot about Shadow of Mordor though! So, I have a bit of a dilemma - I have a shedload of data and quite frankly don’t want to take the time to make pretty graphs because I have what I need. So, What I will do is three things: I will upload a large image of the spreadsheet I made; I will link to the spreadsheet so those who are curious can copy (click on the image to do so) and manipulate the data and I’ll write a little summary of each benchmark or game. Any cells highlighted red mean that the results are dodgy in some way or another and are usually discounted in further analysis - for example, Batman Arkham Knight doesn’t support SLI (something I’m quite bitter about since the other games in the series did) and so is discounted when calculating average SLI scaling. Overclocking methodology is covered after discussion of the results. Ashes of the Singularity I need to point out that although I tried to set every setting to maximum, ‘Shadow Quality’ and and ‘Terrain Object Quality’ would reset to ‘High’. I have to say, I found the game rather unstable, crashing out every now and then. Pascal puts up a strong performance, showing ~60% increase in performance. SLI does nothing for the game, though Pascal is actually hurt by it. Moving on to DirectX12 multi GPU mode, again, Pascal shows ~60% improvement with another GPU. In game, this takes it from being juddery to smooth enough to play since it’s a strategy game. Apologies for the lack of Maxwell Titan DirectX12 multi GPU results - I accidentally deleted them. Whoops! I remember them being along the same lines of improvement as Pascal though for the results, meaning that more than two GPUs for Ashes is pointless at the moment. Pascal eeks out a little more performance when overclocked compared to Maxwell. Batman: Arkham Knight SLI is not supported for this game (cheers Warner Bros). At least the game is playable now I suppose. Having more than one GPU hurts numbers a little - I would’ve thought a secondary GPU would allow Nvidia’s GameWorks (by the way, shouldn’t the company who make SolidWorks have a trademark over ‘Works’?) to be offloaded for improved performance. Anyway, Pascal again shows ~60% improvement over Maxwell, though Maxwell overclocks a little better for this game. Crysis Why is a game that is is closing in on being nine years old still featured in my testing? Because: a) it’s a really fun game to while away an afternoon; b) To this day, it still looks amazing even without mods; and c) When maxed out it’s STILL a difficult game to run. On Windows 7, Crysis suffered from massive SLI stutter issues for me. Windows 10 has seemingly got rid of the problem and now I can run around at full speed, headshotting Koreans and throwing crabs. Happy days! A single Pascal LOVES this game, showing a huge 82% improvement in minimum FPS and a strong ~74% increase in average FPS. Pascal seems to be the first GPU in history to properly tame Crysis, but that’s just my opinion! And so, the first game on the list that allows a comparison of the SLI set ups. It’s neck and neck! No clear winner here - well, if you ignore little things like power consumption, heat, noise...yeah. Crysis doesn’t respond well to overclocking for some reason though. Strange. GTA V So as is seen, max everything out in a top performing game at UHD and you’ll bring any GPU to its knees. Ignoring minimums, Pascal puts on a strong show with ~80% better average FPS for single GPUs. An incredibly strong show SLI wise for Pascal as well: even with Maxwell pulling out ~280% scaling, Pascal storms ahead showing a huge ~80% improvement in minimums and a 20% increase in average FPS. Monstrous. Overclocking favours Maxwell quite heavily to bring it within spitting distance of stock SLI Pascals, but then Pascals opens up the lead again. Hitman: Absolution Another game that Pascal gets on well with, showing ~75% improvement in minimums and ~85% improvement in average FPS compared to Maxwell. SLI performance is pretty close with Pascal pulling just ahead - ignore the strange minimum FPS for Maxwell as the actual benchmark ran very smoothly. Overclocking levels the playing field with Maxwell overclocking better, though Pascal has considerably better minimums, no doubt due to the fact it must be easier to keep two GPUs in sync compared to four. Still, Maxwell’s minimums are nothing to sneeze at. Metro: 2033 To be honest, I probably shouldn’t have used the Metro games for benchmarking as the results can vary wildly for no apparent reason. Still, they’re on my list of ‘bestest games evar’, so I like to see how they do. So yeah, both cards are brought to their knees pretty hard. Pascal shows a huge 124% improvement in minimums and ~60% for average FPS.SLI wise, Pascal also wins quite nicely, with overclocking being similar for both architectures - a little low though compared to other games. Metro: Last Light Best game ever? Certainly a contender! Same story as Metro: 2033 really, though Maxwell has better minimums and closes the gap when overclocked. Now that both games have been mentioned, I should say that the benchmarks are far harsher to run than the actual game. If you can get the benchmark to be okay, you’re sorted ingame! Rise of the Tomb Raider Oh, wait a minute, maybe this is the best game ever? Let’s focus on DirectX11 first. Both GPUs hurting when alone - though Pascal pulls off a , shall we call it 60%, performance improvement. In SLI, Maxwell wins, being around about 17% faster for the average FPS. Overclocking extends Maxwell’s lead considerably, with average FPS just two shy of the magical 60 FPS desired. DirectX12 got rid of an annoying ‘flicker’ bug HBAO+ causes in DirectX11 on a particular map (Syria if I remember correctly) when SLI is used, otherwise performance improvement was inconclusive.. Also, in DirectX12, VRAM usage peaked at an unbelievable 11799 MB! Same story again - a single Pascal is ~60% faster, does better in SLI and overclocks further. And you thought this would be clear cut! Tomb Raider At the time of writing this post/thread/report/whatever, this is the game I’m currently playing through. So good! In windows 7 SLI used to cause terrible stuttering but Windows 10 renders it smooth. The only remaining issue is Lara’s hair when using SLI and TressFX - it bounces around a lot, and looking at it I reckon it’s something to do with each GPU redoing the TressFX calculation FX fresh each frame rather than carrying on from the previous frame. Or something. It does settle down after a little bit, though noticeably faster with less GPUs involved Pascal gets on well with the game, showing ~70% improvement over Maxwell individually. However, the engine absolutely loves SLI and the almightly rendering power of four GPUs comfortably pulls Maxwell ahead of Pascal, with Maxwell minimums above 62 FPS - that’s more like it! Overclocking heavily favours Maxwell, showing ~26% improved performance compared to Pascal. Seriously, 380% SLI scaling for Maxwell - why can’t every game do this?! Unigine benchmarks Bah, not going to talk about them individually! Let’s analyse on score alone - Pascal performs about ~65% better individually but the benchmarks’ excellent SLI scaling pulls off a win for Maxwell. Overclocking favours Pascal but isn’t enough to pull off a win for the new kid on the block. Overclocking Reviewers were complaining that the cards thermal throttled - so instead of whinging about it, I whacked the fan up to 90%, with 100% triggered at 70C. I took the power limit up to 120% and Afterburner set the temperature limit to 90C (during testing the top GPU got to 84C on 100% fan). However, not all was straightforward as it was with the Maxwell Titans. Normally this is a pretty cut and run affair - load up Heaven in a window and keep upping things until stuttering and/or artifacts appear. The Pascal Titan is a very different beast than the 900 series. The boost clock, although not thermally limited, kept going up and down - at one point hitting 2063 MHz! The memory, well...at one point I had it running at just shy of 11500 MHZ! Maybe I’ve landed a couple of super cards? It got to the point where I was deliberately trying to crash it - then it did, and I found that the stupidly high values I’d found didn’t work. I eventually found that setting the core clock didn’t do anything - the core clock was directly correlated to the power limit set. Even when setting the core clock to +0, the boost clock would still take the cards up to ~1850 MHz. After fiddling some more, it appeared that the memory speed may influence the boost clocks - the faster the memory, the faster the core clock as the card automatically found its ‘happy spot’. I was looking for artifacts from overclocking the memory too hard, but the drivers always seemed to crash before I got to that stage, so at least I don’t have to be paranoid about missing that little white flash - you guys know what I’m talking about. So, in a way, overclocking seems to have got real easy as you just up the memory frequency until things crash! The Pascal GPUs seemed to settle on the following: Power limit: 120% Memory clock: +575 MHz (11286 MHz effective) Core clock: Set at +0, but did whatever it wanted to do! It hit 1848 MHz and then seemed to back off to ~1785 MHz usually. I didn’t feel in control of what was happening and I suspect that reviewers may not actually get what’s going on. However I’m far from a professional journalist/overclocker and may be missing something obvious, so we’ll see if there are any revelations in the near future. For the time being, I’m still tinkering and will see if I can regain some control over the core clock. Also, Maxwell Titan overclocking settings (with which the top GPU could reach 92C(!) at 100% fan speed): Power limit: 110% Temperature limit: 91C Memory clock: +465 MHz (7938 MHz effective) Core clock: +60 MHz (1354 MHz effective) Slight thermal throttling occurred for Maxwell - only a few MHz, nothing that would noticeably affect results. Anyway, looking at the results, Maxwell and Pascal seem to overclock equally well at ~12%. Quite significant when rendering at high resolutions. As mentioned in the previous section, each game responds differently to Maxwell or Pascal overclocking and to be fair, Maxwell overclocks better for the most part in actual games (i.e. not Unigine). EDIT 26/08/16 So, I've had the Pascals installed in my HTPC for three weeks now and have discovered a few things. Firstly, the default fan profile can result in the GPUs becoming unstable even at stock. Now, the environment the GPUs is not the best in terms of ventilation, but it's far from unreasonable. I cannot run a Heaven benchmark for more than a few minutes without the drivers crashing; Mirror’s Edge Catalyst also will crash after a ‘while’. If I implement my own fan profile that allows the fans to run all the way to 100%, I get no crashes and the GPU boost clocks surpass 1800 MHz. I must point out this wasn’t an issue in my main PC case, which is extremely well ventilated. The GPUS also seem to start throttling at temperatures lower than the set throttle limits. I approximate that this happens at 80C. Deliberate overclocking wise, I can up the memory to +575MHz (11150MHz effective) and the Boost clock to +185MHz (Core clocks peaking at 2000MHz, typically sitting ~1950MHz). This does mean that the fans are typically up at 100% when I’m gaming, but the fans are actually quite reasonable and are mostly masked by the speakers when I’m playing. Of course, if I’m wearing headphones, it’s a complete non issue. I suppose a few of you feel I ought to water cool the GPUs, but in reality, I don’t need the tiny extra performance that will supply and I intend to replace GPUs on a generation by generation basis from now on, so don’t see the point. Conclusions I’d actually say it’s a hard call between four way SLI Maxwell Titans or SLI Pascal Titans when considering performance alone. Both set ups can outperform each other depending on the game, though obviously Pascal has the advantage of less power draw, noise and temperature. In the past, four way SLI problems would’ve clarified the issue, but it’s not the case anymore. In terms of value for money, well I’m sure everyone reading this already formed their opinion before even clicking on the thread. I’m under no illusion about how expensive these cards are but gaming is my main hobby and to me, the ends justifies the means. For the record, one Maxwell GPU was more expensive than my car at the time of purchase! What next? Well, since the anniversary update of Windows 10 has just been released, a fresh install is required methinks. In regards to upgrading, I think I’ll stick to a per generation upgrade cycle now for the main PC, shifting the older cards into the HTPC and selling the ones coming out of that on. Not for one of the Maxwell Titan Xs though - one GPU and the SLI bridge is going into my hardware display cabinet when it’s retired as a reminder of the four way SLI days which are now technically extinct. Let’s hope DirectX12 matures sooner rather than later. Now, what am I going to do with these cards? As luck would have it, my brother called me while I was playing with them and asked me to spec and build a powerful little PC for him. I think it’s time to sell on the trusty little ‘shoebox’ (my lovely SG07 lounge PC - 980Ti, 4790K) to fund something rather more punchy. To this end, I’m now building a new HTPC system based around the two Pascal Titans while the Maxwell Titans are returning to their home. Overkill? Perhaps, but now I will be able to play my games on the best possible set up for it. Thanks for reading; I hope you’ve enjoyed it and found it useful. I actually enjoy writing up my findings in my own amateur way as it lets me evaluate things rather than wondering if things have actually improved or not. If someone want to make a load of graphs from the data, let me know and I’ll plug them in if you like. Please let me know if I’ve made any mistakes or if you have any questions!