Discussion in 'Article Discussion' started by bit-tech, 13 Jun 2019.
Arma 3 with YAAB.
For gaming specifically, minimum and average frametimes, and frametime spread. Throw FCAT VR at some games.
Just stick to real software like Handbrake, Hamster, Blender etc.
What edzieba said. In addition to that, would be interesting to see at least one game put through the paces twice, once with and once without all the stuff streamers have running in addition to a game.
How easy is it to script automatically and does it have any known bias?
Frame timings is something we're looking into.
That's the plan so far.
The streaming bit is actually a great idea. Should also load up more cores. I'll look into a way of making this reproducible.
Not sure if this is considered a "benchmark" but, specifically for heat & noise charts when reviewing CPU coolers, I would like to see standard PWM control alongside the normal 100% fan measurements.
Edit: Also, is there a way of including a standardised heat measurement in CPU tests like you do with power consumption?
1080p and 1440p should both be tested IMO as from what I've read some games hit a FPS wall so it should lessen the chance of having 10+ 180fps results and 1440p would give an idea of how well a CPU scales upwards.
Civilization 6's AI and DX12 benchmark would feature somewhere.
At least one Vulkan title.
And I'm going to throw in a wild card and suggest something that's cross platform (Linux/Windows) to give an idea of how the underlining OS effects performance, I've not got a clue how viable that is mind you.
As boost is quite a complex thing on these anything you can do to show what the boost clocks speeds are for specific loads, how they drop off per core and temperature and how things like PBO and good cooling impact.
The how and what to use is the question.
Plotting cpu frequencies automatically is something I've recently worked out how to do and will be implemented but the boost point makes a lot of sense.
A more involved but interesting test (probably better fitted to its own feature) would be to take a few 'standard' daily workloads (e.g. gaming, office productivity, image editing, video editing) and stick a system monitor in there throughout the day to continuously record how many - and how loaded - threads are throughout a typical working day, both peak and average. This would be kind of a 'reverse' benchmark, testing multiple workloads against one CPU (the upcoming 3950X would,be perfect for this, having a high core count whilst not being overly bottlenecked in single-thread workloads like a Threadripper or MCC/HCC Xeon would be) but would give some good rules of thumb for "I do X all day, so maybe I won't see any benefit from more than Y cores".
I've done the same thing (along with other stats, like temperature) - it's so easy in Linux (or any POSIX-alike, I'd guess) but I'm willing to bet it's a right pain in the proverbial under Windows.
You can use performance/resource monitor in Windows, it can log it all, not used it for a few years but is easy enough to setup.
The difficulty I guess is finding a repeatable, script benchmark to stress it all.
A few annoyances but not too bad.
The hardest part is that ryzen doesn't play well with the software but I've sorted that out now
Noice. My favourite, and least-comprehensible, graph:
It looked fine until I had to squish it down to a few hundred pixels wide!
that's cool. lacks XKCD styling . plotting clocks and temps over time is the current goal. The next step is a USB monitor so the tester can just launch the script and go home then on return have all the data. Power readings is also something I want to do but getting a power meter that has a USB interface i can interact with in python is proving to be tricky. What are you using for those graphs?
Very much deliberately, yes.
Here's one DIY version for the discerning hacker, or this is a little more professional. If you'd prefer something off-the-shelf, find a UPS and take readings off that - I do that on my server, although maybe don't get the cheapest of the cheap 'cos mine reads anything below 34W as 0W...
Alternatively, it should be pretty easy to get readings out of any of the Wi-Fi-connected smart plugs on the market at the moment.
For gaming, can we add ultrawide resolution?
I have had more time to think about my previous point about heat (and therefore noise).
As multi-core is now more relevant than previously but single core usage is still a major usage factor, it is possible to test temperatures with both single core and multi-core utilisation?
In addition, is it possible to show temperatures with, say, 50% utilisation in addition to the existing idle and 100% figures?
My reasoning is that it would show CPU's that get really hot (which as we all know will increase cooling requirements and potentially fan noise) when using all cores in comparison to when only using a single core and with a 50% utilisation chart, it would show the heat curve between idle and full load. Potentially power consumption can be measured at the same time.
Is this something that others would find usefull?
My motive is simply about heat, power consumption and noise, these things are big bugbears of mine.
3D mark 01
With Zen 2 specifically I'd like to see "multi-tasking" benchmarks to see how memory bandwidth limits or does not limit the Zen 2 cores. By multi-tasking, I often find myself running VMs, data analysis (intense FP arithmetic and RAM usage), web browser, ffmpeg (command line), and image editing/processing (GIMP if not hand written in C/C++ based languages) all at once. I'm not sure there is a single benchmark that can replicate these workloads but given that today people often have dozens of tabs in browsers along with several other mixed workload programs running it would be nice to see a benchmark that tests these together. With the Infinity Fabric and RAM clock divider coming into play I'd also like to see a more academic investigation into RAM latency as a function of RAM subtimings, IF clock, etc.
For the 8+ core Ryzen parts, it would be interesting to look into thermal density. Did AMD design Zen 2 so one chiplet is fully loaded before the other gets any work to do? Will the I/O and cache die contribute a lot of heat (dependent on IF clock and voltage)? Depending on what you find this could rehash some old questions on TIM placement, heatpipe orientation on the base plate of a heatsink and etc.
I'm not much of a gamer.
Separate names with a comma.