Hardware The Problem with Real World Benchmarking

I think that in essence this simply provides a viable solution to something, which to be honest is not as big a deal as the Babco would probably have you believe.

As for serial processing, just how out dated is that? Yes there will still be people using single core CPU's but at least have a benchmark capable of comparing those who use more than a single core. I can't help but wonder if Bapco have actually been updating Sysmark or just giving it a new version number...

It's a darn good thing we have the guys... err... are there any girls? at Bit-Tech to keep us informed and educated and will be good to see what you come up with for your own benchmarking software.

+1 for Bit-Tech

 

what about a website where you could go, select the programs you use and it gives you feedback on how a certain cpu/gpu/computer setup would fare? that way everybody can evaluate a computer/component on it specific needs.

 

Calling Crysis a bench is like saying that because a broken piece of code stresses a system it's a benchmark.

 

I personally think concentrating on CPU benchmarks is a mistake in itself. In my experience at work, a decent amount of fast RAM and a speedy harddrive is by far and away the most important part of getting a computer to run quickly.

We do quite typical office jobs (any high-end application is streamed via Citrix), and what I hear my staff complain about most are things like:

1) My computer takes too long to boot up
2) Opening *insert MS Office app* takes too long
3) My computer slows down when I have too much open

Above something quite simple, the CPU has pretty much no effect on any of the above.

 

Semiaccurate's take on this from some two weeks ago.

http://semiaccurate.com/2011/06/20/nvidia-amd-and-via-quit-bapco-over-sysmark-2012/

What they say and I think is implied here is that the benchmark was being skewed in such a way that made Intel always look good to the detriment of all others. This seems to have particularly rankled with AMD given their the gpu part of thier apu's way out perform Intel's equivilent ...with the benchmark singularly failing to show this.

 

It was implied at the end, but there was also no direct mention of the fact that Intel had complete say in what went into the benchmark, mainly because, as the semiaccurate article points out, the committee would deny any motions not brought by Intel. Even if the results of the 2012 benchmark are completely accurate (which, looking at the record of favoring Intel cpu ID's and ignoring all GPGPU calculations, is hard to justify), the fact that you have a system that accepts input from only one source is suspicion enough to justify all other chip designers pulling out.

 

Why would you ignore the biggest hardware player on the planet? Intel gets that respect for a reason.

 

My thoughts exactly. AMD can't compete on pure CPU performance with Intel so instead they go blowing out all the bells and whistles how GPGPU computing is everything.

Doesn't bode well for Bulldozer methinks :-(

 

Garcia Hotspur

 

From what I've seen so far, AMD CAN compete. The issue is with rigged compilers, cheating benches and the fact that the head of BAPCO is also the head of performance benchmarking for Intel. NOW does it make sense why they walked out?

 

I overall agree with this statement and everyone else who had one similar. HOWEVER, i don't think bulldozer has anything to do with it. i've seen linux benchmarks of bulldozer performance several months ago, and it is definitely good. bulldozer is meant to be cpu only, and if amd really thinks whining and complaining about how gpgpu tests aren't being done, those tests will not help bulldozer at all. llano's cpu portion is pretty bad. i am certain amd knew that llano was not going to defeat intel in cpu tests before they even benchmarked it themselves. but, i'm sure they definitely knew the video was going to win, and thats what they were really making a big deal out of. that being said, a benchmark suite that practically ignores gpu usage makes llano look terrible. when bulldozer is released i'm sure they won't be complaining so much.

i agree with some other posts saying you should pick the type of benchmark you actually perform. so if all i do is modern gaming and web browsing, why do i care about the other scores? if i only do ms office and virtualization, why do i care about the other scores? as bit-tech has mentioned, intel, amd, via, and even nvidia have all taken very different paths (mainly because intel is so anticompetitive so everyone was forced to). One of the greatest differences i've seen in cpu performance was intel's hex core xeon vs amd's 12 core opteron. both cpu models had drastically different strengths and weaknesses, to the point where they aren't really comparable, even though they are both server processors.

 

Meh... Owning a PC over the years is the same as the amount of time it takes to do chores around your house over the years. It takes no less time to clean your house now than it did 50 years ago, despite the vacuum cleaner, the 'special' dusters, techno-science polish sprays, non-smear window cleaners, etc, etc. By the same token, my pc NEVER loads quick enough, could always do with going just that little bit faster, and if I ever see that blue circley thing by my mouse cursor again...!!!

It is interesting that this benchmark is relied upon for mass Government and company purchases, because I thought those purchases were simply awarded to the company who offered the most units for the least money?

 

What a load of guff I don't trust the reviews of this site and i don't think this site is impartial at all.
Plus benchmark talk from you lot is gay

 

People using the word 'gay' as a negative attribute are gay.

 

Oh darling. I am most certainly liking your work! +1

 

I really think it's a moot point, seriously; any PC from the last few years will run youtube, web browsers and spreadsheets fine, and people don't die if it takes a second longer to load a video... face it, the user-perceivable difference between a slightly older machine and a raging multithreaded quadcore beast in average day-to-day apps is next to zero.

On the other hand...

If you are benchmarking it's precisely to see how it will run demanding applications that most users dont need or care about--otherwise you wouldn't bother because you already know it will run and it doesn't matter. So I agree that benchmarking common hardware with atypically demanding scenarios is unfair and completely misrepresentative of what your average user can expect, and I understand why companies are backing out...

And yes, though "can it run Crysis" is a completely abused joke that I wish would just go away already, it's still definitely relevant because even years after its release, alot of brand new latest-gen hardware still struggles with it.

 

Have to agree here.

After working as a IT helpdesk bod for most of a decade, a 'slow' pc is almost invariably caused by lack of ram / slow hdd, rather than cpu.

Only 6+yr old cheap celerons were ever a lack of cpu power problem that i came across, even then a ram/hdd upgrade would minimize the impact.

Out of the market now, but i could see cheap 32gb ssds keeping aging corporate pcs going for way past their usual working lifespan, especially if coupled with a ram upgrade later on. It only really needs to hold the OS and core apps locally, anything else can sit on a server.

 

That's true of Llano, but total conjecture for bulldozer. And I don't think bulldozer is going to be all that great compared to Sandybridge and Ivybridge or even older Nehalem when considering how long it's been around.

The reality is, in the day and age of gpgpu, if office software is going to be the key selling point for cpu performance, which is what AMD wants to win some business, they are making total sense. Llano makes perfect sense for a PC in the office just like the lower end CoreIs, although AMD may be cheaper, finally have the power consumption to compete and should not be resting on their laurels when it comes to opencl or they might as well give up and let Intel steamroll them by not trying to prove their point.

Again, bulldozer has nothing to do with this...

Exactly, the machines at my work would be so much faster with an ssd. All things over the network are load balanced and in raid configurations, so they all scream away... the problem is the 3.5" drives inside them. I just upgraded to an SSD and the difference is huge.

 

The last 3 companies I have worked for over the last 10yrs don't use computers like that anymore. So how useful is something like that to business I wonder.

We are all on virtual servers, I have a core i7 laptop with 8Gb of RAM, half the ram is useless as the machine is running Windows XP, the only process it runs is Citrix and the monitoring software, if there was a netbook with a 1920x1200 screen the company probably would have farmed those out and saved a lot of dough.