1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Hardware The Problem with Real World Benchmarking

Discussion in 'Article Discussion' started by arcticstoat, 5 Jul 2011.

  1. tad2008

    tad2008 New Member

    6 Nov 2008
    Likes Received:
    I think that in essence this simply provides a viable solution to something, which to be honest is not as big a deal as the Babco would probably have you believe.

    As for serial processing, just how out dated is that? Yes there will still be people using single core CPU's but at least have a benchmark capable of comparing those who use more than a single core. I can't help but wonder if Bapco have actually been updating Sysmark or just giving it a new version number...

    It's a darn good thing we have the guys... err... are there any girls? at Bit-Tech to keep us informed and educated and will be good to see what you come up with for your own benchmarking software.

    +1 for Bit-Tech
  2. alpaca

    alpaca llama eats dremel

    27 Jan 2009
    Likes Received:
    what about a website where you could go, select the programs you use and it gives you feedback on how a certain cpu/gpu/computer setup would fare? that way everybody can evaluate a computer/component on it specific needs.
  3. b5k

    b5k New Member

    10 May 2009
    Likes Received:
    Calling Crysis a bench is like saying that because a broken piece of code stresses a system it's a benchmark.
  4. Bauul

    Bauul Sir Bongaminge

    7 Apr 2007
    Likes Received:
    I personally think concentrating on CPU benchmarks is a mistake in itself. In my experience at work, a decent amount of fast RAM and a speedy harddrive is by far and away the most important part of getting a computer to run quickly.

    We do quite typical office jobs (any high-end application is streamed via Citrix), and what I hear my staff complain about most are things like:

    1) My computer takes too long to boot up
    2) Opening *insert MS Office app* takes too long
    3) My computer slows down when I have too much open

    Above something quite simple, the CPU has pretty much no effect on any of the above.
  5. fingerbob69

    fingerbob69 Member

    5 Jul 2009
    Likes Received:
    Semiaccurate's take on this from some two weeks ago.


    What they say and I think is implied here is that the benchmark was being skewed in such a way that made Intel always look good to the detriment of all others. This seems to have particularly rankled with AMD given their the gpu part of thier apu's way out perform Intel's equivilent ...with the benchmark singularly failing to show this.
  6. Lazy_Amp

    Lazy_Amp Entry AMD Engineer

    17 Jul 2010
    Likes Received:
    It was implied at the end, but there was also no direct mention of the fact that Intel had complete say in what went into the benchmark, mainly because, as the semiaccurate article points out, the committee would deny any motions not brought by Intel. Even if the results of the 2012 benchmark are completely accurate (which, looking at the record of favoring Intel cpu ID's and ignoring all GPGPU calculations, is hard to justify), the fact that you have a system that accepts input from only one source is suspicion enough to justify all other chip designers pulling out.
  7. Snips

    Snips I can do dat, giz a job

    14 Sep 2010
    Likes Received:
    Why would you ignore the biggest hardware player on the planet? Intel gets that respect for a reason.
  8. SlowMotionSuicide

    SlowMotionSuicide Come Hell or High Water

    16 May 2009
    Likes Received:
    My thoughts exactly. AMD can't compete on pure CPU performance with Intel so instead they go blowing out all the bells and whistles how GPGPU computing is everything.

    Doesn't bode well for Bulldozer methinks :-(
  9. Bleech

    Bleech New Member

    5 Jul 2011
    Likes Received:
  10. PingCrosby

    PingCrosby New Member

    16 Jan 2010
    Likes Received:
    Garcia Hotspur
  11. KayinBlack

    KayinBlack Currently Rebuilding

    2 Jul 2004
    Likes Received:
    From what I've seen so far, AMD CAN compete. The issue is with rigged compilers, cheating benches and the fact that the head of BAPCO is also the head of performance benchmarking for Intel. NOW does it make sense why they walked out?
    thehippoz likes this.
  12. schmidtbag

    schmidtbag New Member

    30 Jul 2010
    Likes Received:
    I overall agree with this statement and everyone else who had one similar. HOWEVER, i don't think bulldozer has anything to do with it. i've seen linux benchmarks of bulldozer performance several months ago, and it is definitely good. bulldozer is meant to be cpu only, and if amd really thinks whining and complaining about how gpgpu tests aren't being done, those tests will not help bulldozer at all. llano's cpu portion is pretty bad. i am certain amd knew that llano was not going to defeat intel in cpu tests before they even benchmarked it themselves. but, i'm sure they definitely knew the video was going to win, and thats what they were really making a big deal out of. that being said, a benchmark suite that practically ignores gpu usage makes llano look terrible. when bulldozer is released i'm sure they won't be complaining so much.

    i agree with some other posts saying you should pick the type of benchmark you actually perform. so if all i do is modern gaming and web browsing, why do i care about the other scores? if i only do ms office and virtualization, why do i care about the other scores? as bit-tech has mentioned, intel, amd, via, and even nvidia have all taken very different paths (mainly because intel is so anticompetitive so everyone was forced to). One of the greatest differences i've seen in cpu performance was intel's hex core xeon vs amd's 12 core opteron. both cpu models had drastically different strengths and weaknesses, to the point where they aren't really comparable, even though they are both server processors.
  13. greypilgers

    greypilgers New Member

    23 Jan 2011
    Likes Received:
    Meh... Owning a PC over the years is the same as the amount of time it takes to do chores around your house over the years. It takes no less time to clean your house now than it did 50 years ago, despite the vacuum cleaner, the 'special' dusters, techno-science polish sprays, non-smear window cleaners, etc, etc. By the same token, my pc NEVER loads quick enough, could always do with going just that little bit faster, and if I ever see that blue circley thing by my mouse cursor again...!!!

    It is interesting that this benchmark is relied upon for mass Government and company purchases, because I thought those purchases were simply awarded to the company who offered the most units for the least money?
  14. timevans999

    timevans999 old modders friend

    16 Mar 2010
    Likes Received:
    What a load of guff I don't trust the reviews of this site and i don't think this site is impartial at all.
    Plus benchmark talk from you lot is gay
  15. Nexxo

    Nexxo Stopped treating this country as if it was his own

    23 Oct 2001
    Likes Received:
    People using the word 'gay' as a negative attribute are gay. :p
  16. SexyHyde

    SexyHyde Member

    24 Jul 2009
    Likes Received:
    Oh darling. I am most certainly liking your work! +1
  17. OCJunkie

    OCJunkie OC your Dremel too

    19 Apr 2011
    Likes Received:
    I really think it's a moot point, seriously; any PC from the last few years will run youtube, web browsers and spreadsheets fine, and people don't die if it takes a second longer to load a video... face it, the user-perceivable difference between a slightly older machine and a raging multithreaded quadcore beast in average day-to-day apps is next to zero.

    On the other hand...

    If you are benchmarking it's precisely to see how it will run demanding applications that most users dont need or care about--otherwise you wouldn't bother because you already know it will run and it doesn't matter. So I agree that benchmarking common hardware with atypically demanding scenarios is unfair and completely misrepresentative of what your average user can expect, and I understand why companies are backing out...

    And yes, though "can it run Crysis" is a completely abused joke that I wish would just go away already, it's still definitely relevant because even years after its release, alot of brand new latest-gen hardware still struggles with it.
  18. cyrilthefish

    cyrilthefish New Member

    15 Apr 2004
    Likes Received:
    Have to agree here.

    After working as a IT helpdesk bod for most of a decade, a 'slow' pc is almost invariably caused by lack of ram / slow hdd, rather than cpu.

    Only 6+yr old cheap celerons were ever a lack of cpu power problem that i came across, even then a ram/hdd upgrade would minimize the impact.

    Out of the market now, but i could see cheap 32gb ssds keeping aging corporate pcs going for way past their usual working lifespan, especially if coupled with a ram upgrade later on. It only really needs to hold the OS and core apps locally, anything else can sit on a server.
  19. law99

    law99 Custom User Title

    24 Sep 2009
    Likes Received:
    That's true of Llano, but total conjecture for bulldozer. And I don't think bulldozer is going to be all that great compared to Sandybridge and Ivybridge or even older Nehalem when considering how long it's been around.

    The reality is, in the day and age of gpgpu, if office software is going to be the key selling point for cpu performance, which is what AMD wants to win some business, they are making total sense. Llano makes perfect sense for a PC in the office just like the lower end CoreIs, although AMD may be cheaper, finally have the power consumption to compete and should not be resting on their laurels when it comes to opencl or they might as well give up and let Intel steamroll them by not trying to prove their point.

    Again, bulldozer has nothing to do with this...

    Exactly, the machines at my work would be so much faster with an ssd. All things over the network are load balanced and in raid configurations, so they all scream away... the problem is the 3.5" drives inside them. I just upgraded to an SSD and the difference is huge.
  20. sandys

    sandys Well-Known Member

    26 Mar 2006
    Likes Received:
    The last 3 companies I have worked for over the last 10yrs don't use computers like that anymore. So how useful is something like that to business I wonder.

    We are all on virtual servers, I have a core i7 laptop with 8Gb of RAM, half the ram is useless as the machine is running Windows XP, the only process it runs is Citrix and the monitoring software, if there was a netbook with a 1920x1200 screen the company probably would have farmed those out and saved a lot of dough.
Tags: Add Tags

Share This Page