An open letter to the graphics ecosystem

http://www.bit-tech.net/columns/2005/10/21/open_letter_graphics/1.html

 

Good article I must say. Really puts it into perspective about how much work goes on behind the scenes of bit-tech! Can't say I have any answers, but it was a good read, thanks!

 

Well, I don't have the definitive answer to problems, but a good article, I do agree on the bit-tech method, and who knows, this article might lead to better games?

You mentioned about using mid-range systems or not, tbh I think a lot of people here have pretty much high end systems, I know you have limited time, so perhaps some of the forum members here with mid range systems could produce their own reviews using their system? Gives you more time to concentrate on creating reviews on high end systems?

Also, It would be interesting to know how a day goes in the life of a bit-tech staff, or the process you go through from hearing about a new graphics card or game, to the point you finish creating the article.

 

Nice article

All makes sense, I really like the idea of real world testing. Its so annoying when you have to look at a bunch of specs and graphs and it doesnt actually answer the question, is this one actaully worth spending 25% on? is it really that much quicker?

Im not sure if its worth using mid range systems aswel, it would be a nice addition but I dunno if its worth it at your end, maybe if it was just a brief test just to find if there any potential bottlenecks. I suppose if your testing a 6600 GT its seems a bit stupid to stick it in a 2+ grand computer when your trying to do real world tests, it may work really well in it, but then when somones fits it in there fairly average pc runs nowhere near what they expected.

 

I've been trying to match CPUs up with graphics cards for a while now - it doesn't make sense to do 'real world' testing without using a CPU that is reasonably realistic.

 

I do not think that many reviews on hardware review sites, are testing realworld benchmarks. They say they are, thats true. But when i take a look at the spec list of the test system. I am thinking: woooott!?

Only 1 of the 1000 people who play games have a gamepc with an overclocked on crack: fx75, 2x 7800GTX EE in dual 16x SLI and 2 gigs of superiour timed ram. What are they thinking with realworld?

Well... Not my "real" world.

 

I tend to appreciate that mid-range systems are tested, and if they weren't I'd probably stop reading the reviews. Many people on these boards are enthusiasts that still have the 6800LE and 6600GT...and there are quite a few of us who are opposed to honestly going out and dropping $400 on a graphics card. So though nobody really needs to see FEAR running on a 6200, the mid-range setups still provide valuable information to many of us.

 

I'm trying to grasp what you're saying here - whether this is a general comment about 'other' review sites, or about ourselves. Don't forget that our job is to 'show you' the newest products that are released - not just the fastest, the slower, more affordable ones too...

So, what would you like us to 'review' a GeForce 7800 GTX/Radeon X1800XT on? The slowest Sempron or Celeron that we can get our hands on with 256MB of your finest 'major' memory on the cheapest PCI-Express motherboard that we can find?

You may see the system specification for one of those reviews and think... "omfg, wtf"

That is because they are the fastest-available video cards around and we try and balance our system against what we're reviewing. When I look at something a bit more 'mid-range', like, say, the X800GT that was linked... We paired that with a '3700+'... ok, maybe a little bit on the high side from the PR ratings, but I'd hazard to guess that a lot of readers out there have Athlon 64 3000+'s overclocked higher than our 3700+, running at a steady 2.2GHz.

I'm just interested in yours and everybody elses thoughts - we stive to make things even better than they are now. If the information isn't useful and relevant, then we're barking up the wrong tree.

I'm not having a go, I'm merely trying to find more and more feedback to help make our systems a little bit more 'realistic' and balanced for what you guys would buy.

 

Good article. And I agree about more mid-end type review setups. Like me, I'm on a 2500+ with a 9800pro and a gig of Corsair value ram. Not exactly bleeding edge.

 

Yer id call mid-end an oc'd mobile barton/3000+ A64 (2ghz min), gig of branded ram.

The question is whether or not your going by price\components spec or actual performance, if you got a phased mobile barton at 3.0ghz with 2gig of ultra low latency ram i'd call that high end.

I suppose you could do what you done before and just use a lower clock and slacker timings.

 

nah, the barton core is history... just like AGP

 

Yer true, but some of us are still stuck with that kit ... until the end of the month hopefully

 

yes, but with the way that the industry goes... anything that isn't Athlon 64 or Pentium 4 and PCI-Express based could well be considered legacy seeing as those two have been around for ~15-18 months now.

 

I think the way forward for bit-tech would be to have a database of hardware that you can use to pull up a comparison with whatever cards you want. So if i wanna know if it's worth upgrading my old 8500, say, i can pull up a whole bunch of new cards (and some not so new) and compare them myself. If it's done well it'll be by far the best way of knowing what's the best course of action. That way you can make the initial review fairly basic - one reasonably high end rig tested in the way you do now - but then have a few raw numbers to throw in the db for stuff like timedemos, 3dmark, etc.

Hmmm, i was going to elaborate more and do a quick edit, but dinner's ready!

 

Well... That would be something kewl to do!
Do not forget that users see/read a review about the fastest and laters videocard. And its comming out the benches like it is "the 10th wonder of the world". People buy this card and slap it into their system...

But wait! this system does not have the latest and greatest money can buy hardware inside to actualy make use of the 10th wonder...

See, It makes no sence that you need to buy an new proc, memory, mainboard and such, just the keep the also very expensive videocard running like the benches say it would. Maybe some people are diving in their money-pit and can do this, but most "normal" people do not. They just insert the card into the socket a amd athlon3000+ pc they bought last christmass.

I hope you understand what I am trying to point out here. (my english is not perfect but i am trying to)

 

I don't think we will be returning to the past in the way that we do our reviews, so there will not be any time demos, and no 3DMark results either. A database would be nice, but seeing as we focus on actual gameplay with systems that are reasonably representative of what we would say the 'fastest' system someone is likely to pair a particular video card with, it is increasingly difficult to actually put a huge comparison table down on paper.

I have some ideas as to how to cater for your needs, but in practice, I feel that there's at least another full time job required for both implementation and keeping it up to date...

My idea would revolve around the resolution you like to game at and then we could list a choice of video cards based on what we found at the time... again, the only problem is keeping it up to date when new games come out. I don't have the time to evaluate every video card we've ever reviewed (again) when new games come out that are more demanding (like FEAR, for example...)

 

If I remember back, people have found that our settings are actually below what they play the game at with slower hardware than what we were using in our testing system. People have always asked why our settings are so low, its because we're more concerned about worst-case than best-case, even if the system we're using to test the video cards is at the very high end.

 

Nice article, I really approve of the way gfx cards are reviewed at Bit and [H] now. I would *really* like to see more reviews of cards in what I would call the lower mid-range section - £40 to £80, as I think this is what a lot of people, especially those who like playing games now and again and like eyecandy but aren't prepared to pay £££'s for.

A big round-up of modern games on the likes of 9600 Pro\XT, X600, X700, GeForce 5700, 6200, 6600 etc, with lowest prices on these cards found from UK sites, would be brilliant. Setup on a more mid-range system - a 3000 Venice\2*512Mb would seem most appropriate to me. This information seems really hard to find in one place, yet I expect its the range of graphics card which sells in the highest quantity.

 

Congratulations on an excellent article! If I didnt know any better I'd swear you were putting the consumer's interest ahead of the manufacturer's.

Keep up the good work.

 

Well that would certainly be no easier!

My suggestion would actually be really easy to maintain and not too hard to setup - would need fine tuning for presentation etc. Thing is, if i want to upgrade it's no good just seeing a review that compares a 6600gt with a x800xl (or whatever) because I don't know what anything else is capable of (without looking at more reviews). What do i care about the minor difference between two cards when I could get a shitload better for another £50 or not much worse for £50 less. Everything's relative. By all means write a review the way you feel it's best but to 90% of people it's actually no good on it's own, so why not embrace that fact and create something of use.

tbh, I'm fed up of you guys blowing your own trumpets about how well you review stuff (i tink this is the third article in as many months covering this issue) when I'm sat here as a user/reader thinking how i don't like your reviews. It bugs me. It's your choice to make a rod for your own backs and play a game through completely, twice, but i can assure you i couldn't care less. Pick a section which is pretty intensive and play that instead - you'll still be 95% right. I'm one of the majority of people that doesn't actually upgrade all that much and seeing a review of the latest graphics card is of no real consequence to me until i'm in the market for one. So, at the minimum i just want to be able to scan a review, get a feel of what it's capable of and move on. Something i find hard to do with bit-tech's reviews.

I was going to actually delete most of that rant, but i feel like leaving most of it up just to see what reactions people have.