Now I know I'm going to catch a LOT of flak for posting this but I've already made comments about Bit-Techs apparent decline in other posts which people have agreed with aswell, so I feel after reading the only other hardware site I rely on posting about Bit-Tech in a negative way, it might be time for a bit of an open debate to begin. Before I start I would like to say I have been keeping an eye on all the changes around BT for quite some time now, I've read through the columns and articles relating to changes, and I've watched as review standards have changed (and in my opinion dropped) and layouts and formats altered. I realise many of these changes were done with the best of intents to try and improve quality in some cases and to streamline the operation of the site in others. But I'm sorry to say, it has'nt worked, too much has been sacrificed in my opinion and although review standards are on the rise again, it may have come to late. I began my weekly hardware browse with BT this week and then moved on to [H]ardocp, these are the two sites I have relied on as my first stops each week for hardware news, and occasionally some other things, for both the forums and the site content, so you can imagine my surprise when I read this over on the [H]... Bit-Tech Drops Real World Testing One of the leading hardware enthusiast websites in the UK is dropping their real world gameplay comparisons noting resource issues and reader disinterest. I can tell you this, Bit-Tech did a very good job of laying out why they were changing their video card testing approach back in 2005, but I guess they are having a change of heart as stated by Bit-Tech staffer Richard Swinburne. Q: Where's the apples and oranges stuff you guys used to do? It was the best indication of how performance scales at quality and the only thing I personally feels how a GPU really performs in the real world. Benchmarks and FPS graphs just don't do it for me. A: Very few people liked Oranges to Apples so we went back to apples to apples. The best playable settings just physically takes far far too long to test, Tim would be writing the article for three weeks if that was the case. It is sad to see them move away from what I think is an obviously superior form of evaluation. You can read my thoughts on the Benchmarking Future from 2003. I think that our latest video card evaluation of the Radeon HD 2900 XT was a true testament to what we have been doing for 4 years now. The fact of the matter is that you don’t buy $400 video cards to play games at 12x10 resolution, and if you do, I would highly suggest you invest in a new display before you upgrade your video card. The fact is that you don’t buy $400 video cards to play games without AA, AF, incredible HDR lighting, or detailed shadows. Real world gameplay testing shows you exactly what video cards give you the best gaming experience. The video card market is not about frames per second any more; it is about smooth gameplay and immersive digital environments. If someone is suggesting that you spend your money on a video card because it got a better framerate, it could very likely be a misguided opinion. Did our recent evaluation of the 2900 XT give you a different conclusion than many other sites? You are damn right it did. When it comes to immersing the gamer in a 3D environment, the 2900 XT simply does not do the job as well as the less expensive 8800 GTS. Did the 2900 XT give faster framerates at “middle” resolutions or with no AA as proven by many canned benchmarks? Yes it did. What I can’t figure out is why anyone cares that it did. I am sad to see Bit-Tech pulling back from their testing stance, but I will say this, “Welcome to all those Bit-Tech readers that value real world gaming evaluations. You have found a new home, and we are damn glad to have you here.” Gaming experience provided by the hardware we evaluate will continue to be our focus. Now when I see my only other favourite resource openly commenting on BT's performance, after I have made similar statements on these forums, I worry. BT is still one of the most amazing resources, and I wont be disparaged just yet, regardless of the review issues, BT still has some great modding content and sime nice quirky articles with good commentary, I'm just not sure how much longer that will be enough. [H] has the highest review standards on the planet in my opinion and is the standard in reviewing that all hardware sites should be aiming for. I'm even willing to accept that some people dont see it that way, and that there are other perfectly good methods to review styles and testing methodology, but I'm also aware that [H] is a very honourable well run site, and that Kyle would not be making a comment like that if he was'nt as concerned as I am about the way things have been going lately. The last point I would like to make is in regards to the forums, I used to browse BT forums loads, and as with the [H] the content has remained constant, but the one thing there has been a down turn in is cooling. Now this is true of many sites, the interest simply isn't there anymore as it was a few years ago, so for cooling enthusiasts like me we have to work that much harder to find interesting stories and posts. The point I would like to make is that stems directly from the cooling techniques in review on any given site, BT and [H] both used to do a very good job keeping on top of the more extreme cooling methods aswell as provide a good look at current water and air cooling, but again BT has failed in this recently and [H] has remained constant, with perhaps a slight down turn in the extreme end of the colling range, largely gauged by the lack of consumer interest in it these days. To me cooling and hardware are the core of any hardware and modding site, the rest is just nice extras, forums are a natural extension of that and the number of related topics will only increase when the main site content increases accordingly. Now I would like to reiterate what I stated at the begining of this post, I'm not trying to step on anyones toes here, and I am fully aware of why some of the changes that have taken place around BT have occured, what I'm doing with this post is stating a genuine concern as to the apparent falling standards around BT, not just in the past few months but over the past two to three years. I don't want to see BT go any further down, i want to see it come back to what it was, to continue having an excellent modding resource AND an excellent hardware resource in one site, with a fantastics community to boot. So to anyone reading this please take what I've said with a pinch of "sugar", not salt, and then comment away on what you agree and disagree with. Last point "DEBATE", don't flame, that's not why i posted this. Cheers
I dont really post here this often but I do think many Modding forums have declined they all seem to me to have become abit more uptight and less relaxed.
I never saw the site back in the old days, and have no idea what is meant with 'apples and oranges'. I have to say that it is 'changing' here at B-T, but I have no idea wether its getting better or worse. Also, I have to say that most people will be gaming at 1280 x 1024 or 1650 x 1200 and I for one never use AA or AF...
I do miss the old method of testing, however, there are resource issues generally that Bit-tech have to deal with, with hardware constantly changing, drivers constantly changing and so on, every single review that I have read from Bit has had some form of "We had this issue and contacted the manufacturer who we helped to resolve the issue" sometimes it doesn't reach a conclusion either, I remember some motherboard reviews were even at the end of a 4 week testing period the board wasn't stable.
Yeah, Bit-Tech still offer some of the best reviews out there and I know how hard the team have been working, particularly Tim on that HD2900XT article. It is a bit of a shame though that they had to drop the "playable settings" part of the review though.
I must admit to feeling a bit puzzled. You said that you have voiced your concern about BT's apparent decline for some time, Buck, and [H] now appears concerned too. On the other hand you appear to have found in [H] a decent site that scratches your hardware itch, so to speak; [H] also states that they are happy to continue to fill the niche that they perceive BT to have abandoned. BT readers who feel likewise are welcomed over there. So where is the problem? It seems to me that this is an issue of consumer demand and market forces: some people like apples, others prefer oranges. BT cannot cater a whole fruit basket for the whole market, so it has decided to meet the demands of a certain group of readers. Those who feel that does not quite meet their needs, find those catered for by [H]. Everybody happy. According to various analysts, the number of PC users is expected to hit or exceed 1 billion by 2010, up from around 660 million to 670 million today, fueled primarily by new adopters in developing nations such as China, Russia and India. BT sports 31,535 members (at last count, and rising), and [H] even sports 85,330 (no doubt steadily rising too). That is 116,865 catered for; only 669,883,135 more to go... I think both sites are doing rather well for themselves, as are sites like ProCooling and 2CPU, each of which caters for a specific set of interests. Now I don't want to sound all complacent like "there is plenty more fish in the sea"; BT does care about its audience after all, but I for one just don't see a problem. I can't help but wondering: if you are happy with going to [H], and [H] are happy with what they are doing, then why all this concern with BT? Why are you all worrying about BT when you have a decent alternative available in [H] and [H] are quite happy doing what they do? It seems to me there is an issue being made of what is not an issue. It seems to me that this "concern" is just a veiled way of criticising others while singing one's own praises. If you are happy with [H], and [H] is happy with themselves, there should be no reason to express "concern" with BT.
Well said Nexxo - I like BiT just the way it is atm, and I'm sure it'll continue growing as it has done for many years now. If you like [H]'s way of doing things, then please, don't worry about leaving. Personally, I prefer the current method of reviewing, because not only do we get apples-to-apples, but we have things like decent image quality reviewing and the likes.
I have to say that I find the [H] article a little like Tesco being concerned that Sainsbury's are changing their carrier bag designs... I like bit, it's one of the few sites I visit regularly and participate in the community, but if there's some info I need that bit doesn't provide, then I'll look elsewhere for it. It's the same with supermarkets: if Tesco doesn't have what I need to cook dinner, I'll go to Sainsbury (or, heaven forfend, even Asda or Morrisons); I won't stop shopping at Tesco, nor will I stop visiting bit (sorry to disappoint on that last one )
But bit-tech still has comprehensive good reviews that give lots of information. [H] doesn't. The best playable settings were still really useful though (surely it wouldn't take that long).
All fair points guys, and I don't intend to stop using BT as a valuable resource, my "concern", and i do mean concern Nexxo so please don't try and make it seem otherwise, arises from the cross comparison between [H] and BT I have often used for reliable information in the past. Yes there are plenty of sites out there that deal with different types of benchmarking, many cooling sites for example which I rely on for the kind of detailed watercooling tests I require and would'nt expect to find on BT or the [H] currently. I'm fully aware of the financial and time constraints such testing would place on many sites trying to cater to a particular need. For me it's not a question of either/or as has just been stated, it's more that in the past I've been able to gather a very clear picture from comparing reviews and articles from both sites, very often the conclusions have been identical even where testing setups and methodologies have differed slightly. However this is'nt the case anymore, either test setups have changed to much to use as a reliable cross comparison or certain aspects of a review no longer exist, in both [H] and BT reviews, and I find myself looking at four or five sites to gather the same information I would have gotten from two. I'm not an avid forum poster as you might have guessed, I find myself ghosting both the [H] and BT forums an equal amount, I find more modding content here, and more cooling content there, that's the way it has always been and is what originally lead to me to use both sites as my main resources. But i do find making a post like I have here is useful for those readers on these forums and on the BT site aswell as the [H] site who may indeed end up in a "well I'll just go over there" line of thought as you've just pointed out. Both sites have a great deal to offer in different ways, I would just like to see that cross comparsion i used to be able to rely on return, and yes of course it's just my view.
I'm sure that your concern is genuine, even if (in my opinion) undue. I was referring to [H]ardOCP's "concern". But back to your concern, which... So you are worried because different sites now employ different test methodology, and thus may arrive at different conclusions? Welcome to the wonderful world of scientific research... But seriously: your argument, if I understand correctly, is basically a scientific one about replicability of results. But while replicabililty of results undeniably is a cornerstone of scientific research, so is symmetry: whether the same results and principles apply under very different conditions. One could therefore equally ask what is the point in many different sites doing exactly the same thing with exactly the same hardware to arrive at exactly the same conclusion, all the time. There is something to be said for the same hardware being subjected to different hypotheses and different test conditions, not only because this may reveal flaws in your original method or hypotheses, but also because it reveals more about the hardware's (variability of) performance in different situations. Another problem is how to decide which is the "best" test design/method. Just because BT employs a different test methodology than [H], does not necessarily mean that BT's approach is worse than that of [H]. In that case, I could make the (equally invalid) argument that BT has progressed to a more valid testing approach, while [H] is still using an outdated, dogmatic methodology. Your job, researcher (should you choose to accept it), is to assimilate all the varying bits of research into an overall picture, balancing the relative significance of different results under different conditions and obtained by different methods as they apply to your particular circumstances. This is not easy, but then again, real-life conditions never are. What you are saying is that you value testing of replicability of results over testing of their symmetry. That's cool --it may be more relevant to how you use your computer. But it's a big world out there, with many, many different configurations of PC being used in many, many different ways. What is good for the goose may be no good for the duck. Different strokes for different folks, and all that. BT caters perhaps more to one crowd, while [H] caters more for another. While the wiser amongst us may appreciate and value the variety of both.
I think there has been a decline in all modding forums for quite a while, the only mods that seem to get attention anymore are those that require you to have access to massively expensive machinery or require you to quit your job to have a hope of completing by hand. then there are of course the gimick mods. modding is a very individual thing, you want to make something that is yours, that stands out from the crowd and that there is nothing like anyware else on this planet. 5 years ago when I got into modding you would see something massively innovating on the modding scene every few days, now you would be lucky to find something that has not been done before every month. There is only so long we can keep innovating with what we put computers in/ how we cool them etc. when we all run out of ideas we will either start copying each others mods which defies the very basis of modding or we stop modding all together. As for bit-tech i'm in two minds about the development of the site, alot of things have Defiantly changed for the better, some things however are not so good. The amount of insider information that bit-tech aquires is great.The information they get always appears much more indepth then other sites and it seems as if the bit-tech crew have good relations with the companys whos products are reviewed here. The new site layout turned out great, I can actually find old articles now and even venture to pages other then the front page and forums. the pictures on bit-tech are always excellent, no product reviews with someones horribly lounge carpet in the background. as for what’s not so good, well I remember there not being near as many ad's. this of course was probably unavoidable if the crew were ever going to have any one work full time on the site. but this does not take away from the fact that there are a lot of items advertised on here that have never been reviewed by bit-tech, but by them being advertised on here we are under the illusion that they are a good buy.I also remember when something would not be posted on here as breaking news 2 weeks after the entire internet found out about it, not really sure what this is about guys but it seems to have been happening more and more recently. anyways after writing this im feeling a bit depressed that my hobby mite die off... im gonna go mod something to make me feel better
It is true that mods have become more sophisticated, more expensive, and that it therefore is harder to create something that stands out. However I think this is partly due to the attitude of the community. We only have ourselves to blame. Modest, beginner, low budget (or just plain out-there) mods that perhaps five years ago would have met with constructive comment and encouragement, are now being met with ridicule and put-downs. Meanwhile, those who laugh the loudest have not got a single mod in their name --it is all pre-modded towers with the same hardware, all looking much the same. And yet there is so much room for truly original mods. I have a design for an orignal self-built aluminium case in progress in SketchUp; I come across beautiful electronic components that have not yet been used in any mod practically every week, there are tons of original ideas that require no significant resources, but that I have not seen executed yet (perhaps I will... ). Pre-mods have made people lazy and unimaginative. Any modding site starts with the community that contributes to it. If you want to recapture the spirit of modding from the early days, let's not get sniffy about hand-painted cases and blowhole mods just because we bought some shiny Lian-Li cases.
I think you've missed my point here Nexxo, i was agreeing with you that differences in testing are what makes for that well rounded look at a particular product, obviously i will have to be a bit more specific though. Take the R600 review, since it's the most recent and the one which is closely matched with very few differences between [H] and BT it's most likely the best currently to use as a case study for this debate. The architecture and technology outlines are very similar so we can skip this, each presenting slightly different opinions on the roadmap and making some points about the R700, aswell as it's current position, all good points in both. Then we have the test setups, in the case of this particular product it all works out. We have [H] with one set of hardware and BT with another, whilst both are using their own test benches along with a group of high to mid-end cards from across the range. Then we have the tests themselves, again this is fine, both have slightly different amounts of information and some include information that the other does'nt, for instance power consumption benches and AA/AF comparisons, along with slightly differing testing with regards to duration and games. All of which like you say, adds up to a very rounded view of this particular product, all from reading two sites. Now let's wind the clock back a few months, very different story can be seen with regards to mid-range and high-end test setups. In order to make the same easy comparison between two sites now you have to factor in the mid range hardware that was primarily in use by BT at the time, versus [H]'s more balanced mid & high-end testing, and the favouring of Intel over AMD solutions by both sites, not such a big deal, apart from when you see the end numbers from some of the testing. To gauge that you then have to go find yet another review site. It's a trend which many seem happy with, and yes it's a good scientific method to have lots of different possible implementations of hardware from mid-range to high-end, but that's something that used to be seen far more on both sites than it is these days. Mid-range setups have now taken a back seat when it comes to testing, for some that makes reading reviews with the aim to upgrade a tricky prospect, and requires yet more digging, so I don't think it's unreasonable that people should'nt have to look through ten sites to gauge what will work as a good upgrade. I'm not an upgrader myself, I build new builds every 2-3 years from scratch and then OC it until it reaches the end of it's lifespan, but for others out there they come to rely on as few sites as possible, just like me, to give them the information they need. Time is a valuable thing, and checking through 10 sites to come up with a balanced decision is not something i ever remember having to do 2-3 years ago, simply because the number of test setups and varying levels of hardware being tested was for more extensive than it is now, of course this is true of nearly every site out there. But the job of a review and therefore a review site is to make it as easy as possible to come to that opinion. Not many people will spend £600 on cooling and bench it for two weeks before they even install it like I do for example, even less will spend another week benching the hardware after install and running 48 hours of CPUBurn to make sure there OC is truly stable, most people just want it work and have the best they can afford for their time and money. So yes Nexxo i agree with you completely, but only because i have that time to spend and because of the way I invest in my hardware, both for play and for work. Ask those who don't and I'm sure you will find they really don't want to spend their time to go through ten sites for an accurate picture of one piece of hardware, when each review is now catering to a smaller and smaller group.
Some of the best mods don't cost the earth. For example, Modsquad does amazing work, and its brilliant becuase its simple yet flamboyant, Also, we've had great mods where someone has fully built a custom water-cooling set-up from lumps of copper. We've got the water-eroctor mods going on ect. I'd say that for sure, original Ideas have been used, but whats wrong with taking a proven formula and changing it slightly to fit in with what you want?
See now that's why I will always value BT above all the review stuff that's being talked about here, old school ghetto modding . So many creative people behind the scenes that really the stuff that's being talked about here with reviews and how we all like to use hardware sites is 'almost' beside the point of why the sites here in the first place, and what it first grew around. Still love to see people on these forums asking those questions that have been asked a thousand times before, can get a tad annoying, but it's still great to see people starting out. My Baby It's not pretty i know, but at least it's original, well tested, and inspired by a lot of learning over the years, think that's what the modding bugs all about in the end, does'nt matter if it's performance or pretty in the end, it's modding . "im gonna go mod something to make me feel better" Amen to that!.
I'm sorry, but you are right that I am missing your point. It sounds like you think BT has declined in quality because it now does not follow the same testing methodology as [H] and therefore does not come up with the same results. This annoys you because now you have to read more sites to get some sense of the replicablity of [H]'s results, which happen to be what you are most interested in because their test conditions happen to most closely resemble the conditions of your PC. But many people visit BT because they are overclockers who wish to invest in the latest-statest hardware. So they want new hardware to be tested on high-end machines which resemble their high-end platform. Same difference. It all sounds to me that you are arguing that BT has declined because you feel that it does not readily deliver exactly what you want on a silver platter anymore. It seems kind of strange to me that someone who will quite happily spend three weeks testing his hardware somehow feels put off by the idea of reading more than two or three web reviews to decide on a product. Sorry if I don't accept it as a valid basis for a critique of BT.
