Columns Game Phone Home!

Maybe you *should* care about the Steam Hardware Survey. Not because it has any specific information on your players (it probably doesn't), but because Steam's data collection processes are far and away the best in the industry, and other game developers could learn a lot about player information gathering from Valve.

It's not just the Hardware Survey, either. It's percentage of players with Achievement 24 unlocked, time required to complete Level 4, number of times died on Level 7, all broken down by difficulty levels, etc. - exactly the depth of playtesting data you're looking for. And, of course, Valve is rather infamous for their exhaustive playtesting procedures. Play through Episode One or Episode Two or Portal and keep the commentary mode on. Listen to how many things they added in, changed, or eliminated because of direct player opinion, difficulties, emotional responses. The fact is that (funding to pay playtesters aside - though I second Xir's comments there) what you're wishing for is already reality.

The mistake here is that everybody confuses data collection with DRM. This is a confusion only worsened here by the allusion to BioShock, I might add - a game which, to my knowledge, does not collect anything other than number of times activated (nor does it do so very well). The recent Mass Effect announcement (phoning home every ten days), even despite the later retraction, doesn't do anything to help clarify the distinction, either. You have to admit, though, after having so many of our (legal) games being mugged and held hostage by SecuROM and the like, people have come to see almost any form of 'phoning home' to be an invasion of privacy. And in most cases we've been proven quite right.

The missing piece here is trust. Gamers trust Valve. Why? Because Steam keeps our games working, across computers, automatically updated. There is one activation and then the game is tied to your account - and as long as you have the account, you will *always* have your game. Upgrading, formatting, and the like don't faze it. There is personalized recourse for misplaced mistrust on their part. And (in gamer years) it's been this way for a very, very long time. Steam has earned our trust.

Obviously, given that prerequisite, it would be extraordinarily difficult for any other developer to throw in a 'data collection' feature and expect us all to bend over and put up with it. But I think as long as you're honest (and up-front!) about your intentions, you can probably get people to give you what you seek.

 

As I've said before, the biggest thing that burns me up is that you have to connect to the internet to authenticate a game that can be played as "Single player". Perhaps I want to keep one of my laptops free of virus/malware and such, so I do not connect it to the internet. It's a good laptop and can play games, but now I can't play this game because I have to authenticate online. If they want to get that extreme with validation, give me a 1-800 number to call to validate the game, don't force me to connect online. I've got two laptops that haven't been connected online in about a year or more. I strictly use them to play games when I'm travelling or to work on documents which I can move via USB flash drive to whatever PC I need. BTW, I've yet to download a game patch that couldn't be loaded from this PC to one of the laptops via a 4Gig flash drive.

Piracy is bad, DRM is even worse. I have no problems against voluntary data collection, but don't require me to get online to authenticate the game or to periodically reauthenticate it.

 

Greetings!

Quoted from the article:

Check Valves hardware survey.

Maybe you should care.

Both.

Ever checked the Achievements menu in HL2 games?

Set up a beta test with all the above and find volunteers.

Developers and publishers are losing not only the trust but also the respect of paying customers with the DRM implementations.
A game asking, or sneaking behind me, to send data home gets blocked.

 

Valve's hardware survey is not exactly representative.

Why?

How can you have both XP and Vista-only? Bear in mind that even if you could this would double development times, drive up prices and piss of loads of Mac and Linux users etc, as well as causing consolidation and problems when MS updates next.

I have - but the achievements are there for about two or three games IIRC from a single developer / publisher in just the hardcore gaming market and are easily skewed by gamers who want to achievement whore it. Ever seen the servers when a new TF2 achievement pack goes out?

That's called testing. It's a good point and something people should do more of - but finding volunteers to go through a proper testing schedule is next to impossible. Testing is a job and people want to be paid for it. That alone puts it out of the realms of smaller, indie developers.

That is your perogative - but you'll just end up getting left behind as the market changes, I think. I understand your point of view completely, but I think as long as phone home device is transparent to users, non-invasive and stable then it'll be fine. Especially in the ways that Cliff describes it.

Also, you do realise that Steam phones home, right? All games on Steam have to activate online after a set period despite offline mode. Valve don't talk about it or let you know about it, but it IS there and they DO do it. Steam actually takes an awful lot of information from you without you knowing, but that info is handled responsibly and stably. It effects everything from knowing what games you have so it can update them (which scans your account and drives) and display games on your front page, to tracking which Steam updates you've recieved (so it can show you updates of the new releases on Steam), to asking you if you want to be part of the hardware survey (without telling you in detail what it will be scanning).

I love Steam and I love Valve - but they really shouldn't be referred to as the posterboy of the PC community. It isn't all true and it isn't good for them.

 

I think someone already mentioned the idea of setting up a LAN festival or something to do this. It'd be a good way of doing it that wouldn't be outside of the realms of a small developer, especially if they could get event sponsorship from some of the bigger hardware manufacturers/publishers out there - who knows, they might even pick up a publishing deal out of it (although that is being rather optimistic)... offer a rebate on the cost of the LAN fest entry for those who want to act as guinea pigs.

 

A LAN festival test isn't what's required though, at all. There's a HUGE difference between somebody actually TESTING a game and getting a bunch of people to just play a game and *maybe* stumbling across some texture glitches.

 

Uh... are we testing bugtesting/Quality Assessment, or are we talking balance tweaking?

I thought the whole point of what was being said in the article was that it wasn't so much going to be bugtesting the game, as balance testing the game - whether a certain weapon dominates, or a power is used (or not, whichever) and whether costs, tactics, AI routines etc have to be tweaked? Seeing how gamers play the game, and adjusting the game accordingly.

If we head into the 'bugtesting' realm... then a similar routine to Microsoft is sensible - "Oh, your game crashed or there were texture glitches - do you want to send blah-blah an error report? Y/N?"

 

Greetings!

Not representative of gamers systems?!
What game genre isnt represented in Steams catalogue?

Because he wants to know what hardware gamers have.

He shouldnt have XP or Vista-only code, he should code for both OSes.

Make achievements more commom in games and make some of those hidden. Am curious how Director will work in L4D.

Will have to settle with a limited beta release and collect data via dedicated forum, then.

Market is always changing, if it goes to places where I dont want to go, it wont get my money. So far, only Bioware lost money with me, in September Maxis will lose money too.

Am aware of that. As Cliff Harris wrote, trust plays a part. I trust Valve and the Steam platform, I am not going to trust every developer/publisher out there, though. Not with draconian DRM infection spreading.

 

To be honest, there are so many flaws with what you say that it's hard to tackle them head on. Everything you say is plausible in an ideal world, but realisitically is completely impractical for a indie game dev. If you really think it's that easy then I'd love to see you put your money where you mouth is. There is a huge difference between people who play an open beta and people who properly test a game.

http://www.calendarlive.com/music/cl-wk-tester24nov24,0,169991.story

 

The idea that game developers should get game testers working for free is ridiculous, they are a business that is there to make money and they make plenty of it (profit after all costs of manufacture, distribution advertising etc), more money than any of us will ever see in our whole life.

They won't give out free copies of their games so why should they get free labour from game testers?

I did some game testing for THQ in Frontlines: Fuel of War, in the open beta test.
Despite them having a competition to name servers and them using several of the names I entered (they were original names not entered by anyone else) I never got the copy of Frontlines that was the prize for them using any of the names, never even heard from them about it and when I tried pulling them up on it on their forums (after e-mailing them several times) the thread I started was deleted quickly, they had also moved the competition thread to a part of the forums where I could not access so I could not get the proof that I had entered the names.

I put this expectation of game testers to work for free in the same boat as big companies that expect the government to pay for them to upgrade facilities, expand operations or start new operations when the company can easily afford it from their profits, how dare a company that makes for example $X hundred million in profits expect the government to cough up $X million to help them make more money when they could easily use some of that $X hundred million profit, I know they usually have shareholders to look after but part of running a business is re-investing some of the profits to expand and improve the business, it is just greed to expect handouts.

The argument that the government will get a return from increased tax or the benefit to the community makes it ok is ridiculous, the company can afford it and should pay for it themselves, the money they expect from the government should be going to what governments were set up for in the first place, funding healthcare, education and infrastructure from our tax dollars.

If they can't keep the business going on their own then they have no place being in business, if they can't accept a little less profit now in order to increase profit in the future then boo smegging hoo


As for data collection it must be completely optional and transparent otherwise they can smeg off!

 

Greetings!

Impratical as in "not able to do" or as in "not easy to do"?

 

I'll reiterate what someone else has said--DRM and collecting data don't have to have anything to do with each other. The inconvenience/creepiness/danger of DRM shouldn't interfere with collecting data.

As for the desire to collect data...umm...it's been going on for at least a decade. Microsoft has been collecting data in both automated fashion (logging & transmitting data to servers) and in-person testing like you describe (it's actually called user-research, not testing).

References:
* automated logging: see Halo 2 & 3's webpage--massive amount of stats about gameplay, available to individual players.
* user-research: finding people to test games is actually quite easy, and you don't have to set up a LAN tournament (in fact, I advise against that). See the Wired article about the user-research Microsoft did for Halo 3 (600 unique users played, for a total of 3000 hours--and that doesn't include the beta). http://www.wired.com/gaming/virtualworlds/magazine/15-09/ff_halo

The reason that I'm only citing Halo stuff is because they talk about it publicly. But in the past few years, Valve, Infinity Ward, and Bungie have all given presentations where they cite how critical playtesting/user-research is to polishing a game.

And, as for the costs and practicality of doing user-research as an indie game developer, the costs are quite reasonable. (Automated data logging systems are very time & resource intensive right now, and probably beyond the reach of indie developers.)

To learn more in general, just put in 'games user research' into Google. Eventually, you'll even come across my consulting company by the 2nd page.

 

I just has to register to say, C. Harris, what a crock of s....

You say you want to collect this information from your customers, well have you even bothered to try asking them. Have you tried to distribute a system information application with your games that has the option to "phone home" after asking if it may, or even better let it create a plain text file that could be sent back so people know what's being sent, now what's wrong with that? If you have tried that and people aren't letting it phone home maybe that's the message you should be getting, it's none of your business unless the customer decides to make it yours. Trying to justify this feces by saying you just want to know their computer setup is horse manure.

As for DRM, the only thing it manages to accomplish is pissing off legitimate customers. I know people who have had to turn to piracy just to use something they've already purchased, Bioshock for example since it happens to be the picture used, You think it stops piracy? You think it even delays it? If you want to fight piracy lower the damn prices, the cost of games is totally ridiculous.

I will never purchase anything that has to phone home to work, let alone something that can be deactivated from afar. Do you think that's stopped anyone from using any such product?

"software that traps us could also be used to free us."
Oh wow, really.

 

wow so much hate from that last post. I'm just trying to make better games, and automate the process of asking gamers what they want and enjoy
Don't worry, it's clear you would prefer game developers to ignore what YOU want and like. That's fine with me. Nice rant about DRM too, not vaguely the topic of the column, but hey, don't let that get in your way right?

 

Greetings!

Its clear FCHarris want developers to know what he doesnt want, DRM infestation.

 

sorry guysi have no idea about the politics but isnt that what game testing and, as mentioned before, beta versions before full released versions go out???