Until thought control becomes accurate enough, physical interfaces will remain necessary to interact with video games. Consequently, these products can and should be considered as tangible objects, or more accurately, products experienced physically. The first contact one has with a video game is in fact with the controller of the platform it is played on.
This past year, casual gaming has had a huge boost in consumption while alternative controllers such as the Wiimote and Rock Band game accessories have attracted numerous gamers, novice and seasoned alike.
Would it be too adventurous to state that these two events are linked, that these new ways of controlling games (read : new ways of experiencing a video game product) have brought some much needed fresh air in this industry ? You'll see for yourself, but I assume it's the case. Details on their way, followed by some projections on the role multi-touch interfaces could play in casual gaming.]]>
Not before too long ago, when a novice decided to jump in the water of video gaming, she agreed on a game content she was willing to play with : story, rules, goals and graphics, all these aspects help to get her into this new experience. Then, she confronted herself with the game controller.
For a person who's never hold one, this object can seem mysterious : a certain number of buttons with enigmatic significations are placed in a possibly meaningful way on a shape that seems to be adapted to double hand grabbing. For the most curious or motivated ones, this will look like a challenge, a new territory to explore and own. For others, it can be a straight turn off, an invitation to go elsewhere, rent a movie, surf the net, what have you. The controller acts like it's standing in their way to the game content. It says "you don't drive a video game, you PILOT a video game, my friend".
As much as they can lure a curious mind away, these console controllers can also act as limiters to more advanced gamers looking for a more intense experience of their games. They acknowledge this by purchasing more sophisticated controllers :
Different reasons make these objects more effective at the task of controlling a game :
1. better materials, better technology, better ergonomics, such as in the mouse case.
2. the use of known codes, previously learned conventions, allow the player to dive deeper in the game. The steering wheel or the joystick are such controllers, they mimic the ones used in real life situations simulated by the game.
3. by inducing a body posture the player would have in the real life situation simulated by the game, in the guitar or WiiZapper cases.
Combined or not, these principles help the player to focus more on the consequence of their gestures, the game content, rather than on the execution of the gestures. Ultimately, the aim of these controllers is to disappear, just as a music instrument vanishes from a musician's mind map, allowing him to be in his musical landscape. This conscience threshold is critical to the immersive experience of a game. You can have the biggest screen, the better sound system, or the most realistic graphics silicon chips can output, if you have to think of what button to press to initiate action A, you'll never reach the kind of game immersion level children get to when they play with, well, whatever seems appropriate to them.
Confronted to a game, the cycle of thoughts of the gamer goes like this :
1. a situation is presented on the screen.
2. this situation is analyzed in order to
3. make a decision to modify this situation
4. the translation of this decision is made through the control pattern of the game, resulting in
5. a gesture performed with measure in time and strength to modify the game situation.
6. and back to 1.
Hardcore gamers spend enough hours playing to make the controllers they use natural extensions of their body. They can master a game's control pattern very quickly, and it's easier for them to get into more complex control patterns because of their acquired skills. Steps 4 and 5 are not much of their concern, and they can focus on the game content.
Casual gamers, by definition, don't have the time to master and remember extensive control patterns. They have to make a effort to accomplish steps 4 and 5, altering their focus on the situation analysis and decision making. Therefore, casual games must be conceived with an easy control pattern in order to let users master it quickly and effortlessly.
The problem for game developers and editors is that they have a limited set of possibilities to level up the 4th and 5th steps experience quality. Unless you're Nintendo and you're able to produce game platforms yourself. When they introduced the Wiimote and the DS's touch screen, they went over the limitations inherent to classical button based controllers. In the Wiimote's case, the 4th step of the cycle just disappears : this controller allows the user to perform real life moves to control virtual content. The control patterns are based on known gestures which consequences in a given context have been acquired by just growing up. The transparency in the cycle described above resides in the fact it's easier to associate a gesture to a consequence than to associate a neutral button to this same consequence. The control pads buttons have to remain neutral in order to adapt themselves to the different games they will be used for. The player then has to make a semantic association between a given button and a consequence, an extra concentration effort that is not present in a gesture based control pattern, making it better suited to casual gamers.
This evidence is certainly a key point to the Wii system commercial success. An other aspect of the Wiimote that is helping a player to get a better grip on the game he's playing is the continuous link existing between the player's gestures and their consequences in the game. Similarly to the way a kite is controlled, every gesture is understood by the game system ; wether it's actually playing a role in the game situation is another story. The important thing is the feedback loop allowing the player to immerse himself in the game more efficiently. The classical control pad, in contrast, only understands button pressing or the manipulation of its joysticks (if it still can be called such) when it's the case. The link between the player's gestures and the manipulated virtual object is existing by intervals, these interruptions altering the player's immersion.
The Wiimote success lies in these two qualities : it's based on meaningful gestures and it provides a bridge between the player's movements and the game content. These two elements have, by the evidence they provide, considerably lowered the level of expertise needed to enjoy video games, thus allowed many potential players to entertain themselves in a new way.
The multi-touch interface shares these two principles with the Wiimote : the use of gestures and the possibility to provide a continuous cause to effect loop. Implementing it in gaming devices could bring some effortless gratifying experiences to gamers.
As from today, the most successful use of this technology in consumer electronics is Apple's iPhone and iPod Touch. The interface of these two products is based on the simulation of real life objects, such as buttons or paper sheets (for lists or pictures). The possibility to use gestures to navigate these interfaces renders the control pattern transparent to the user : you only have to decide and to act without thinking.
The short history of iPhone / iPod Touch and games is already a great success : the page listing webapps on Apple's website shows that nearly one third of the apps are games. In January 2008, they were 176 and today they are 329. This progression rate illustrates how much of a game magnet these devices are.
Now, that been said, one problem remains for these devices : the size ratio between your fingers and their screens. In order to control the interface, you have eat some screen real-estate. It's less of a problem with games like puzzles where you don't have to interact continuously. But if you need to make a permanent contact with the screen to play, then you're somehow lowering the visual experience quality of the game. The screen size is going against one of the two principles that provide a good gaming experience, the ability to establish this bridge between the user's gestures and the game content.
This constraint can generate some nice workarounds, but in the end it can be perceived as a limitation.
✤
There might be a different scenario though : what if the game is not displayed on the iPhone / iPod Touch screen ? These machines may work as independent mobile devices, but they are part of a bigger whole : you need a personal computer to use them. Which has a screen, a bigger one. Having the iPhone / iPod acting as a controller for a game screened on the computer could offer a better gaming experience, both visually and control-wise. Many examples of iPhones being used as remote controls for the mac have surfaced recently, so it doesn't seem far fetched to apply it to games.
The downfalls of this situation are that it's no more mobile gaming we're talking about, and that you may not be able to reproduce the same array of simultaneous controls 14 buttons control pads provide. For the first one, I guess it's not that bad since most of gaming is sedentary, and that many games are and will be available for the mobile context. The second one has no turn around, but as said earlier casual gaming requires simple controls, and button combos have little say here. If you like to play video games as if you were playing vibraphone with six mallets, you still have great opportunities around. But for the average player, two simultaneous controls will be enough.
How would that work ? Let's illustrate this with an good old fave, Tetris.
The first piece appears.
Tap the screen to 'grab' the piece.
Slide your finger sideways to position it.
Rotate your finger to rotate the piece.
Slide your finger down to make the piece fall.
All these actions were done without lifting the finger from the iPhone / iPod Touch's screen surface, and the first tap doesn't need to be located precisely (although the center would be more appropriate to let you slide comfortably).
Even if there's still the option to consider the location of a tap to launch an action in the game, the advantage of this controller is the recognition of gestures. In more complex games, a circle gesture could be associated with action A, a V gesture with action B, a three fingers slide on the right with action C, and so on...
Other parameters such as the sliding speed, the slide celerity, its angle, the pressure duration,... can be used to either fine tune the game controls and/or increase their quantity.
This leaves room to build a simple set of controlling gestures that can evolve into a more complex one without having to interfere with the hardware : it makes a great adaptive controller that can match different levels of gaming expertise. Either within a single game, or to match different gaming styles.
These devices have also other features that could be used to enrich the way we play games : they connect to the internet so gaming data could be easily downloaded or shared. They are screens so gaming content could be displayed on these and allow the gameplay to be dispatched on two screens (let's think of Sam Fischer's opsat in Splinter Cell). They have storage, so they could act as memory cards, giving the opportunity to play the same game session on different computers. They manage sound pretty well* so they could be a secondary source of audio content during a game. I'm no coder myself so I can't provide you a working example, but I believe this is a good opportunity to build some great gaming experiences.
Still, just as with chocolate cake, it's not because you have a great recipe that you'll be able to cook a great one : you need to be a good cook too. At the beginning I was talking about how the control pads were in fact the tangible skin of video games. In this case, gestures will play this role for the player, they will be part of the product definition. Their choice will be critical, and getting a coherent gestures set that will act as a vocabulary will be essential to get a great product in the end.
Gaming and Apple seem to have had a rather difficult relationship, from what I've read so far. A lot of people in this industry seem to believe Apple is not interested into getting involved and making efforts to meet their product development specs. I don't buy it. Apple is more and more an entertainment company, and I can't see them ignore the revenues they could get from the fast growing casual gaming market. They proved they cared yesterday at the iPhone SDK presentation, showing 3 games in total. Wait a few updates to the Apple TV, and I'm pretty sure we'll see some gaming there as well. Will it use Touch products as controllers ? We'll see.
Beyond the iPhone / iPod touch scenario, I think multi-touch technology can bring a lot to the gaming industry. Looking at the classic control pads now, I only see the whack-a-mole level of gaming interactions, and it's not going anywhere for me. Gestures have always been close to games, mastering them is part of the pleasure we get from playing. This technology provides us with a better way to get input from them, and this is why it looks promising.
]]>
