Next-Gen in Motion: Rare, Kinect & Xbox One

"It's like the Mario 64 demo, but with your own face".

Bridging the gap, a similar comment was heard at Microsoft's offices in Madrid, when the company and Rare live-demoed the latest progress made with Kinect (the camera bundled with every Xbox One console). It was a surprising double feature: first up, the public one, focused on Kinect Sports Rivals when it comes to producing caricature players or "champions" of the respective sports. Then, much more amazingly, the private one, where we discovered what the sensor is capable of nowadays "behind the scenes".

But that doesn't mean that Kinect Sports Rivals is a bad start for the tech on Xbox One. In fact, it has new features when compared to the original, giving us a very clear indication of where this is heading: greater precision and response when played in a variety of conditions and configurations. We hope that, after secretly witnessing what Kinect is actually capable of doing, applications grow much more complex and innovative in the future, but this first example is very easy to appreciate.

With the old hardware and software, you had to perform exaggerated body gestures, there were some errors and delays in detection, and environmental conditions (lighting, distances) had to be optimal. In most of the six Kinect Sports Rivals events, subtle movements are now read and almost immediately reflected on the screen. For example, it can follow your hands in real time when climbing, your fingers when shooting or if your body tilts mere inches when controlling the jet skis.

Videos that illustrate this, and the video interview you can watch on page 2 of this article, will give you a better idea of the six sports (three returning, three new) included in Kinect Sports Rivals, showing off the changes introduced by the so-called "Kinect Version 2" and other more unrelated new features, such as the new environments, modifiers, cloud features and unlocks.

The biggest surprises came with the creation of characters by "reading" real people, and from the subsequent private technical demonstration. Nick Burton, chief of new technology at Rare, showed off the studio's internal working tool, with a NUI (Natural User Interface) View that keeps reflecting what Kinect is seeing in a variety of formats. With different viewing modes, Burton proved that it is true that technology now depends much less on the lighting conditions thanks to the much-trumpeted photons-bouncing system, and made it clear that there is no need for as much distance to the sensor, and that the virtually recreated models are much more consistent. The technology's ability to pick up colours, sounds and depth also convinced. But what really left the audience amazed was when Burton performed various gestures and movements to show off about the ease which with the system renders and animates a skeleton "with six new joints", follows the movements of the fingers and how it scans a skull and instantly recreates something with great fidelity.

The polygonal facial model resulting from this private demo, and from the "point cloud" which is taken from measurements made by Kinect to create a detailed mesh, is way more detailed than the stylised version Kinect Sports Rivals produced afterwards, hence our surprise. The implementation of this tech in this game is fun, and it works fine, but what's there beneath is an impressive capture of your body that promises much more for the future.

How to create a character in Kinect Sports Rivals by scanning a real player

The following GRTV video combines another demonstration, the public one with Rare's Harry Robinson creating a champion in Kinect Sports Rivals (for the first four minutes), with the interview with the pair of Rare developers (in the last 20 minutes):

"This is the experience that we've created surrounding the champion creation. We wanted to make something that felt magical to the player. The notion of scanning someone in was a quite cool feature (...). What we're telling the players is that these cubes are pieces of 'digital DNA' that are going to be used to construct them as a champion in the world of Kinect Sports Rivals. Initially I'm given a basic outline of the character (...) Kinect scans me to see how tall I am, how wide I am, and I choose if I want to be an adult or a junior, and gender. [Harry moves and fools around while the sensor registers it with its cubic character] (...) The magic happens with the face. Kinect will look at my face and will identify the colour of my skin, my hair, hairstyle, if I wear glasses or beard, and finally it will look at my completely unique face shape and use that to create a stylized version of me. No other game or experience have done this before, scanning my face in 3D and use that to create a character".

Excerpts of the interview:

On possible future applications and games controlled only by the face

Burton: "If you look at the depth map, we can get the exact facial gestures... we could even make a game controlled only with your tongue (laughs )"

Robinson: "I'm not convinced with doing a game controlled only with your face, but I can imagine all sorts of experiences that utilize your face in unique and cool ways. In a game like Resident Evil all zombies could be based on your friends!

Burton: "It might not even go into games. What does that mean for example for hands-free devices, of for disabled?"

Has the motion controlled games fever passed?

Burton: "It's not going away. There was a 'wow, motion gaming is amazing' big push, but if you look back, to when we went from joysticks to joypad in the 80s, that kind of happened when the NES came out and everyone went like 'amazing', and then everybody else did it. That's what's probably happening here, because people are not doing what they did with Kinect Version 1 and running in the spot until they sweat. People will just forget about if it's motion gaming or controller gaming, and just use what's most appropriate for the experience (...). Had an argument a few weeks ago, someone told me those [Xbox One] joypads had accelerometers so they could read this [Dead Rising 3 shaking], and I was like 'no, they didn't, that's Kinect'. It doesn't matter, it's just cool to do it.

On scanning a champion...

Robinson: "It's really strange, when you scan people, some people is uncanny, you can look at that character and it really looks like that. And then there are others and it's not uncanny. The good thing is everyone is unique, everyone is different. You don't get someone that looks completely nothing like themselves. For me, that's one of the most important things that we've done. I mean, even when we don't nail it 100%, it's still a good enough approximation, and when we nail it, it's just the magic that blows people away."

Burton: "But I think there's something else. With the new level of fidelity we get from Kinect we've been able to bring a new depth to motion gaming. If we can run a sweepstakes in the office of who can do the fastest lap in one of the wave races, and still watch the times getting nicked away like if you were playing a normal racer like Forza Motorsport, but with motion gaming, that's that new level".

Kinect Sports Rivals teams, currency and DLC

Robinson: "Initially, the teams are kind of fun characters that introduce you to the world of KSR and teach you how to play each game, then eventually they pitch you against some of the hard events, and at the end of that process you pick a team to join. Moving forwards, the teams are used to drive competitions".

"The currency is just coins [not real money]. If you can win you earn coins that allow you to buy things from the in-game store. The better you are and the more difficult opponents you play against, the more coins you're going to win."

"[Could we get brand new sports via DLC?] It could be... (Laughs) we're not announcing what it's going to be yet. It could be something on those lines and it could be something else. A lot of what we're going to do in the future is going to be based on what we see players doing in our game".

What does Rare think about a cheaper, Kinect-less Xbox One?

Burton: "I think those people [asking for it] are going to be surprised of what we can do with Kinect. To be taken out and not having available for everyone then limits what you can do, and so, on a personal level, I think it would be a shame to do that."

Robinson: "I agree with Nick, I think it'd be a shame because I think if it's there, you've got the guarantee that every user's going to have it. I totally understand why people want it cheaper, but I'd personally keep it".

Burton: "The other thing is, of course, over the lifetime, every piece of technology reduces the price. That's not me saying there's going to be a price reduction".

With Nintendo apparently content with its compelling but somewhat left behind MotionPlus-IR combination, and Sony focused on a Virtual Reality system, Microsoft seems more determined than ever to make progress in this area with Xbox One. Having witnessed the potential of the latest version of Kinect, as always, it's simply a matter of games and applications having to convince their audience at the end of the day.