Eidos Metrics Suite: Playing Underworld might make Origins better

[nomedo]

Today i read something very interesting. Something that i didnt knew Eidos did and this should be very interesting to all of us here on the forum.

Playing Tomb Raider Underworld might actually make "Tomb Raider Origins" better!

How? The answer is appearently called Eidos Metrics Suite, A program monitoring how people play Tomb Raider Underowld. This information comes from a Research Paper at IT University of Copenhagen on "Player Modeling using Self-Organization in Tomb Raider: Underworld" by Anders Drachen, Alessandro Canossa and Georgios N. Yannakakis. Link to paper

here are some interesting portions of the report:

On what Eidos Metircs Suite is:

The suite is an instrumentation system which is designed to record game metrics from EIDOS games in production and post-launch.

Data from the TRU game were extracted from the SQL server system and preprocessed. The dataset used for the experiments reported in this paper contains live data, i.e. data from people playing the finished, published version of TRU in their natural habitats.

The data collection process is completely unobtrusive since data was gathered directly from the game engines of subjects playing TRU in their natural habitat (via the Xbox Live! web service) rather than in a laboratory setup.

On how much data recorded:

The dataset used was collected during November 2008, and includes entries from 25240 players. Note that at the moment of writing there are over 1 million recorded gameplay sessions of TRU which will form the basis for future research.

On what the data is for:

A variety of different playing characteristics (game metrics) are collected for internal analysis work carried out by Crystal Dynamics. Several gameplay features are logged for each player, such as the completion time of each game level, the number of times the player died, as well as the 3D coordinates of the player.

On what the researchers focused on:

Causes of death: TRU features a variety of ways in which players can die, which can be grouped into three overall categories that encompass all possible ways players can die.

-Opponent; the percent of total number of deaths caused by any computer-controlled opponent existent in the game over the total number of deaths. (28.9%)

-Environment; the percent of total number of deaths caused by the environment over the total number of deaths. (13.7%)

-Falling; the percent of total number of deaths caused by a failed jump over the total number of deaths. (57.2%)

Total number of deaths: The total number of deaths of each player, D. A total of 190936 deaths were recorded, giving a mean value of approximately 140 per player, with the best and worst player dying 16 and 458 times, respectively.

Completion time: The time (in minutes) required for
each player to compete the game. The average completion time of the game in the sampled data is 550.8 minutes...
...varies from 2 hours and 51 minutes to 28 hours and 58 minutes.

Help-on-Demand: The number of times help was requested. There are more than 200 registered puzzles in the game, which the players have to solve in order to progress through the game narrative...
...Value ranges from 0 to 148,with an average of 29.4 per player.

They researches then go on to describe how they analysed the data with ESOM, but thats way over my head so ill skip to th interesting parts

On the results:

The highestperforming toroid-shaped ESOM trained on the data reveals four clusters of playing behavior — labeled as Veterans,
Solvers, Pacifists and Runners — which are characterized by specific patterns of play, after the analysis of the Umatrix and its corresponding component planes. Importantly, the existence of four clusters of behavior, even in a fairly linear and restricted game like TRU, shows that players utilize the affordance space and flexibility offered by the design of the game, rather than simply using one specific strategy to get through the game.

For example, the Pacifists are experts in terms of navigation and move rapidly through the virtual environment, but also respond badly to threats that
are moveable or unexpected; whereas Solvers are excellent at solving puzzles, respond readily to moveable threats but die often from falling and are slow to complete the game.

On the 4 types of players:

Veterans as they are the most well performing group of players despite the high number of environment-related deaths. Likewise,
cluster number 2 corresponds to players that die quite often mainly due to falling; it takes them quite a long time to complete the game; and they do not appear to ask for puzzle hints or answers.
Solvers, because they are adept at solving the puzzles of TRU. Their long completion times, low number of deaths by enemies or environment effects indicate a slow-moving, careful style of play with the number one cause of death being falling (jumping).
Pacifists as they die primarily from active opponents. The total number of their deaths varies a lot but their completion times are below average and their help requests are minimal indicating a certain amount of skill at playing the game.
Runners, is characterized by players that die quite often and mainly by opponents and the environment. These players are very fast in completing the game (similar to the Veterans), while having a varying number of help requests which cover the majority of the H value range.

On what this knowledge will bring to the future games:

The player types identified by the ESOM can then assist a machine learner trained on sequences of player actions over the various levels of the game using hidden Markov models or recurrent artificial neural networks. This work will be carried out on the current Tomb Raider game and any future installations in the series, in collaboration with Crystal Dynamics/EIDOS.

This is plausible since the proposed approach provides the opportunity to evaluate in detail if a game design works as intended, by recognizing the patterns in how the game is played, and comparing this with the intentions of the design. Optimization of game design features — i.e. making sure that no feature of the game is under-used or misused — and phenomenological
debugging — i.e. debugging of playing experience and game balancing — can also benefit from player behavior modeling.

Furthermore, information about the different player types can be used during play to dynamically alter in-game controllable parameters (e.g. help on demand accessibility, difficulty of jumps) to adjust to the needs and skills of the player type identified in real-time and ensure variation in gameplay.

in other words:
Anders Drachen, Alessandro Canossa and Georgios N. Yannakakis may just have made us the biggest favour known to fans by analysing this data and working with Eidos and Crystal Dynamics.

So if you are a Xbox 360 player, jump in to Underworld some more and think about what type of player you are, how you play might affect the way Tomb Raider Origins will be played

interesting read, isnt it?

[miss.haggard]

Hmm, quite interesting. Too bad I sped through TRU because I wanted to get Laras crying over with

[tlr online]

Very interesting. Thanks nomedo.

[Sir Croft]

That's really cool and smart! Too bad it's only for the Xbox 360, though, I'd love to be part of that research.

[tombraiderluka]

SO I guess we should take time and relax while playing this

[ThatNorskChick]

That's very interesting, thanks Nomedo . Does this mean they're keeping track of how many times I purposely throw Lara off cliffs for amusement? Will that skew the data?

I'll have to chill out on my next play through.

[nomedo]

btw on Georgios Yannakakis,Associate Professor at the IT University of Copenhagen, page this is a proceeding for SIG in Milan in September 2009

A. Drachen, A. Canossa, and G. N. Yannakakis, "Player Modeling using Self-Organization in Tomb Raider: Underworld'' in Proceedings of the IEEE Symposium on Computational Intelligence and Games, Milan, September, 2009

and in the paper its stated that Crystal Dynamics had this integrated for the release of TRU. And by the time this paper was written, over 1 million sessions of TRU had been played on Xbox Live. thats great news also.

i had no idea they had this Eidos Metrics Suite integrated in the game and that they are actually researching for "dynamically alter in-game controllable parameters". that could be very interesting to see in the next game if they can make it work.

i just couldnt stop smiling when i read the paper.

[takamotosan]

Let's hope they use this to our advantage

[KyleCroft]

Everyone speed through it as fast as you can so TRO is longer

[matrix54]

Big Brother is watching you

at least this time its benificial