Game Career Guide is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Get the latest Education e-news
  • How To Measure Fun For Game Designers

    - Roman Szomolai

  • Thought experiment/model

    Sometimes you don't have to run the whole game to see how it works. Sometimes it would actually be impractical because you only need to know a portion of the game (like economy). Sometimes you need to cut through complexity or you just don't have another 10k players to play game 2 two hours a day. Then you are going for models. Don't forget that map is not territory; by creating a model you are omitting some parts of the whole system. .

    • Minimal variant: describe all system parts and relationship and then start imagining how it would work together, Simply ask "what if" question; small mathematical model #EveryDayIsSpreadsheetsDay.

    • Advanced variant: Stochastic machine learning mathematical model predicting most possible outcomes based on system set up. Ask your soon-to-be-very rich friend about this.

    Domain analysis

    In domain analysis you will go through competitors games in certain domains and look for commonalities and specifics. Domain can be anything from approach to 3rd person camera to genre. At the start of any domain analysis there are questions that you like to answer, things that you want to focus on. For example if you would do domain analysis on battle Royale you may find out that all of them have shrinking level mechanisms in one way or another.

    • Minimal variant: play some competitors' games and look how they do stuff. Look at videos of playthroughs. Write down how they handle specific cases.

    • Advanced variant: Make in-depth analysis on specific mechanics including data mining. Frame per frame description of actions. Look into interviews with creators. Plate all the games in genre and list all of the parts and relationships in them.

    Quantitative analysis

    You are measuring numbers and then using statistical analysis to get new knowledge. In this approach, more is usually more. This is the world of KPI and the F2P market and there is a lot of great material on this even on Gamasutra. No need for me to go into detail. 

    • Minimal variant: play game and note every time you die in specific level; Performing simple student tests on your marks per death in level; put simple scale (1 - 5) in feedback form and then look at average. 

    • Advanced variant: full blown analytics, buying data from big brothers. (I am leaving legality and morality out of it for now). There are plenty of companies who make their living just by this just for games. Don't be afraid of them.


    You can actually ask for people's opinion. There are a bunch of problems here that are connected to all qualitative analyses. Question form, tone of your voice, even time since playthrough can change outcome. Neverthless, you may find out very specific new information about your game.   

    • Minimal variant: Let somebody play your game and ask them "what do you think about it?", just let them talk. Don't comment it, don't defend your game, just ask and let them talk.

    • Advanced variant: Have full on randomized research with a preset of meticulously chosen questions and trained interviewers. 

    Take out

    1. You don't need to know what x, just how it manifests in the world

    2. Don't be afraid of measurement, it is simple, just follow the major steps.

      • Define what is your decision

      • Build your theory

      • Define your best possible hypothesis 

      • Define what are the best possible proxies

      • Measure it! 

      • Improve your theory, adjust hypothesis

      • Repeat b) - f) until you have good enough information to make a decision.

    3. Beware the noise! Corroborate and try to refute your hypothesis. Combining different types of measurements and repeated measurements will help a lot.

    4. Low quality measurement is better than non:

      • We don't need to know what something is, we just have to know more about it than before. 

      • We don't need perfect measurement, just better than before. 

      • We don't need perfect theory, just better than before.

    5. Measure with why (decision) in mind to prevent wasting resources.

    Do you want to know more?

    • Hubbard, Douglas W., How to measure anything: Finding the value of intangible in Business. 3rd edition. Wiley: 2014. (Main inspiration for this blog and major inspiration in my work)

    • Popper, Karl: The logic of Scientific Discovery. Routledge: 2002 (oh yeah, I am going there. Only for intellectually brave)

    • Kuhn, Thomas. The structure of scientific revolution. University of Chicago Press : 2012. (scientists are people too)

    • Whelan, charles. Naked statistics: Stripping the Dread from the Data. W. W. Norton Company: 2014.

    • Seidman, I. Interviewing as qualitative research: A guide for researchers in education and the social sciences. Teachers College Press: 2006.

    • Thought experiments,

    • Woodward, Matt. Balancing the Economy for Albion Online. (GDC talk on albion online economy, great primer into economy and how to model it.)

    • Ruskin, Elan. Three Statistical Tests Every Game Developer Should Know. (quick primer into statistics for game dev)

    • Collins, Steven. A/B Testing for Game Design Iteration: A Bayesian Approach (You probably didn't learn anything new from blog above, and that is ok. You still can leave comments about hating frequentism or smtg and then watch this video.)

    • Silver, Nate. Signal and the Noise: Why So Many Predictions Fail - but Some Don't.  Penguin group: 2012.

    • Sasassovici, Alex  and Miravete, Beatriz. How to Use Machine Learning, Live Telemetry Analysis, and Computer Vision to Manage Communities. (oh yes, you can improve every facet of the game by measurement, even community management)

    • ...and many many more! go and explore, don't forget to measure your progress ;-)


comments powered by Disqus