You Beat Your High Score, But Are You Getting Better?

High scores are the pride and joy of many a video game enthusiast.  After all, who doesn't want to be the best at something, if only in your own particular clique?

But how can you be sure that you really are the best, or even better than you were when you started playing?  Even in pure games of chance, your high score will gradually improve the longer you play, simply because you've played more games and had more opportunities to luck into a high score.  It would be nice to know if there is any simple way to distinguish random improvements in your high score from improvements in skill, so as a starting point, let's look at a pure game of chance.

The Coin Game

Imagine a very simple game where the goal is to flip a coin so that it comes up heads as many times in a row as possible.  Your score is the number of times you were able to flip heads before first getting a tails, so a score of zero means that it came up tails on your first flip, a score of one means that you flipped heads once before it came up tails, and so on.  This is similar to a video game with sequential levels, where beating a level (flipping heads) gives you an opportunity to beat the next one, while dying (flipping tails) means that your game is over.

Question:  If I play the coin game 1000 times, how many times will I beat my previous high score?

Who would turn down an opportunity to play 1000 coin games?  Well, actually, rather than use an actual coin and risk injuring my flipping thumb (and boring myself stupid), I'll simulate a coin flip with a random number generator that returns either 0 or 1, where 0 is tails and 1 is heads.  Here is what I get if I simulate the coin game 1000 times:



The first game, marked with a blue star in the lower left corner of the plot, is always going to be a high score because there were no previous scores to beat.  In this case, it came up heads, giving a score of one.  The next blue star indicates when I first flipped more than one heads, and it turned out to be a pretty good run where I scored 5 points.  All subsequent high scores were only modest improvements, ending at around game 800, where I achieved my sixth and final high score of 9 points.  The odds of flipping nine straight heads with a fair coin is 1/2⁹, or 512-to-1 odds, so it's not surprising that I did this once in 1000 games.

Is this a typical outcome?  Yes, fairly typical.  When I run the coin game simulator many times, I find that I achieve exactly six high scores about 25% of the time, and six or more high scores about 50% of the time.

Beating Randomness

As it turns out, the result I found with the coin game is similar to what you will get in any game where randomness dominates the outcome^*.  In a 2016 paper, Aaron Isaksen and Andy Nealen at NYU found that the probability of achieving a high score in a game with purely randomized outcomes is 1/N, where N is the number of games played.  This means that, on average, the odds of achieving a high score in the tenth game are 10:1, in the hundredth game they are 100:1, in the thousandth game 1000:1, and so on.  I tried simulating random scores from various other simple scenarios, like rolling 50 dice or drawing from a normal distribution, and sure enough they all resulted in about the same frequency of high scores.

Of course, most video games are not purely random.  If I'm getting better at a game, presumably I will achieve high scores more frequently than this baseline result, but how frequently do I need to achieve high scores in order to be confident that I'm getting better?  In the following table, I try to answer that question by giving the number of high scores needed to beat 95% of simulations of a game of chance. 

Number of High Scores Needed to Be Confident You're Improving

Games Played	High Scores
10	6
25	7
50	8
100	9
250	10
500	11
1000	12

This means that if I play a video game 100 times and achieve 9 or more high scores, the outcomes probably aren't random and it's very likely that I'm improving.  The opposite result, where you get fewer high scores than given in the table, is less conclusive.  You might be getting better or you might not, but if you are, it's not rapidly enough that it's clearly reflected in your high score count.

In general, the best way to be sure you're getting better is to keep track of all of your scores and look for improvements in the average, but very few people have the time or motivation to do that. The nice thing about counting high scores and using the table is that you don't even need to keep careful track of the number of games you've played -- you only need a rough idea.  The difference between your target number of high scores after 100 games is only one more than after 50, and only one less than after 250. 

The other nice thing about counting high scores is that it's about beating yourself, not some random person whose score may or may not be a useful baseline.  Instead of trying to be the best -- I mean, how many people really want to try for the all-time high score in Pac-Man or Donkey Kong -- the goal becomes finding ways to be noticeably better.  Personally, I find that much more motivating and fun.

A Test Case: Asteroids (1979)

Now let's put high-score counting to the test in a real video game.  I recorded my scores in 110 games of Asteroids (which I previously discussed here and here), with the following results.

According to the table, I would have needed nine high scores in 100 games to be sure I was getting better.  As it turned out, I got 12 high scores, an outcome that only occurred 8 times in 5000 simulations where the scores were randomized.  So I was almost definitely getting better.

I'll admit, I was a little surprised by this outcome.  At a glance, my scores do seem fairly random and I didn't feel like I was making massive strides in my performance with time.  Most notably, the frequency of stinkers seemed to be about the same in the first 20 games as in the last 20, even if my higher scores were rising.  The plot is a little bit deceptive because the game awards extra lives every 10,000 points, making it seem as if my highest scores are bigger performance outliers than they really are, but it's still surprising that the one time I passed 20,000 points I was also able to pass 30,000.

Postscript

I'll be making more use of the high score counts table, and plots like those above, in future blog entries to see how my performance changed with time in different kinds of games.  Arcade games, like Asteroids, are usually designed with a high level of difficulty, so I wouldn't be surprised if I end up below my target number of high scores in many cases.  Regardless, it will be fun to find out!

^*Actually, the coin game is a little bit unusual in that there is a non-negligible probability of tying your high score. The odds of beating your high score came out a bit less than the other games that I simulated, likely because of the existence of this extra outcome.