Statistical significance: Is my new bot really better than the previous one?

Next: A first glimpse

If you’re coding battle AIs you may want to compare different versions of your bot’s AI. The purpose of this paper is to see whether and how much the new bot is better than the old one, thanks to the concept of statistical significance.

Table of content

  • In the first chapter, through examples we’ll have a first glimpse of what statistical significance is.
  • In the second chapter, we’ll see formulas for statistical significance and win rate, with their use scope, numerical interpretations and how to use them in practice.
  • In the third chapter, we’ll see features that must or should be implemented for the formula of statistical significance to be correctly and efficiently used.
Create your playground on
This playground was created on, our hands-on, knowledge-sharing platform for developers.
Go to
codingame x discord
Join the CodinGame community on Discord to chat about puzzle contributions, challenges, streams, blog articles - all that good stuff!
Online Participants