If you’re coding battle AIs you may want to compare different versions of your bot’s AI. The purpose of this paper is to see whether and how much the new bot is better than the old one, thanks to the concept of statistical significance.
Table of content
- In the first chapter, through examples we’ll have a first glimpse of what statistical significance is.
- In the second chapter, we’ll see formulas for statistical significance and win rate, with their use scope, numerical interpretations and how to use them in practice.
- In the third chapter, we’ll see features that must or should be implemented for the formula of statistical significance to be correctly and efficiently used.
