Verification
Why nothing the browser reports reaches the leaderboard.
Why your score is not the score
Anything the browser reports can be edited by whoever owns the browser. So the number your tab shows while training is a preview, and it never reaches the leaderboard.
The board is written by the verifier and nothing else.
A seed you have not seen
Every submission is assigned a random evaluation seed on the server at the moment it is filed. Your agent is replayed on that seed, five runs by default, and scored on what it does there.
You cannot search for a seed your agent happens to be lucky on, because you do not have it until after the policy is sealed.
- You submitThe policy is hashed and stored. Nothing about it can change after this.
- The server draws a seedRandom, chosen here, and not before. You never had it to train against.
- It plays five runsYour agent, our machine, that seed.
- The score is writtenOnly the verifier writes to the board.
What is checked
The policy has to match the level and difficulty of the submission it was filed under, or a run could be swapped for an easier map after a harder multiplier was recorded.
Policies are capped at 2,000,000 parameters, and you may only have one run being verified per level at a time.
Copying does not help
A policy taken from someone else still has to be replayed on a seed chosen after it was submitted. Memorising one layout does not survive that, so a copied agent has to actually generalise, which is the thing worth measuring.
