Welcome to the 3rd edition of Reproducibility Challenge @ NeurIPS 2019! One of the challenges in machine learning research is to ensure that published results are reliable and reproducible. In support of this, the objective of this challenge is to investigate reproducibility of empirical results submitted to the 2019 Neural Information Processing Systems (NeurIPS). We are choosing NeurIPS for this challenge because the timing is right for course-based participants (see below), and this time we focus on accepted papers which will be made available publicly on Open Review on acceptance notification.

Read our blog post explaining the thought process behind the NeurIPS Reproducibility Challenge.

In our last iteration, we ran the challenge on ICLR submitted papers. Check out the accepted reports in our journal publication at ReScience.

How to participate

Participating Institutions

If you want your course to be listed here, please drop a mail to us.



For any challenge specific queries, registration of your graduate course and sponsorship proposals, contact Koustuv Sinha or Reproducibility Challenge Organizers.