Reproducibility Challenge @ NeurIPS 2019

Welcome to the 3rd edition of Reproducibility Challenge @ NeurIPS 2019! One of the challenges in machine learning research is to ensure that published results are reliable and reproducible. In support of this, the objective of this challenge is to investigate reproducibility of empirical results submitted to the 2019 Neural Information Processing Systems (NeurIPS). We are choosing NeurIPS for this challenge because the timing is right for course-based participants (see below), and this time we focus on accepted papers which will be made available publicly on Open Review on acceptance notification.

Read our blog post explaining the thought process behind the NeurIPS Reproducibility Challenge.

In our last iteration, we ran the challenge on ICLR submitted papers. Check out the accepted reports in our journal publication at ReScience.

• News : After extensive review process, 10 reports are selected for publication in ReScience Journal. All reports (and their reviews) to be hosted indefinitely on OpenReview.
• News : We will be back for NeurIPS 2020! Read our roadmap here.

How to participate

• Check the Registration page for details on claiming a NeurIPS accepted paper from our OpenReview Portal, and then start working on it.
• Check the Task Descriptions page for details on the task and different tracks
• Check the Resources page for available resources
• Keep an eye on the Important dates and deadlines
• Submit your report in the same OpenReview Portal

Participating Institutions

• CSCI2951-F: Learning and Sequential Decision Making, Brown University
• Deep Learning, Advanced (DD2412), Division of Robotoics, Perception, and Learning (RPL), KTH (Royal Institute of Technology)
• COMP 551: Applied Machine Learning, Fall 2019, McGill University
• IFT 6269 Probabilistic Graphical Models , Fall 2019, Université de Montréal
• CS 433: Machine Learning, Fall 2019, EPFL

If you want your course to be listed here, please drop a mail to us.

Sponsors

• Cloud Platform Sponsor : Code Ocean

Contact

For any challenge specific queries, registration of your graduate course and sponsorship proposals, contact Koustuv Sinha or Reproducibility Challenge Organizers.