MLRC 2023
Final decisions for MLRC 2023
We are now releasing the final list of decisions for MLRC 2023. This list includes the previous partial list published on July 5th, 2024. We have given additional time to TMLR to complete the reviews, however it is unfortunate that few papers are still awaiting a decision due to unresponsive Action Editors from TMLR. As we need to wrap up this edition, we are proceeding with the final list of 22 accepted papers. Congratulations to all!
If you are an author of the below mentioned papers and have not submitted the form with the camera ready items, please consider doing so at the earliest. We will reach out to the accepted authors soon with the next steps. We will also announce the best paper awards and share details on the logistics of NeurIPS poster session in the coming weeks.
Update, Sept 13th, 2024: A couple of papers received acceptance status post our final date of MLRC 2023 acceptance. We have now incorporated them too in the final list.
An update on decisions
July 5th, 2024
We initially communicated to have all decisions of MLRC 2023 out by 31st of May, 2024. Unfortunately, several submissions are still under review at TMLR, and we are waiting for the final decisions to trickle in. Overall, MLRC 2023 had 46 valid submissions, out of which we have recieved decisions on 61% of them. We are in touch with TMLR to expedite the process of decisions for the remaining submissions - we expect all decisions to come in by the next couple of weeks.
Until then, we are happy to announce the (partial) list of accepted papers. Congratulations to all 🎉! If you are an author of the below mentioned papers and have not submitted the form with the camera ready items, please consider doing so at the earliest. We will reach out to the accepted authors soon with the next steps.
(partial paper list removed as we release the final list above)
[Deprecated] Call For Papers
We invite contributions from academics, practitioners and industry researchers of the ML community to submit novel and insightful reproducibility studies. Please read our blog post regarding our retrospectives of running the challenge and the future roadmap. We are happy to announce the formal partnership with Transactions of Machine Learning Research (TMLR) journal. The challenge goes live on October 23, 2023.
We recommend you choose any paper(s) published in the 2023 calendar year from the top conferences and journals (NeurIPS, ICML, ICLR, ACL, EMNLP, ICCV, CVPR, TMLR, JMLR, TACL) to run your reproducibility study on.
In order for your paper to be submitted and presented at MLRC 2023, it first needs to be accepted and published at TMLR. While TMLR aims to follow a 2-months timeline to complete the review process of its regular submissions, this timeline is not guaranteed. If you haven’t already, we therefore recommend submitting your original paper to TMLR by February 16th, 2024, that is a little over 3 months in advance of the MLRC publication announcement date.
Key Dates
- Challenge goes live: October 23, 2023
- Deadline to share your intent to submit a TMLR paper to MLRC: February 16th, 2024 at the following form: https://forms.gle/JJ28rLwBSxMriyE89. This form requires that you provide a link to your TMLR submission. Once it gets accepted (if it isn’t already), you should then update the same form with your paper camera ready details. Your accepted TMLR paper will finally undergo a light AC review to verify MLRC compatibility.
- We aim to announce the accepted papers by
May 31st, 2024July 17th, 2024, pending decisions of all papers.
Contact Information
- For query regarding MLRC 2023, mail us at: mlrc-2023@googlegroups.com.
- For general queries, media, sponsorship, partnership requests, mail us at reproducibility.challenge@gmail.com.
Organizing Committee
- Koustuv Sinha, Meta (FAIR)
- Jessica Zosa Forde, Brown University
- Mandana Samiei, McGill University, Mila
- Arna Ghosh, McGill University, Mila
- Lintang Sutawika, Eleuther AI
- Siba Smarak Panigrahi, McGill University, Mila