MLRC 2025

Welcome to the home of ML Reproducibility Challenge. This is an annual event promoting research into reproducibility of Machine Learning literature. (v1, v2, v3, v4, v5, v6, v7). This conference is an unique venue in the Machine Learning community to share, disseminate and discuss reproducible methods and tools, investigate reproducibility of papers accepted for publication at top conferences, and test generalizability of scientific findings by adding novel insights and empirical results.

Venue

We are happy to announce that the Princeton Laboratory for Artificial Intelligence is hosting MLRC 2025, which will be held in-person as a one-day conference, a Princeton University, NJ, USA on August 21st, 2025. The conference will be single-track, with a mix of invited talks, oral presentations and poster sessions. Checkout our announcement blog for more details!

Accepted Papers

We are now happy to announce the list of accepted papers at MLRC 2025 - congratulations to all authors! We will notify you shortly on the camera ready instructions.

• Do not trust what you trust: Miscalibration in Semisupervised Learning, Shambhavi Mishra, Balamurali Murugesan, Ismail Ben Ayed, Marco Pedersoli, Jose Dolz
• Multivariate Dense Retrieval: A Reproducibility Study under a Memory-limited Setup, Georgios Sidiropoulos, Samarth Bhargav, Panagiotis Eustratiadis, Evangelos Kanoulas
• Improving Interpretation Faithfulness for Vision Transformers, Izabela Kurek, Wojciech Trejter, Stipe Frkovic, Andro Erdelez
• Reassessing Fairness: A Reproducibility Study of NIFA’s Impact on GNN Models, Ruben Figge, Sjoerd Gunneweg, Aaron Kuin, Mees Lindeman
• A reproducibility study of “User-item fairness tradeoffs in recommendations”, Sander Honig, Elyanne Oey, Lisanne Wallaard, Sharanda Suttorp, Clara Rus
• Reproducibility study of: “Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals”, Tijs Wiegman, Leyla Perotti, Viktória Pravdová, Ori Brand, Maria Heuss
• On the Generalizability of “Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals”, Asen Dotsinski, Udit Thakur, Marko Ivanov, Mohammad Hafeez Khan, Maria Heuss
• Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models, Atharv Mittal, Agam Pandey, Amritanshu Tiwari, Sukrit Jindal, Swadesh Swain
• Reproducibility Study of “Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation”, Jose L. García, Karolína Hájková, Maria Marchenko, Carlos Miguel Patiño
• Remembering to Be Fair Again: Reproducing Non-Markovian Fairness in Sequential Decision Making, Domonkos Nagy, Lohithsai Yadala Chanchu, Krystof Bobek, Xin Zhou, Jacobus Smit
• Reproducibility Study of ’SLICE: Stabilized LIME for Consistent Explanations for Image Classification’, Aritra Bandyopadhyay, Chiranjeev Bindra, Roan van Blanken, Arijit Ghosh
• GNNBoundary: Finding Boundaries and Going Beyond Them, Jan Henrik Bertrand, Lukas Bierling, Ina Klaric, Aron Wezenberg
• Revisiting Discover-then-Name Concept Bottleneck Models: A Reproducibility Study, Freek Byrman, Emma Kasteleyn, Bart Kuipers, Daniel Uyterlinde
• Benchmarking LLM Capabilities in Negotiation through Scoreable Games, Jorge Carrasco Pollo, Ioannis Kapetangeorgis, Joshua Rosenthal, John Hua Yao
• ModernTCN Revisited: A Critical Look at the Experimental Setup in General Time Series Analysis, Önder Akacik,Mark Hoogendoorn
• Reproducibility Study of “Improving Interpretation Faithfulness For Vision Transformers”, Meher Changlani, Benjamin Hucko, Aswin Krishna Mahadevan, Ioannis Kechagias
• A Reproducibility Study of Decoupling Feature Extraction and Classification Layers for Calibrated Neural Networks, Eric Banzuzi, Johanna D’ciofalo Khodaverdian, Katharina Deckenbach

We have sent you an RSVP for attendance and application for travel grant, please check your registered inbox!

Important Dates

• Submit to TMLR OpenReview: https://openreview.net/group?id=TMLR
• Deadline to share your intent to submit a TMLR paper to MLRC: February 21st, 2025
• This form requires that you provide a link to your TMLR submission. Once it gets accepted (if it isn’t already), you should then update the same form with your paper camera ready details.
• Cutoff deadline for receiving TMLR decisions: June 20th, 2025
• Deadline for announcing accepted papers: June 27th, 2025
• Conference day: August 21st, 2025 at Princeton University, NJ, USA

Keynote Speakers

• Arvind Narayan, Professor of Computer Science at Princeton University & Director of the Center for Information Technology Policy
• Soumith Chintala, Founder of Pytorch, Research Engineering Lead at Meta
• Jonathan Frankle, Chief AI Scientist at Databricks
• Jesse Dodge, Senior Research Scientist, Allen Institute for AI
• Stella Biderman, Executive Director, Eleuther AI

Organizer

• The Princeton Laboratory for Artificial Intelligence

Co-Organizers

General Chair

• Koustuv Sinha, Meta

Program Chairs

• Jessica Forde, Brown University
• Adina Williams, Meta
• Angela Fan, Meta
• Mike Rabbat, Meta
• Naila Murray, Meta

Local Chairs

• Arvind Narayanan, Princeton University, Senior Local Chair
• Peter Henderson, Princeton University, Local Chair

Senior Program Chair

• Joelle Pineau, Meta / Mila - Quebec AI / McGill University

Event Sponsors

The Machine Learning Reproducibility Challenge Conference is hosted by Princeton University’s AI Lab, with generous support from the following sponsors:

• Meta AI
• OpenAI

Contact

• For queries related to the conference, or sponsorship requests, please contact us at ailab@princeton.edu and mlrc-2025@googlegroups.com or reproducibility.challenge@gmail.com
• Follow us on Social media for updates: Twitter (@repro_challenge), BlueSky (@reproml.org)