Resources

This page lists some useful resources which you can use for the challenge.

Reproducible Code

Code submission are mandatory for all submitted papers. We recommend that you:

• Archive your implementation using the Software Heritage Archive, which provides a permanent, citable identifier for source code. Include a link to the archived code in your paper.
• Document your code appropriately
• Have a README.md file which describes the exact steps to run your code. You can refer to the ML Code Completeness Checklist to write the README file and make sure your code submission is complete.
• See this blog post on best practices for reproducibility.

Suggested Readings

• Online Proceedings of MLRC
• Arvind Narayanan et al, 2023; Talk: Evaluating LLMs is a Minefield
• ACL 2022 Tutorial on “Towards Reproducible Machine Learning Research in Natural Language Processing”
• NAACL 2022 Reproducibility Track
• ML Reproducibility Checklist
• ML Code Completeness Checklist
• ML reproducibility tools and best practices
• Joelle Pineau’s Keynote talk on Reproducibility at NeurIPS 2018