Online Proceedings

2025

🥇 Best Paper   🥈 Outstanding Paper

• Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models Atharv Mittal, Agam Pandey, Amritanshu Tiwari, Sukrit Jindal, Swadesh Swain 🥇
• Do not trust what you trust: Miscalibration in Semisupervised Learning Shambhavi Mishra, Balamurali Murugesan, Ismail Ben Ayed, Marco Pedersoli, Jose Dolz 🥈
• ModernTCN Revisited: A Critical Look at the Experimental Setup in General Time Series Analysis Önder Akacik,Mark Hoogendoorn 🥈
• Multivariate Dense Retrieval: A Reproducibility Study under a Memory-limited Setup Georgios Sidiropoulos, Samarth Bhargav, Panagiotis Eustratiadis, Evangelos Kanoulas
• Improving Interpretation Faithfulness for Vision Transformers Izabela Kurek, Wojciech Trejter, Stipe Frkovic, Andro Erdelez
• Reassessing Fairness: A Reproducibility Study of NIFA’s Impact on GNN Models Ruben Figge, Sjoerd Gunneweg, Aaron Kuin, Mees Lindeman
• A reproducibility study of “User-item fairness tradeoffs in recommendations” Sander Honig, Elyanne Oey, Lisanne Wallaard, Sharanda Suttorp, Clara Rus
• Reproducibility study of: “Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals” Tijs Wiegman, Leyla Perotti, Viktória Pravdová, Ori Brand, Maria Heuss
• On the Generalizability of “Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals” Asen Dotsinski, Udit Thakur, Marko Ivanov, Mohammad Hafeez Khan, Maria Heuss
• Reproducibility Study of “Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation” Jose L. García, Karolína Hájková, Maria Marchenko, Carlos Miguel Patiño
• Remembering to Be Fair Again: Reproducing Non-Markovian Fairness in Sequential Decision Making Domonkos Nagy, Lohithsai Yadala Chanchu, Krystof Bobek, Xin Zhou, Jacobus Smit
• Reproducibility Study of ‘SLICE: Stabilized LIME for Consistent Explanations for Image Classification’ Aritra Bandyopadhyay, Chiranjeev Bindra, Roan van Blanken, Arijit Ghosh
• GNNBoundary: Finding Boundaries and Going Beyond Them Jan Henrik Bertrand, Lukas Bierling, Ina Klaric, Aron Wezenberg
• Revisiting Discover-then-Name Concept Bottleneck Models: A Reproducibility Study Freek Byrman, Emma Kasteleyn, Bart Kuipers, Daniel Uyterlinde
• Benchmarking LLM Capabilities in Negotiation through Scoreable Games Jorge Carrasco Pollo, Ioannis Kapetangeorgis, Joshua Rosenthal, John Hua Yao
• Reproducibility Study of “Improving Interpretation Faithfulness For Vision Transformers” Meher Changlani, Benjamin Hucko, Aswin Krishna Mahadevan, Ioannis Kechagias
• A Reproducibility Study of Decoupling Feature Extraction and Classification Layers for Calibrated Neural Networks Eric Banzuzi, Johanna D’ciofalo Khodaverdian, Katharina Deckenbach

2023

🎓 Poster : NeurIPS 2024 Poster Sessions, Dec 10th to 15th, 2024, Vancouver, Canada

• GNNInterpreter: A probabilistic generative model-level explanation for Graph Neural Networks; Ana-Maria Vasilcoiu, Batu Helvacioğlu, Thies Kersten, Thijs Stessen; 🎓 Poster
• Explaining Temporal Graph Models through an Explorer-Navigator Framework, Miklos Hamar, Matey Krastev, Kristiyan Hristov, David Beglou
• Reproducibility Study of “Explaining RL Decisions with Trajectories”, Clio Feng, Colin Bot, Bart den Boef, Bart Aaldering
• Transfer Learning with Informative Priors: Simple Baselines Better than Previously Reported, Ethan Harvey, Mikhail Petrov, Michael C. Hughes; :mortar_board Poster
• Reproducibility study of FairAC, Gijs de Jong,Macha Meijer,Derck W.E. Prinzhorn,Harold Ruiter; 🎓 Poster
• On the Reproducibility of Post-Hoc Concept Bottleneck Models, Nesta Midavaine, Gregory Hok Tjoan Go, Diego Canez, Ioana Simion, Satchit Chatterji; 🎓 Poster
• Reproducibility Study of “Learning Perturbations to Explain Time Series Predictions, Jiapeng Fan, Paulius Skaigiris, Luke Cadigan, Sebastian Uriel Arias
• Explaining RL Decisions with Trajectories’: A Reproducibility Study, Karim Ahmed Abdel Sadek, Matteo Nulli, Joan Velja, Jort Vincenti
• Classwise-Shapley values for data valuation, Markus Semmler, Miguel de Benito Delgado
• Reproducibility Study of “ITI-GEN: Inclusive Text-to-Image Generation”, Daniel Gallo Fernández, Răzvan-Andrei Matișan, Alejandro Monroy Muñoz, Janusz Partyka; 🎓 Poster
• Reproducibility study of “Robust Fair Clustering: A Novel Fairness Attack and Defense Framework, Kacper Bartosik, Eren Kocadag, Vincent Loos, Lucas Ponticelli, 🎓 Poster
• CUDA: Curriculum of Data Augmentation for Long‐Tailed Recognition, Barath Chandran C; 🎓 Poster
• Reproducibility Study of “Explaining Temporal Graph Models Through an Explorer-Navigator Framework”, Christina Isaicu, Jesse Wonnink, Andreas Berentzen, Helia Ghasemi; 🎓 Poster
• Reproducibility Study of “Robust Fair Clustering: A Novel Fairness Attack and Defense Framework”, Iason Skylitsis, Zheng Feng, Idries Nasim, Camille Niessink; 🎓 Poster
• Reproducibility Study on Adversarial Attacks Against Robust Transformer Trackers, Fatemeh Nourilenjan Nokabadi, Jean-Francois Lalonde, Christian Gagné; 🎓 Poster
• Reproducibility study of “LICO: Explainable Models with Language-Image Consistency”, Luan Fletcher, Robert van der Klis, Martin Sedlacek, Stefan Vasilev, Christos Athanasiadis; 🎓 Poster
• On the Reproducibility of: “Learning Perturbations to Explain Time Series Predictions”, Wouter Bant, Ádám Divák, Jasper Eppink, Floris Six Dijkstra; 🎓 Poster
• Reproducibility Study: Equal Improvability: A New Fairness Notion Considering the Long-Term Impact, Berkay Chakar,Amina Izbassar,Mina Janićijević,Jakub Tomaszewski; 🎓 Poster
• Chain-of-Thought Unfaithfulness as Disguised Accuracy, Oliver Bentham, Nathan Stringham, Ana Marasović
• Unmasking the Veil: An Investigation into Concept Ablation for Privacy and Copyright Protection in Images, Shivank Garg, Manyana Tiwari
• Studying How to Efficiently and Effectively Guide Models with Explanations” - A Reproducibility Study, Adrian Sauter, Milan Miletić, Ryan Ott, Rohith Saai Pemmasani Prabakaran; 🎓 Poster
• Reproducibility Study Of Learning Fair Graph Representations Via Automated Data Augmentations, Thijmen Nijdam, Taiki Papandreou-Lazos, Jurgen de Heus, Juell Sprott; 🎓 Poster

2022

Editorial: ML Reproducibility Challenge 2022 — Sinha, K., Bleeker, M., Bhargav, S., et al.

🥇 Outstanding Paper   🥈 Outstanding Paper (Honorable Mention)

• A Replication Study of Compositional Generalization Works on Semantic Parsing Sun, K., Williams, A., and Hupkes, D. 🥇
• Pure Noise to the Rescue of Insufficient Data Lee, S.R. and Lee, S.B. 🥇
• On Explainability of Graph Neural Networks via Subgraph Explorations Mahlau, Y., Kayser, L., and Berg, L. 🥈
• “Towards Understanding Grokking” Alexander, S., Ildus, S., and Evgeniy, S. 🥈
• Reproducibility Study of Behavior Transformers Moalla, S., Madeira, M., Riccio, L., and Lee, J. 🥈
• Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization Erkol, M., Kınlı, F., Özcan, B., and Kıraç, F.
• End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking McLeish, S. and Tran-Thanh, L.
• Label-Free Explainability for Unsupervised Models Langezaal, E.R., Belleman, J., Veenboer, T., and Noorthoek, J.
• Exploring the Representation of Word Meanings in Context Brivio, M. and Çöltekin, Ç.
• Intriguing Properties of Contrastive Losses Marini, L., Nabeel, M., and Loiko, A.
• Bandit Theory and Thompson Sampling-guided Directed Evolution for Sequence Optimization Žontar, L.
• Hypergraph-Induced Semantic Tuplet Loss for Deep Metric Learning Yuan, J. and Le-Phuoc, D.
• Easy Bayesian Transfer Learning with Informative Priors Špendl, M. and Pirc, K.
• On the Reproducibility of CartoonX Dubbeldam, E., Eijpe, A., Ruthardt, J., and Sasse, R.
• Reproducibility Study of “Label-Free Explainability for Unsupervised Models” Pariza, V., Pal, A., Pawar, M., and Faber, Q.S.
• FOCUS: Flexible Optimizable Counterfactual Explanations for Tree Ensembles Morita, K.
• Fairness Guarantees under Demographic Shift Buchner, V.L., Schutte, P.O.O., Allal, Y.B., and Ahadi, H.
• DialSummEval - Evaluation of automatic summarization evaluation metrics Camara, P., Kloos, M., Kyrmanidi, V., Kluska, A., Terlou, R., and Krause, L.
• On the Reproducibility of “FairCal: Fairness Calibration for Face Verification” Don, M., Chatterji, S., Kapralova, M., and Amaudruz, R.
• Reproducibility Study: Label-Free Explainability for Unsupervised Models Garcarz, S., Giorkatzi, A., Ivășchescu, A., and Pîslar, T.-M.
• Numerical influence of ReLU’(0) on backpropagation Martorella, T., Contreras, H.M.R., and García, D.C.
• Hierarchical Shrinkage: Improving the Accuracy and Interpretability of Tree-Based Methods Mohorčič, D. and Ocepek, D.
• Reproducibility study of Joint Multisided Exposure Fairness for Recommendation Hu, A., Ranum, O., Pozrikidou, C., and Zhou, M.
• Exploring the Explainability of Bias in Image Captioning Models Türk, M., Busser, L., van Dijk, D., and Bosch, M.J.A.
• Reproducibility study of ‘Proto2Proto: Can you recognize the car, the way I do?’ Bikker, D., de Kleuver, G., Hu, W., and Veenman, B.
• Reproducibility Study of “Focus On The Common Good: Group Distributional Robustness Follows” Simoncini, W., Gogou, I., Lopes, M.F., and Kremer, R.
• Reproducibility study of “Label-Free Explainability for Unsupervised Models” Papp, G., Wagenbach, J., de Vries, L.J., and Mather, N.
• Reproducibility study of “Explaining Deep Convolutional Neural Networks via Latent Visual-Semantic Filter Attention” Buis, E., Dijkstra, S., and Heijermans, B.
• Reproducibility Study of “Quantifying Societal Bias Amplification in Image Captioning” Baratov, F., Yüksel, G., Petcu, D., and Bakker, J.
• On the reproducibility of “CrossWalk: Fairness-Enhanced Node Representation Learning” Zila, E., Gerbscheid, J., Sträter, L., and Kretschmar, K.
• Reproducing FairCal: Fairness Calibration for Face Verification Greven, J., Stallinga, S., and Seljee, Z.
• Reproducibility Study of ‘CartoonX: Cartoon Explanations of Image Classifiers’ Taslimi, S., Foeng, L.C.A., Kayal, P., and Patra, A.P.
• Reproducibility Study of “Latent Space Smoothing for Individually Fair Representations” Merk, D., Smit, D., Beukers, B., and Mendsuren, T.
• Variational Neural Cellular Automata Aillet, A. and Sondén, S.
• If you like Shapley, then you’ll love the core Benmerzoug, A. and de Benito Delgado, M.
• A Reproduction of Automatic Multi-Label Prompting: Simple and Interpretable Few-Shot Classification Livernoche, V. and Sujaya, V.
• G-Mixup: Graph Data Augmentation for Graph Classification Cordaro, D., Cox, S., Ren, Y., and Yu, T.
• Exploring the Role of Grammar and Word Choice in Bias Toward African American English (AAE) in Hate Speech Classification Bose, P., Pandey, C.S., and Fund, F.
• RELIC: Reproducibility and Extension on LIC metric in quantifying bias in captioning models Antequera, P., Gonzalez, E., Grasa, M., and van Raaphorst, M.
• VAE Approximation Error: ELBO and Exponential Families Kyrylov, V., Bedi, N.S., and Zang, Q.
• CrossWalk Fairness-enhanced Node Representation Learning Moens, G.J., de Witte, J., Göbel, T.P., and van den Oever, M.
• CrossWalk: Fairness-enhanced Node Representation Learning Pantea, L. and Blahovici, A.
• Masked Autoencoders Are Small Scale Vision Learners: A Reproduction Under Resource Constraints Charisoudis, A., von Huth, S.E., and Jansson, E.
• G-Mixup: Graph Data Augmentation for Graph Classification Omeragić, E. and Đuranović, V.

2021

Editorial: ML Reproducibility Challenge 2021 — Sinha, K., Dodge, J., Luccioni, S., et al.

🥇 Best Paper   🥈 Outstanding Paper

• Reproducibility Study of “Counterfactual Generative Networks” Bagad, P., Hilders, P., Maas, J., and de Goede, D. 🥇
• An Implementation of Fair Robust Learning Hardy, I. 🥈
• Learning to count everything Kljun, M., Teršek, M., and Vreš, D. 🥈
• Strategic classification made practical: reproduction Kolkman, G., Athmer, J., Labro, A., and Kulicki, M. 🥈
• Exacerbating Algorithmic Bias through Fairness Attacks Tafuro, M., Lombardo, A., Veljković, T.H., and Becker-Czarnetzki, L. 🥈
• Counterfactual Generative Networks Ankit, A., Ambekar, S., Varadharajan, B., and Alence, M.
• Does Self-Supervision Always Improve Few-Shot Learning? Ashok, A. and Aekula, H.
• Weakly-Supervised Semantic Segmentation via Transformer Explainability Athanasiadis, I., Moschovis, G., and Tuoma, A.
• Data-Driven Methods for Balancing Fairness and Efficiency in Ride-Pooling de Boer, S., Cosma, R.A., Knobel, L., Koishekenov, Y., and Shaffrey, B.
• Reproducibility Study - SCOUTER: Slot Attention-based Classifier for Explainable Image Recognition Burger, M., ter Burg, K., Titarsolej, S., and Khan, S.J.
• AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients Buvanesh, A. and Panwar, M.
• GANSpace: Discovering Interpretable GAN Controls Dasu, V.A. and T.K., M.M.
• Replication study of “Privacy-preserving Collaborative Learning” Drabent, K., Wijnja, S., Sluijter, T., and Bereda, K.
• A Cluster-based Approach for Improving Isotropy in Contextual Embedding Space Džubur, B.
• Reproduction and Extension of “Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation” Eaton, E. and Naghavi, P.
• Reproduction Study of Variational Fair Clustering Eijkelboom, F., Fokkema, M., Lau, A., and Verheijen, L.
• Understanding Self-Supervised Learning Dynamics without Contrastive Pairs Höppe, T., Miszkurka, A., and Wilkman, D.B.
• Domain Generalization using Causal Matching Jiles, R. and Chakraborty, M.
• Reproducibility Study of ‘Exacerbating Algorithmic Bias through Fairness Attacks’ Kirca, I.-A., Hamerslag, D., Baas, A., and Prent, J.
• Thompson Sampling for Bandits with Clustered Arms De Luisa, A.
• Projection-based Algorithm for Updating the TruncatedSVD of Evolving Matrices Chen, A., Matsumoto, S., and Varma, R.S.
• Background-Aware Pooling and Noise-Aware Loss for Weakly-Supervised Semantic Segmentation Mehta, A., Uppal, K., Jadhav, K., Natarajan, M., Agrawal, M., and Chakravarty, D.
• Reproducibility Study: Comparing Rewinding and Fine-tuning in Neural Network Pruning Mikler, S.
• Replicating and Improving GAN2Shape Through Novel Shape Priors and Training Steps Galatolo, A. and Nilsson, A.
• Value Alignment Verification Panigrahi, S.S. and Patnaik, S.
• Replication Study of “Fairness and Bias in Online Selection” Petcu, R., Praat, P., Wijnen, J., and Rerres, M.
• Differentiable Spatial Planning using Transformers Ranjan, R., Bhakta, H., Jha, A., and Maheshwari, P.
• Solving Phase Retrieval With a Learned Reference Rucks, N., Uelwer, T., and Harmeling, S.
• Reproducibility Report: Contrastive Learning of Socially-aware Motion Representations Sen, R., Sinha, S., Jha, A., and Maheshwari, P.
• From goals, waypoints and paths to long-term human trajectory forecasting Shukla, A., Roy, S., Chawla, Y., et al.
• Graph Edit Networks Stropnik, V. and Oražem, M.
• Badder Seeds: Reproducing the Evaluation of Lexical Methods for Bias Measurement van der Togt, J., Tiyavorabun, L., Rosati, M., and Starace, G.
• Transparent Object Tracking Benchmark Trojer, Ž.
• Replication Study of DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networks Shulev, V., Verhagen, P., Wang, S., and Zhuge, J.
• Privacy-preserving collaborative learning with automatic transformation search Warmerdam, A.T., Loerakker, L., Meijer, L., and Nissen, O.
• Robust Counterfactual Explanations on Graph Neural Networks Wilschut, R.I., Wiggers, T.P.A., Oort, R.S., and van Orden, T.A.
• Lifting 2D StyleGAN for 3D-Aware Face Generation Yılmaz, D., Kınlı, F., Özcan, B., and Kıraç, F.
• Learning Unknown from Correlations: Graph Neural Network for Inter-novel-protein Interaction Prediction Zrimšek, U.
• Nondeterminism and Instability in Neural Network Optimization Ahmed, W. and Samuel, S.

2020

Editorial: ML Reproducibility Challenge 2020 — Sinha, K., Dodge, J., Luccioni, S., Forde, J.Z., Stojnic, R., and Pineau, J.

• Explaining Groups of Points in Low-Dimensional Representations Verma, R., Wagemans, J.J.O., Dahal, P., and Elfrink, A.
• On end-to-end 6DoF object pose estimation and robustness to object scale Albanis, G., Zioulis, N., Chatzitofis, A., Dimou, A., Zarpalas, D., and Daras, P.
• Neural Networks Fail to Learn Periodic Functions and How to Fix It Arvind, M. and Mama, M.
• Deep Fair Clustering for Visual Learning Teule, T., Reints, N., Gerges, C.A., and Baanders, P.
• Training Binary Neural Networks using the Bayesian Learning Rule Garg, P., Singhal, L., and Sardana, A.
• Reproducing Learning to Deceive With Attention-Based Explanations Habacker, R., Harrison, A., Parisot, M., and Snijders, A.
• Parameterized Explainer for Graph Neural Network Holdijk, L., Boon, M., Henckens, S., and de Jong, L.
• Don’t Judge an Object by Its Context: Learning to Overcome Contextual Bias Kim, S.S.Y., Zhang, S., Meister, N., and Russakovsky, O.
• Reimplementation of FixMatch and Investigation on Noisy (Pseudo) Labels and Confirmation Errors of FixMatch Li, C., Tu, R., and Zhang, H.
• A Reproduction of Ensemble Distribution Distillation Liiv, T., Lennelöv, E., and Norén, A.
• Learning Memory Guided Normality for Anomaly Detection Stephen, K. and Menon, V.
• Spatial-Adaptive Network for Single Image Denoising Menteş, S., Kınlı, F., Özcan, B., and Kıraç, F.
• Warm-Starting Neural Network Training Kireev, K., Mohtashami, A., and Pajouheshgar, E.
• Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings P, J.J. and Sardana, A.
• Explaining Groups of Points in Low-Dimensional Representations Reijnaers, D.J.W., van de Pavert, D.B., Scheuer, G., and Huang, L.
• Hamiltonian Generative Networks Balsells Rodas, C., Canal Anton, O., and Taschin, F.
• Satellite Image Time Series Classification with Pixel-Set Encoders and Temporal Self-Attention Schneider, M. and Körner, M.
• Reproducibility report of “Interpretable Complex-Valued Neural Networks for Privacy Protection” Sheverdin, A., Knijff, A., Corten, N., and Lange, G.
• Rigging the Lottery: Making All Tickets Winners Sundar, V. and Dwaraknath, R.V.

NeurIPS 2019

Editorial: NeurIPS 2019 Reproducibility Challenge — Sinha, K., Pineau, J., Forde, J., Ke, R.N., and Larochelle, H.

• A comprehensive study on binary optimizer and its applicability Nayak, N., Raj, V., and Kalyani, S.
• Generative Modeling by Estimating Gradients of the Data Distribution Matosevic, A., Hein, E., and Nuzzo, F.
• Unsupervised Scalable Representation Learning for Multivariate Time Series Liljefors, F., Sorkhei, M., and Broomé, S.
• Tensor Monte Carlo: Particle Methods for the GPU Era Kviman, O., Nilsson, L., and Larsson, M.
• Hamiltonian Neural Networks Garg, A. and Kagi, S.S.
• Zero-Shot Knowledge Transfer via Adversarial Belief Matching Ferles, A., Nöu, A., and Valavanis, L.
• When to Trust Your Model: Model-Based Policy Optimization Liu, Y., Xu, J., and Pan, Y.
• Unsupervised Representation Learning in Atari Alacchi, G., Lam, G., and Perreault-Lafleur, C.
• One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers Gohil, V., Narayanan, S.D., and Jain, A.
• Improved Calibration and Predictive Uncertainty for Deep Neural Networks Singh, A. and Bay, A.

ICLR 2019

Editorial: ICLR Reproducibility Challenge 2019 — Pineau, J., Sinha, K., Fried, G., Ke, R.N., and Larochelle, H.

• Meta-learning with differentiable closed-form solvers Devos, A., Chatel, S., and Grossglauser, M.
• Variational Sparse Coding de la Fuente, A. and Aduviri, R.
• Learning Neural PDE Solvers with Convergence Guarantees Bardi, F., von Baussnern, S., and Gjiriti, E.
• h-detach: Modifying the LSTM gradient towards better optimization Didolkar, A.