Teaching Fairness, Accountability, Confidentiality, and Transparency in
  Artificial Intelligence through the Lens of Reproducibility
  In this work we explain the setup for a technical, graduate-level course on
Fairness, Accountability, Confidentiality and Transparency in Artificial
Intelligence (FACT-AI) at the University of Amsterdam, which teaches FACT-AI
concepts through the lens of reproducibility. The focal point of the course is
a group project based on reproducing existing FACT-AI algorithms from top AI
conferences, and writing a report about their experiences. In the first
iteration of the course, we created an open source repository with the code
implementations from the group projects. In the second iteration, we encouraged
students to submit their group projects to the Machine Learning Reproducibility
Challenge, which resulted in 9 reports from our course being accepted to the
challenge. We reflect on our experience teaching the course over two academic
years, where one year coincided with a global pandemic, and propose guidelines
for teaching FACT-AI through reproducibility in graduate-level AI programs. We
hope this can be a useful resource for instructors to set up similar courses at
their universities in the future.
