File size: 8,481 Bytes
d3c36f7 9871131 d3c36f7 9871131 d3c36f7 9871131 d3c36f7 9871131 d3c36f7 9871131 d3c36f7 f1c6d9b d3c36f7 9871131 d3c36f7 9871131 d3c36f7 9871131 f5272ee 9871131 d3c36f7 9871131 d3c36f7 9871131 d3c36f7 af13c8e d3c36f7 af13c8e d3c36f7 9871131 d3c36f7 742836b d3c36f7 742836b 32f609d 742836b 32f609d 742836b 32f609d d3c36f7 a0ec4db d3c36f7 a0ec4db d3c36f7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 |
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="description"
content="Empirical Benchmarking of Algorithmic Fairness in Machine Learning Models">
<meta name="keywords" content="Machine Learning, Bias Mitigation, Benchmark">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>BMBENCH: Empirical Benchmarking of Algorithmic Fairness in Machine Learning Models</title>
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
rel="stylesheet">
<link rel="stylesheet" href="./static/css/bulma.min.css">
<link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
<link rel="stylesheet" href="./static/css/bulma-slider.min.css">
<link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
<link rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
<link rel="stylesheet" href="./static/css/index.css">
<link rel="icon" href="./static/images/favicon.svg">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script defer src="./static/js/fontawesome.all.min.js"></script>
<script src="./static/js/bulma-carousel.min.js"></script>
<script src="./static/js/bulma-slider.min.js"></script>
<script src="./static/js/index.js"></script>
</head>
<body>
<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-1 publication-title">BMBENCH: Empirical Benchmarking of Algorithmic Fairness in Machine Learning Models</h1>
<div class="is-size-5 publication-authors">
<span class="author-block">
<a href="https://kleytondacosta.com" target="_blank">Kleyton da Costa</a><sup>1, 2</sup>,</span>
<span class="author-block">
<a href="https://utkarshsinha.com" target="_blank">Cristian Munoz</a><sup>1</sup>,</span>
<span class="author-block">
<a href="https://jonbarron.info" target="_blank">Bernardo Modenesi</a><sup>3</sup>,
</span>
<span class="author-block">
<a href="http://sofienbouaziz.com" target="_blank">Franklin Fernandez</a><sup>1,2</sup>,
</span>
<span class="author-block">
<a href="https://www.danbgoldman.com" target="_blank">Adriano Koshiyama</a><sup>1</sup>,
</span>
</div>
<div class="is-size-5 publication-authors">
<span class="author-block"><sup>1</sup>Holistic AI,</span>
<span class="author-block"><sup>2</sup>Pontifical Catholic University of Rio de Janeiro,</span>
<span class="author-block"><sup>2</sup>University of Utah,</span>
</div>
<div class="column has-text-centered">
<div class="publication-links">
<!-- PDF Link. -->
<span class="link-block">
<a href="https://arxiv.org/pdf/2011.12948" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fas fa-file-pdf"></i>
</span>
<span>Paper</span>
</a>
</span>
<span class="link-block">
<a href="https://arxiv.org/abs/2011.12948" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="ai ai-arxiv"></i>
</span>
<span>arXiv</span>
</a>
</span>
<!-- Code Link. -->
<span class="link-block">
<a href="https://github.com/google/nerfies" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-github"></i>
</span>
<span>Code</span>
</a>
</span>
<!-- Leaderboard. -->
<span class="link-block">
<a href="https://github.com/google/nerfies/releases/tag/0.1" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="far fa-images"></i>
</span>
<span>Leaderboard</span>
</a>
</div>
</div>
</div>
</div>
</div>
</div>
</section>
<section class="hero teaser">
<div class="container is-max-desktop">
<div class="hero-body">
<img src="./static/images/bmbench.png" alt="BMBENCH Image" width="100%">
<h2 class="subtitle has-text-centered">
<span class="dnerf">BMBENCH</span> framework and pipeline process.
</h2>
</div>
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<!-- Abstract. -->
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Abstract</h2>
<div class="content has-text-justified">
<p>
The development and assessment of bias mitigation methods require rigorous benchmarks.
This paper introduces BMBench, a comprehensive benchmarking framework to evaluate bias
mitigation strategies across multitask machine learning predictions (binary classification,
multiclass classification, regression, and clustering). Our benchmark leverages state-of-the-art
and proposed datasets to improve fairness research, offering a broad spectrum of fairness
metrics for a robust evaluation of bias mitigation methods. We provide an open-source repository
to allow researchers to test and refine their bias mitigation approaches easily,
promoting advancements in the creation of fair machine learning models.
</p>
</div>
</div>
</div>
<!--/ Abstract. -->
</section>
<section class="section">
<div class="container">
<table class="table is-striped is-hoverable is-fullwidth">
<thead>
<tr>
<th>Task</th>
<th>Stage</th>
<th>Download Link</th>
</tr>
</thead>
<tbody>
<tr>
<td>Binary Classification</td>
<td>Preprocessing</td>
<td><a href="https://huggingface.co/datasets/holistic-ai/bias_mitigation_benchmark/resolve/main/benchmark_binary_classification_preprocessing.csv">Download CSV</a></td>
</tr>
<!-- Additional rows can be added here -->
</tbody>
</table>
</div>
</section>
<style>
.table {
width: 100%;
margin-top: 20px;
}
.table th, .table td {
padding: 10px;
text-align: center;
}
.table a {
color: #3273dc;
text-decoration: none;
}
.table a:hover {
text-decoration: underline;
}
</style>
<section class="section" id="BibTeX">
<div class="container is-max-desktop content">
<h2 class="title">BibTeX</h2>
<pre><code>@article{dacosta2025bmbench,
author = {da Costa, K., Munoz, C., Modenesi, B., Fernandez, F., Koshiyama, A.},
title = {BMBENCH: Empirical Benchmarking of Algorithmic Fairness in Machine Learning Models},
journal = {ICCV},
year = {2025},
}</code></pre>
</div>
</section>
<footer class="footer">
<div class="container">
<div class="content has-text-centered">
<a class="icon-link" target="_blank"
href="./static/videos/nerfies_paper.pdf">
<i class="fas fa-file-pdf"></i>
</a>
<a class="icon-link" href="https://github.com/keunhong" target="_blank" class="external-link" disabled>
<i class="fab fa-github"></i>
</a>
</div>
<div class="columns is-centered">
<div class="column is-8">
<div class="content">
<p>
This website is licensed under a <a rel="license" target="_blank"
href="http://creativecommons.org/licenses/by-sa/4.0/">Creative
Commons Attribution-ShareAlike 4.0 International License</a>.
</p>
</div>
</div>
</div>
</div>
</footer>
</body>
</html>
|