Why do the facets display the wrong amount of results ?
All that time, I was using the default Search bar from Drupal 8 (which comes with the Search API module). 

The thing is, the results in the facets were the ones that were given after Solr handled the search, and not the Search module from Search API. 

For basic requests like a few keywords, both search engines yielded the same results, so the facets corresponded to the result, but my configuration of Solr didn't seem to handle double quotes.

In short : Search API handles the double quotes and displays the good results, but Solr didn't handle the double quotes so the facets results were wrong.