How do I fetch versions of a module compatible with latest core version?
You can use https://updates.drupal.org/release-history/search_api_solr/current to get the versions either prefixed with 8.x- or without an API compatibility prefix. (The latter is the case for SEMVER versions.)

The output from that URL contains information about this release, for example.

&lt;release&gt;
&lt;name&gt;search_api_solr 4.1.1&lt;/name&gt;
&lt;version&gt;4.1.1&lt;/version&gt;
&lt;tag&gt;4.1.1&lt;/tag&gt;
&lt;status&gt;published&lt;/status&gt;
&lt;release_link&gt;https://www.drupal.org/project/search_api_solr/releases/4.1.1&lt;/release_link&gt;
&lt;download_link&gt;https://ftp.drupal.org/files/projects/search_api_solr-4.1.1.tar.gz&lt;/download_link&gt;
&lt;date&gt;1591714636&lt;/date&gt;
&lt;files&gt;
&lt;file&gt;
&lt;url&gt;https://ftp.drupal.org/files/projects/search_api_solr-4.1.1.tar.gz&lt;/url&gt;
&lt;archive_type&gt;tar.gz&lt;/archive_type&gt;
&lt;md5&gt;7f2dbc01229b118c02c66f4ac210d430&lt;/md5&gt;
&lt;size&gt;3207604&lt;/size&gt;
&lt;filedate&gt;1591714636&lt;/filedate&gt;
&lt;/file&gt;
&lt;file&gt;
&lt;url&gt;https://ftp.drupal.org/files/projects/search_api_solr-4.1.1.zip&lt;/url&gt;
&lt;archive_type&gt;zip&lt;/archive_type&gt;
&lt;md5&gt;511e33c0b0ae47a0bd455109f87d07d1&lt;/md5&gt;
&lt;size&gt;3792678&lt;/size&gt;
&lt;filedate&gt;1591714636&lt;/filedate&gt;
&lt;/file&gt;
&lt;/files&gt;
&lt;terms&gt;
&lt;term&gt;
&lt;name&gt;Release type&lt;/name&gt;
&lt;value&gt;Bug fixes&lt;/value&gt;
&lt;/term&gt;
&lt;/terms&gt;
&lt;security covered="1"&gt;Covered by Drupal's security advisory policy&lt;/security&gt;
&lt;core_compatibility&gt;^8.8 || ^9&lt;/core_compatibility&gt;
&lt;/release&gt;


It also contains data for this release. (This shows that the URL doesn't return information only for SEMVER versions.)

&lt;release&gt;
&lt;name&gt;search_api_solr 8.x-3.9&lt;/name&gt;
&lt;version&gt;8.x-3.9&lt;/version&gt;
&lt;tag&gt;8.x-3.9&lt;/tag&gt;
&lt;status&gt;published&lt;/status&gt;
&lt;release_link&gt;https://www.drupal.org/project/search_api_solr/releases/8.x-3.9&lt;/release_link&gt;
&lt;download_link&gt;https://ftp.drupal.org/files/projects/search_api_solr-8.x-3.9.tar.gz&lt;/download_link&gt;
&lt;date&gt;1581700341&lt;/date&gt;
&lt;files&gt;
&lt;file&gt;
&lt;url&gt;https://ftp.drupal.org/files/projects/search_api_solr-8.x-3.9.tar.gz&lt;/url&gt;
&lt;archive_type&gt;tar.gz&lt;/archive_type&gt;
&lt;md5&gt;5050a8e2bd6b4965d414cade7aae3dc8&lt;/md5&gt;
&lt;size&gt;1052117&lt;/size&gt;
&lt;filedate&gt;1581700341&lt;/filedate&gt;
&lt;/file&gt;
&lt;file&gt;
&lt;url&gt;https://ftp.drupal.org/files/projects/search_api_solr-8.x-3.9.zip&lt;/url&gt;
&lt;archive_type&gt;zip&lt;/archive_type&gt;
&lt;md5&gt;140f15134d4e8f2239b96e682186e021&lt;/md5&gt;
&lt;size&gt;1278054&lt;/size&gt;
&lt;filedate&gt;1581700341&lt;/filedate&gt;
&lt;/file&gt;
&lt;/files&gt;
&lt;terms&gt;
&lt;term&gt;
&lt;name&gt;Release type&lt;/name&gt;
&lt;value&gt;Bug fixes&lt;/value&gt;
&lt;/term&gt;
&lt;term&gt;
&lt;name&gt;Release type&lt;/name&gt;
&lt;value&gt;New features&lt;/value&gt;
&lt;/term&gt;
&lt;/terms&gt;
&lt;security covered="1"&gt;Covered by Drupal's security advisory policy&lt;/security&gt;
&lt;core_compatibility&gt;^8.7.7 || ^9&lt;/core_compatibility&gt;
&lt;/release&gt;


Unfortunately, there isn't a way to get all the versions that are compatible also with Drupal 9 and not just with Drupal 8, using an updates.drupal.org URL. For that, you need to parse the XML output and filter those versions containing ^9 in the &lt;core_compatibility&gt; tag.

As side note, the all, 8.x, 7.x end-points are considered legacy. I guess that means they could be removed in the future.

References


Update status XML