How can ADS-B replace primary radar when FlightRadar24, using ADS-B, is so inaccurate?
TLDR

How will ADS-B actually reliably replace primary radar?

It will not.

Will GPS have become mandatory by 2020?

Strictly speaking no, effectively yes.

There are so many problems with that quote from FR24 that I don't know where to start.
Let's take it apart and analyse it sentence by sentence.

The percentage of aircraft equipped with ADS-B receivers is steadily increasing though

Totally irrelevant. An ADS-B receiver allows reception of ADS-B signals from other aircraft. These can be used for example to create a Cockpit Display of Traffic Information (CDTI) or augment TCAS. The number of aircraft with this so called ADS-B IN is increasing, but the equipage percentage is probably below 2% at the moment.
If you are concerned with using ADS-B operationally for ATC then airborne receivers are not of interest. Instead you should be looking at the transmitter side.

[ADS-B receivers ...] will become mandatory for most aircraft around the world by
2020.

ADS-B receivers are not mandatory in any foreseen regulation yet. ADS-B transmitters are / will become mandatory in many parts of the world. The EU and USA rules mandate ADS-B in 2020.

When mandatory, ADS-B will replace primary radar

ADS-B is not intended to replace primary radar. As a new addition to the surveillance technology mix, it will not replace any other technology in the short term.
ADS-B is a form of cooperative surveillance, meaning that the aircraft or vehicle that is supposed to be tracked needs to cooperate by transmitting the ADS-B signal. Primary Surveillance Radar (PSR) on the other hand is a non-cooperative surveillance technology; it works by detecting signals reflecting off the target of interest. PSR can detect aircraft that do not transmit any signals. Since PSR and ADS-B are fundamentally different in their working, they have fundamentally different use cases. ADS-B will not replace primary radar and never was intended to.

will replace primary radar as the primary surveillance method used by ATC

Primary Surveillance Radar is not the primary surveillance method used by ATC in many parts of the world. Since PSR only gives azimuth and range measurements essential information like aircraft identification and altitude is missing. Therefore Secondary Surveillance Radar (SSR) is the primary surveillance method used for ATC since it provides altitude and identity in addition to azimuth and range measurements.
Secondary radar, like ADS-B is a form of cooperative surveillance; it relies on the aircraft replying to an interrogation from the radar. ADS-B can and will replace a number of secondary radars, but not all of them.

Now for the rest of your question.

it seems to me that a large portion of aircraft do not actually use GPS, but instead rely on inertial navigation systems or similar.

Indeed a large number of aircraft do transmit their position on ADS-B that is based on a non-GPS source, usually from the INS. INS can not provide the accuracy and integrity that GPS can provide and therefore you see large position errors on FR24. The fact that the position is not to be trusted is reported in the ADS-B message in the quality indicators, but unfortunately FR24 doesn't use this information.
All the ADS-B mandates require that the system is able to provide high integrity data, which effectively means that GPS is required as position source, although theoretically other means are possible as well. Therefore in 2020 all aircraft subject to the various ADS-B regulations will have to be GPS equipped.

Of course, the ATC doesn't need millimeter precision, or even an
accurate position within a kilometer,

ATC does not need millimeter precision (GPS wouldn't provide that anyway) but they do need position accuracy within a kilometre and sometimes within about 10 meters (e.g. runway entry stop bar monitoring). ADS-B  with appropriate GPS source will be able to provide that.