A large set of decimetric and microwave observations of solar millisecond radio spikes is compared and correlated with simultaneous hard X-ray observations in the 25-438 keV range, recorded by the Hard X-Ray Burst Spectrometer (HXRBS) on board the Solar Maximum Mission satellite. Radio spikes have time scales of 20-200 ms, while hard X-ray bursts rarely show fine structures shorter than 1 s. The main conclusions of a statistical analysis of a) the association rate, b) the correlation degree, and c) relative time delays between hard X-ray and radio spike emissions, are: 1) Radio spike bursts reveal the highest hard X-ray association rate of any kind of coherent solar radio emission; approximately 95% of the observed events occur during (mostly impulsive) hard X-ray bursts. Conversely, almost-equal-to 2% of all HXRBS flares produce radio millisecond spike bursts. 2) Spike bursts are preferentially associated with the impulsive phase of the hard X-ray event; about half of all observations are closely associated with the main peak of a simultaneous X-ray event. 3) We find a significant temporal correlation between spike clusters and hard X-ray events. About 43% of the compared events show hard X-ray time profiles that mimic the concentration of simultaneous radio spikes. This also holds for small fluctuations with time scales of the order of a few seconds. 4) The durations of spike events and associated hard X-ray bursts closely correlate, the X-ray bursts being somewhat longer than the radio events. 5) The radio emission is usually delayed in respect to hard X-ray bursts, typically a few seconds. The relative delays seem to be restricted to the following rules: The radio event usually starts before the peak time of the hard X-ray event (73 out of 84 cases), and similarly, the X-ray burst almost always starts before the maximum of spike burst activity (83 out of 84 cases). This is shown to hold for a wide range of burst durations. We have not found any convincing relation concerning the relative time differences of the end times, however. 6) The observed delays of the order of several seconds put some constraints on the acceleration and propagation of particles. The delays are found to be incompatible with the build-up time of a loss-cone distribution (for maser emission) by an order of magnitude, if the, same population of electrons is responsible for radio and X-ray radiation. Rather, considering the smallness of the radio spike sources, the proportionality of the X-ray flux with the spike rate, and the spatial 'spread of a propagating electron population, it is argued that both the time delays and the quantization into discrete radio events are caused by the operation of the accelerator.