How can I remove a reflection onto an object but keep the shadow?
Those single pixel reflections are commonly known as fireflies.
Fireflies can often be removed by disabling caustics, as they are usually a result of an unlikely indirect sample of a lamp.
If you don't want or need caustics, then simply enable No Caustics in Render settings &gt; Light paths.
Clamping
If you want to keep caustics, you might try using the Clamp options in Render settings &gt; Sampling.
The clamp options will ensure a sample does not return a higher value than the clamp value, but be aware that this can potentially affect highlights and other intentionally bright areas of the render.
To help avoid this, in 2.70 you can now clamp indirect samples (where fireflies are usually found) separately from direct samples.

Also see:

How to reduce fireflies in Cycles?

the Reducing noise section of the wiki.