Should the tester start flickering when I touch my chandelier's top wiring cover?
The type of proximity  tester you are using is fine for quick checks on conductors and surfaces, but not always reliable. Since they will detect voltage at a short distance, it often gives false positives due to the fact that a hot conductor is close to some other part, such as your canopy. Often an AC voltage will induce a phantom voltage on adjacent metal conductors that your tester will detect.

The only certain way to tell, if in fact you have voltage on your canopy, is to use a Volt meter. Since voltage on a grounded part such as your canopy indicates a short to ground, it should trip your breaker. Regardless, use a meter and test the canopy to the actual ground conductor for any voltage. Also test hot to neutral and neutral to ground. 

Another test you can try with your proximity tester is to drop the canopy and move it away from the hot wire. I bet the positive reading will disappear. Try this with the light on and off. If you still get a positive reading with the light off, you may in fact have a short to the casing of the fixture. If the positive is only when the light is on, then check to assure the neutral is not attached directly to the fixture case, as this will cause a positive reading.

Caution is always advised with working with electricity. If you continue to get positive readings or really don't understand the theory of what I have described, then get some help from an electrician to be safe.  Good Luck.