Why are goosebumps so ineffective at keeping us warm?
When your brain, the hypothalamic temperature centers in particular, detects that the temperature is too warm or cold, it initiates a number of controls to try and correct this.

Goosebumps appear due to piloerection. This is one of the reactions that occur when the temperature is too low.

This causes hairs to stand on end as a result of contractions in muscles attached to hair follicles called arrector pili. 

This particular reflex is not actually important in human beings. However, in animals, this mechanism allows entrapment of a layer of air allowing insulation. This way, the heat loss is greatly reduced.

The other mechanisms are very adept at maintaining temperature in the human body. These include sweating, dilation (vasodilation) and constriction (vasoconstriction) of skin blood vessels and increasing and decreasing the body's heat production.

For example, when it's too hot, dilation of the skin blood vessels can increase heat transfer by up to eight times.

Source: Guyton and Hall. Medical Physiology. 11th ed. Elsevier Saunders.