How to Determine An Element's Colour
Consider the energy required to elevate electrons from one "shell" to another, and the frequency of visible light that has this amount of energy.  The substance is more likely to absorb wavelengths of light matching the energy difference of that band gap, and reflect or transmit others.  The colors NOT absorbed can reach your eye as an observer, so (assuming a white light source) you see white (which is all the colors together) minus the colors absorbed by the material (matching the band gap).  For example, gold (Au) absorbs the blues and greens and leaves just the colors adding up to what you now think of as "gold."  If your light source is not white, what your eyes see will also be missing any colors your light source was missing.
The crystal structure can also make a difference, e.g. compare the colors of carbon (C) as graphite vs. diamond.

Useful links: See also this same question on ResearchGate, WebExhibits, and Quora.