How to visualise GIST features of an image
Given that code is trivial for both (GIST + Network and Raw Pixel + Network), you can try three approaches for a given project.

GIST + Dense layers (GIST is not space-distributed)
Raw Pixels + CNN + Dense Layers
Raw pixels + CNN + Dense + input layer 2 (GIST) + Dense

For some projects, GIST can help since it is an abstract feature that CNN might or might not learn.
EDIT: This paper compares GIST and CNN
Regarding:

Is there a way to visualize the extracted features on top of the image?

This can be done with an attention layer in approach 3 (CNN + GIST).
CNN provides spacial distribution (Required for visualization) and dense layer that merges CNN's output with GIST can be used with an attention layer.
Paper for visualization