Which component in my tap water "scale" does not react with acid?
A likely candidate is calcium sulfate, which forms a relatively soft mineral (gypsum), hence removable with a knife, but it has limited solubility in water and does not react with common acids.

See here for more on water hardness.