How to remove line sample?
Basically you have to re-initiate it and then cancel that operation.
If you have a line you can just LMB on the button once to activate it then RMB on it again.
You can also invoke the space-function search menu and search 'Sample Color' activate it and hit ESC to cancel it.