Should I remove this saddle valve, if so what is the best way to do so?
Ed has the important part of the answer. Saddle valves are trouble, and since you're not using it, it's easy to eliminate. Even if you were using it, it makes sense to replace with a soldered-in valve.

If this is copper tubing, it might be pretty easy to replace this valve with a slip coupling. It is especially easy if the tubing is long enough to flex it laterally to slide a coupling on. That is a straight copper coupling that doesn't have a ridge or other bumps in the middle. 


turn off the water
using sandpaper now to clean the tubing is easier than after the cut
remove the saddle valve
cut the tubing right at the pierced hole, either with a tubing cutter or hack saw
de-burr the cut, and clean the tubing of any crud from cutting
clean and flux the coupling
flex the two cut ends apart enough to slide coupling on
bring cut ends together again
center the coupling over the cut. Mark the tubing if necessary
solder