This white wire is hot right?
Having a white that is hot at a switch is called a switch leg and code required the white wire to be always hot so it would not be mistaken for a neutral if it was on the switched side this has been code for decades.
Put a wire nut on the black to white and there will be no more switch interference with the receptacle.
Note if there is not a light in the room it is a requirement to have a switched receptacle so taking out the switch may violate code.