How do I allow users to just click a marker on an OpenLayers map to go directly to the node? I am using the OpenLayers module and it appears that I can add a popup and a tooltip behaviour if a users clicks or hovers over a marker on a map, but I only need users to be able to click the marker to go to the node (no popups). Is there a config I am missing to do this?
Usually jumping to new page just by clicking object is not what users want, as they need to be sure first that the clicked object is correct one, this is why there is no built-in function and you do not see this much in the wild.

OpenLayers module has behaviours for this kind of actions. It seems that the core module does not have what you expect, and nobody has done it yet, so no easy configuration option :( Here is the guide how to create new behaviours from scratch. I would use Popup behavior as sample.