Is the front center pull brake supposed to be free floating?
There are two types of caliper brakes, the side-pull and the center-pull. You have the center-pull but are used to side-pulls.

Side-Pull

You're used to the side-pull. There, the cable comes to one side and squeezes the arms together. Because it's not symmetrical, it uses the pressure of the brake shoes when full-squeezed to center itself around the single mounting pivot. By design, the entire brake can pivot around that mounting bolt:

[



Center-Pull

Your photo shows a center-pull caliper brake. It uses a straddle cable that goes to a yoke (both of these are missing in your example but can be seen below). The straddle/yoke design tends to self-center -- at least if the straddle cable isn't too deformed with age. The brake arms themselves don't pivot around the mounting bolt, but instead pivot on the pivots on the left and right side of the mounting assembly (they had red washers in the photo below). Because of the dual-pivots there is no need for the mounting bolt itself to pivot as part of the brake's normal operation. You can loosen the mounting bolt to recenter by eye and then tighten it back up, but it doesn't float around the center the same way as a side-pull:

[

Both images from wikimedia commons.

A footnote...

Furthermore, I should note that these brake designs from the 1970s were built with much larger brake pad clearances than contemporary brakes. Your shoes might float a good 3-7mm above the rim on either side. Compare that to contemporary brakes which have very close rim clearances, just 1-2 mm. This means that you don't have to be as precise centering with these older brakes -- and you should allow more pad clearance than you might be used to if you have a contemporary bicycle as well. And you'll never get these old brakes to brake as well as a contemporary brake, they will always be weaker and more rubbery in feel.