How does the efficiency of a piston aircraft change with altitude?
You are right, flying higher will be more efficient. However, the inverse relation between thrust and speed in propeller aircraft means that the available speed range is much smaller, and thus the possible altitudes are more restricted. Simply put, the possible gains are smaller.

If you look at the thermodynamic cycle of a turbo-compound engine, it looks almost like that of a jet. Still, the high bypass ratio translates into a lower optimum flight speed and the lower power-to-weight ratio usual for piston aircraft limits the maximum altitude and makes them more susceptible to adverse winds. Also, turbo-supercharging, which is much more efficient that supercharging, was developed to a practical solution only a few years before gas turbines were introduced, so pre-WW II piston engines relied on the less efficient supercharger for boosting power, reducing the benefit from flying higher yet again.

Why historic flights were performed at lower altitudes has probably more to do with the low ceiling of the initially heavy airplane and the lack of supercharging than with a deliberate choice by the crew. Maybe the lack of cabin heating or de-icing equipment, or even an open cockpit, made lower, warmer altitudes also preferable. Another reason might be the need for terrestrial navigation in the absence of today's infrastructure of radio beacons and GPS satellites. That flying higher translates into better range was known as early as WW I.