What made this small light weight bee hive-like structure?
This is a mantis (Mantodea) egg sack, or ootheca. The female mantis makes it in the fall and it will remain over winter until (juvenile) nymph mantises emerge from it in the spring.

The following site introduces how to raise mantises from eggs but also has some general background information including on how seasonal temperature changes can affect the ootheca:

https://www.keepinginsects.com/praying-mantis/caring-for-mantis-eggs/