What would happen if you connect a solar panel to a buck-boost converter?
I think i am missing something, but here are my two cents:

The buck/boost will operate on the input voltage given by the solar panel. The internal switch control will determine if it works as buck or as boost (obviously, if the solar voltage is lower than 5V it is a boost, if it is higher it is a buck). 

Depending on the size of the load, the solar (input) voltage may drop. If the voltage gets too low for the boost to handle, the output voltage will also drop. 

Solar panels shouldn't be excessive loaded, because their efficiency drops in that case. Because of that there are special buck/boost ICs which provide point of load control for the solar panel (BQ24650 for example). In essence they lower their output power to not overly load the solar panel.