Create a power supply that accepts 3.7 V - 5 V and outputs 5 V
No, it will not work if you connect the 5V charger to it. The module in your post is a boost converter, which means the input voltage has to be lower than the output voltage.

If you connect the lithium battery to it fully charged (4.2V) you can use the module, since 4.2V is less than 5V.

You don't mention if you want your system to be automated or illustrate how you are connecting the lithium-ion battery and the 5V charger to the load. You could implement a manual toggle switch to switch between the lithium and boost converter to the 5V charger. 

I would also recommend you use something like this: https://www.diyelectronics.co.za/store/solar-management-modules/2160-solar-power-manager-5v.html

It accepts your 5V charger through a micro-USB connector that charges your lithium battery and also outputs a regulated 5V DC supply. In addition, this solar management module also acts as an uninterruptable power supply (UPS). When the 5V charger is disconnected the battery automatically kicks in to still supply your load with a continuous 5V regulated supply. Please note refer to the documents provided in the URL above on how to connect the peripherals to the solar module.