Any reliability concern if I don't add UVLO protection circuit?
I have seen this happen many times during early product development. Usually nothing will permanently break (I've never seen reset loops cause a hardware failure). But there is kind of an understanding that it is undesirable.

Most systems I have seen have battery voltage monitoring by way of an ADC input on the processor. Also, most modern systems have the ability to shutdown under software control. Because of this, there is no need for a hardware UVLO. Here is how a typical system works. The voltages vary.

During operation, if the system voltage reaches 3.5V, you warn the user somehow. When it drops to 3.4V, you unconditionally shut down the system into its lowest power state. (Voltages may vary in your system).

During boot up, very early, before you do anything that would cause a power spike, or be detectable by the user (like turn on a backlit display or play a sound or whatever), you check the battery voltage. If it is below say 3.55V, you do not fully boot up. Return instead to the low-power state. The operation shutdown voltage is LOWER than the minimum boot-up voltage. This way, if the unit shuts down due to low voltage, the user cannot just power it back on.

Also, if it does somehow get into a reboot loop despite your efforts, it will be looping before the battery voltage check which means it will be before any user noticeable events occur. So, the user will never know about the reboot loop. Better not to loop, but if you do, then it is better if the user doesn't know.

Here is another point. If you use hardware UVLO, you have to fully consider what triggers it, how it is released, and what will happen when it is released. For one thing, the UVLO needs to have some hysteresis. If it is triggered at 3.4V, it shouldn't release until a somewhat higher voltage like 3.5V or even more. Otherwise there is a chance you will get your reset loop anyway.

The reason is that the battery voltage sags under system load, then it rebounds when the load is cut off. If the UVLO has no hysteresis, it will just bounce back and forth in a loop as it goes over and under the threshold. But with hysteresis, once it cuts out, it will need a somewhat higher voltage before it reconnects the load. So no bouncing.