Should I add an undervoltage protection for lithium battery-powered projects?
It depends on whether you have a cell or a pack.  A cell (usually referred to as a battery) is only the element itself.  A pack contains a cell(s) but will have additional protection, usually including undervoltage protection as well as thermal sensors and sometimes charge management/coulomb counting circuitry.