How can I regulate the voltage of a Li-ion battery with the least drop out voltage?
Use a buck boost regulator - it would continue to produce 3.3 volts all the way down from probably over 5 volts to possibly 2.5 volts. Obviously you have to ensure that the Lithium battery doesn't sink too low or it will become damaged but that's another problem that is solved by using a comparator and a regulator shut-down circuit. Here's an idea: -



Or....



Or...



Or maybe try using their search engine here