Step-up or series lithium cells?
I don't have enough reputation to add a comment, but I think it is important to note here that the form factor alone doesn't tell you either the nominal voltage, capacity or the safe charging voltage of a cell - these metrics vary between chemistries.

A lithium iron phosphate (LiFePO4) cell has a nominal voltage of 3.3V and must not be overcharged above 3.65V - at 4.5V it is in danger of violent rupture, disassembly and fire - despite being the relatively "safe" li-ion chemistry.

Lithium Cobalt Oxide (LiCO2) cells can be charged up to 4.7V, though this is frequently limited to 4.2V as there are diminishing returns but increased hazards above this voltage.

The low voltage cutoff presents other dangers - LiFePO4 cells can safely be discharged down to 2 or 2.5V, while for LiCO2 it is 3V. Ignoring these limits causes permanent damage which can result in dangerous failures at a future time in the life of the cell.

I've given two examples, but there are a number of different chemistries within the li-ion family, and it is the chemistry itself which is of utmost importance when deciding on safe operating conditions.

Unfortunately the chemistry is not always obviously labelled, which is why it is imperative that you use the datasheet to identify the Safe Operating Areas of the specific cell you have - whatever the formfactor (18650 is just any cell 18mm wide by 650mm long). 

From these key characteristics (nominal, max, min voltages, temperature range, capacity) the chemistry can usually be deduced where it is not overtly stated, but either way a Material Safety Data Sheet ought to be available for any cells worth using in your design.