Is there a risk of 18650 li-ion battery explosion in a low current application?
No one here will give you a safety guarantee!

There simply are no guarantees that can be given.

However what you describe is far from the typical scenario where lithium based cells cause safety issues. Safety issues generally occur:


when fast charging
when charging doesn't stop when it should have
the cells are mistreated
there is too much mechanical stress on the cells (Samsung Galaxy Note 7 debacle)


If you use a safe charging solution (a dedicated chip which stops charging when needed) and a battery protection solution (like the DW01 you propose) then the risk of any incident happening should be fairly minimal.

Personally I would not consider the risk large enough to worry about.

Oh and a small tip: If you do decide to go for the NiMh cells instead, be sure to use "Low Self Discharge" cells for that as most "normal" high capacity NiMh cells have a very high self discharge. Some of them leak themselves empty within a month.