Which of sodium or lithium will be more reactive with oxygen?
Reactivity of Group 1 and 2 elements increases as you go down the periodic table.
So sodium is more reactive than lithium. Sodium will react with oxygen forming $\ce{Na2O}$ (sodium oxide). Lithium forms lithium oxide $\ce{(Li2O)}$.

Basically, there are general trends (reactivity with acids, air, etc.) down a group or along a period.