Will mixing hydrogen and oxygen without a spark result in combustion?
For hydrogen to burn or explode in air, its concentration needs to be at least 4%.

Unless you're using very high power levels, you're unlikely to reach that concentration in the room's air. You could ignite bubbles of hydrogen as soon as they pop at the surface, but if you don't, the hydrogen (with about 1/8 the density of air) will rise quickly, diffusing into the air as it goes.

If you're doing this using a battery, or a small line-operated power supply, there's very little danger of explosion.