Is it expected that a naive path tracer takes many, many samples to converge?
Without explicit light sampling, I'd expect very slow convergence.

Any path that doesn't happen to hit the light source before being terminated will be useless (contributes zero). Since you have a small light source, the vast majority of paths will not hit it. It would be interesting to add some statistics to your tracer to see how many paths return zero vs nonzero. I'd guess you'll find something like 99–99.9% of them are zero, so you're only getting one useful sample out of every 100–1000 or so paths traced.

Naive path tracing is more reasonable with e.g. a sky light, which almost every path can be expected to hit.