Changing bulb in a pinhole downlight
As suspected by Ecnerwal and JACK, pulling down with moderate force eventually released the whole fitting, which was being held in place with sprung fins. It did result in a bit of the ceiling plaster coming down as dust in the process, but it seems to be the way it is expected to work. Pushing back up caused it to latch back in place.