What kind of electrical connector is this and how do I remove it?
It's a push-in connector. Basically the splice version of a "backstab". Not the best connector, to be sure.  

Do not cut.  Firmly grasp the splice connector, then steadily tug and twist the  -- 

-- Actually, forget that.  The wires in the ceiling are already too old, short and fragile.  Leave the connector alone and cut the modern, plastic black and white wires about 6" from the connector.  If you dislike this plan because you plan to reuse the old lamp, then cut it at the halfway point. 

Strip the ends of those wires you just cut, and wire-nut your new fixture to that.  

Also these types of connector are single-use, so once you drag a wire out of a void, you have damaged the spring and can't use it again. This is also true even if you use a proper release mechanism.  I would say "just cut" because it leaves the wire stub in the hole, preventing reuse; but wire length is precious and not to be squandered, that's why you have a short-wire problem today. 

Note that the above part of my answer violates Code.  The Code answer is the wires in the ceiling are simply too short.  Loosen the cable clamp and see if you can pull extra cable length in. If not, replace the entire cable in the ceiling/wall.  It must have 6" of length beyond the cable clamp, and 3" beyond the ceiling surface.