How to chain two recessed lights together?
You've got it.  Just make sure that in the light fixtures you use all black in one connector, all white in the second and all bare wire in the third (from your hand-drawn wiring, it looks like that's how you plan to do it).  That way, the lights are wired in parallel so that they both get the full mains voltage across them when you turn on the switch.

Also make sure that the switch is connected to the live (black in this case) wires in the two cables, which means that no electricity can reach the light fixture when the switch is off (which could save your life if you accidentally touch the live terminal in the fixture when changing a bulb).