Flashing leds from bicycle tail light
The "hard black centre" of the button is carbon impregnated rubber.  It creates a connection between the PCB traces that are exposed (no green solder mask) on the right of the PCB.  This is a common cheap way of making buttons, and is found everywhere in small / cheap devices, such as your TV remote control.

As has already been mentioned, the fourth component you didn't mention is the black blob of epoxy with a microchip underneath it.  This is the main control of the circuit - the "brains" if you will - that tells the LEDs just what to do and when to do it.

This "chip on board" is incredibly cheap to manufacture in huge quantities, and you'll usually find it in such things as calculators, watches, kids hand-held games, etc.  It can be quite fragile though, and prone to breaking if the PCB is flexed, which might be what has happened in your case, especially as the PCB seems to be cheap SRBP (Synthetic Resin Bonded Paper), which is quite flexible as PCBs go.

It's also worth noting that all these components, including the LEDs, are sensitive to the polarity of the power.  Connecting the batteries the wrong way will most definitely not work, and in a worst case scenario can actually kill the chip, especially in cheap devices like this where there is no money spent on "hardening" the circuit against misuse.