What is Mode B on the transponder?
Mode B

SSR modes B and D are not used today, but mode B has been used in UK, Australia and some other countries for some time, and it was compliant with mode A:


  In reply to interrogation in Mode-A and Mode-B information of
  identity, which is set by the pilot, is sent to the ground
  interrogator.


Source: Radar Theory for Area/Approach Radar Controllers

Modes A and B are used with a squawk code of 12 bits, offering 4096 different identifiers. In both modes the code is selected on the SSR transponder panel, using 4 octal digits (0 to 7).


SSR transponder panel, source

For a transponder handling multiple modes, the correct mode must be selected in parallel of the squawk code.

The interrogator (SSR ground station) can request identifiers using one mode at a time, and receive answers from transponders set to that mode. Aircraft using code 1234 will only reply when interrogated using the mode their are currently set to.

All legacy modes, military and civil and mode S

SSR interrogation modes are identified by the timing between pulse P1 and P3 (P2 is used to reject interrogations from secondary SSR lobes). 

  

SSR modes identification, adapted from source

While mode A is compliant with military mode 3 (both often referred to as 3/A), mode B is purely civil. Its purpose is assumed to be an extension of the mode A, giving 4096 additional codes to ATC.

ICAO Doc 9684 which is the reference for SSR doesn't even mention modes B and D anymore.

Mode S specificities

To be exhaustive, let's mention Mode S visible at the bottom of the previous image.

Mode S (for select) allows to interrogate transponders one at a time when necessary, to prevent FRUIT. In addition of the squawk code, mode S transponders are given by authorities a unique (and quasi-permanent) 24-bit address. This address is used by SSR interrogators to specify which transponder is the recipient of the interrogation.

The address is sent in a datablock which modulation is not a pulse like P1 and P2, but a burst of carrier which phase is reverted when bit is 1 (see a detailed explanation in section Mode S of this answer).

This burst is led by a synchronization of phase reversal (SPR) burst, at the timing of legacy pulse P6, which role is to prepare the receiver to detect possible phase reversals events, hence to decode bits at the right time. In other words this is a clock synchronization. The data themselves can be a series of 56 bit when only the address is required, or 112 bits when the "interrogation" carries an ADS-B message too (with this message, we are entering a data link mode, this is where the ATC future lives).