How did aircraft flying in formation use flares to communicate with each other?
This break down of a B24 Missions seems to outline the usage of flare on a mission well:

Assembly
After takeoff, a pilot usually kept on a straight course for about two
minutes, in part because the B-24 was not very maneuverable until it
had gained speed and altitude. The pilot then climbed at a
predetermined rate (about 300 feet per minute at 150 mph Indicated Air
Speed) to the assembly area assigned to the 392nd. A ship, usually
41-23689, Minerva, orbited a radio beacon at the prescribed altitude and fired designated colored flares to signal 392nd BG a/c. The
specified assembly altitude was based on cloud conditions. Clear skies
meant assembly could be at 5,000 to 10,000 feet. Often, though, pilots
had to climb to 20,000 feet or higher to get above the clouds, relying
on instruments to maintain headings, speeds and timed legs.

When on the actual mission...

Mission Routes
Routes were scheduled to avoid major flak areas as much as possible
and to mislead enemy fighters about the intended target. Each route,
then, included several course changes until the Initial Point (IP) was
reached. The IP was usually an unmistakable landmark, both visually
and on radar. At the IP, the lead plane alerted the formation to turn
toward the target by signal flares or by opening its bomb bay doors.
All other planes promptly opened their own bomb bays and made the turn
while simultaneously aligning in trail. On the bomb run, Groups were
usually two to five miles apart.

Then on the way home:

Wendling
By this time, the formation was down to a few hundred feet. While in the landing pattern circuit, the engineer confirmed the
wheels were down and locked. Planes with wounded aboard or severe
battle damage fired two red flares and landed first.

A few sources cite the M8 Flare gun being the one that was likely used during WWII. There is a discussion that covers some of the colors and cartridges here.

This book which talks about flare usage quite a bit, discusses a lot of the ground &lt;-&gt; air communication done with flares:

The increasing amount of airfield bombardment led to the establishment
of false aerodromes. These false fields gave the appearance of active
aerodromes by day and night in the hope that enemy bombers would be
fooled into releasing their bomb loads a mile or two from the real
airfields. At night, active fields remained concealed in darkness
until an airplane was properly identified through flare signals.

It goes on to say:

Signaling devices were necessary for communications with the ground.
The flare gun provided the most basic means of signaling. The Allies
used the Chobert, more commonly known as the Very pistol. British and
American night squadrons used three different colored Very cartridges,
red, white, and green. Each cartridge had a different milling along
its metal base flange so that aviators could quickly feel for the
correct shell in the dark.

Some flares were also used for ground illumination

Illuminating flares were developed for target acquisition and landing
assistance. One of the most important was the Holt wingtip landing
flare. Useful in an emergency or under inadequate field lighting
conditions, the Holt flare was ignited electrically (by dry cell
battery) from a push button in the cockpit. Typically the flare would
burn for a maximum of two minutes at up to 20,000 candlepower.