Rigid Body Dynamic Animation Problem
This approach took a shortcut in that the individual cubes forming the letter 'b' were all joined as one object.  The explode modifier was used to break it up.  This avoids the physics problem.

The shards that result from the explosion modifier have been rendered invisible in favour of cube shaped particles to maintain the illusion that the cube-stack is what is breaking up.

The explosion modifier and the original 'b' was, at the appropriate frame, switched to a second emitter, (3D &amp;'B' shaped) that emits the same sized and coloured cubes as used to form the letter.



Note - there is an invisible catch plane placed just above the visible ground in order to catch the cubes as they fall.

When particle objects sit on the ground, they do so at their origin points.  This would usually be at their center, so they 'sink into the mud'.  To keep their feet dry this catch plane is set up with the required physics and a little higher than the visible ground which becomes just a prop.

-======================

After some two days of experimentation and with your apparent leaning to use the original cubes for the break up, I came up with this...

To use the stacked cubes in your uploaded Blend file, the timeline was set to frame 1, then each and every cube had it's physics reset by switching the Rigid body button off then on again.  That restored the physics.  

As there are some 1040+ cubes only a quarter were actually processed this way.

In order to hold them mid-air, the Animation box is ticked and Dynamic box unticked, and both keyframed, again for each and every cube.

At the impact frame these two tick-boxes have to be keyframed again but in reverse.  The cubes then fall to the ground.  

(ignore the green bar)



Granted it looks good but of the two methods, the Explode modifier gives far better control over the break-up appearance, directional bias, and general cube behavior.  With the letter solid it's also a simple matter to manually animate it's lurch on impact.

-==============-

You can skip the reversing tick boxes step, by rendering the cubes dropping from frame 1.  Animate a solid duplicate of the letter in a separate video and mix the two in the VSE.