Which breaker should trip with a ground fault with a subpanel?
Looking at a diagram, is often a good way to understand a problem. Below is a simple diagram that shows the fault current path.

You can see that the fault current will flow through all the breakers, and return to the source (the transformer) along the grounding conductor.
You might be inclined to think that the fault current will be extremely high (tens of thousands of amperes). However, due to the fact that wires have resistance, the current might be surprisingly low.
Without knowing the exact length and size of each wire, it's not possible to approximate the resistance.  If you did have that information, you could calculate the resistance.  With that, the voltage, and Ohm's law, you could calculate the fault current.
For this example, we'll assume the fault current is less than 100 amperes.
Because the current is not above the instantaneous trip level of any of the breakers, the short-circuit protection of the breakers will not trip.  However, circuit breakers also have thermal protection, which opens the circuit based on overheating  caused by current flow (overcurrent).
Each breaker will trip according to it's trip curve, based on multiples of current over the rated current of the breaker.  Basically, as the fault current flows, the thermal protection device in each breaker begins to heat up (along with all the wiring in the circuit).  Obviously the larger breakers (100 &amp; 200 in the diagram) can handle more heat (current), so they're going to be able to handle the less than 100 ampere fault current.
Since the fault current is higher than the rated current of the smaller breaker (20 in the diagram), the thermal protection device will begin to overheat.  With such a high current flowing through it, the device will likely open within a few seconds.  However, if the current was lower, it could take much longer to trip (even minutes).
If we take another example, where we've figured the fault current at 150 amperes. Even though the current is now higher than the rated current on three of the four breakers, the smaller breaker is likely still going to trip first.  This is because the time before the thermal protection of a breaker trips, is based on the the amount of fault current above the rated current.
The fault current is only 1.5 times higher than the larger breaker, but 7.5 times higher than the smaller breaker.  Because of this, the smaller breaker will trip sooner.  Thermal protection is designed this way, so that loads can draw over the rated current, but only for a limited amount of time. This allows things like motors to start, without tripping the breaker.
In most real world applications, the smaller breaker will trip first.  If the resistance of the fault circuit was low, it's possible for the fault current to be above the instantaneous trip level of all the breakers.  In that case, the first breaker (main) will likely trip first.
If any of the breakers are GFCI breakers, and the fault is to ground, the GFCI breaker will trip first.

More realistic example
This example will use the diagram above, but will attempt to estimate a more realistic fault current.  We'll say that there's 100' of 3/0 CU. wire from the pole to the main service panel lugs.  3/0 CU. is 0.0000766 ohms/ft., so that's 0.00766 ohms.
0.0000766 * 100' = 0.0076 ohms
Next there's 50' of 3 AWG CU. from the main panel feeder breaker, to the second panel main lugs. #3 CU. is 0.000245 ohms/ft., so that's 0.01225 ohms.
0.000245 * 50' = 0.01225 ohms
Next there's 25' of 12 AWG CU. from the second panel breaker, out to the fault.  #12 CU. is 0.00193 ohms/ft., for a total of 0.04825 ohms.
0.00193 * 25 = 0.4825 ohms
Now that we've reached the fault, the current has to follow back along the grounding conductor. The grounding conductor is made up of 25' of 12 AWG CU., 50' of 8 AWG CU., and 100' of 3/0 CU. back to the pole.
0.00193 * 25' = 0.04825 ohms
0.000764 * 50' = 0.0382 ohms
0.0000766 * 100' = 0.00766 ohms
Totaling up all the resistances, we end up with 0.16227 ohms.
0.00766 + 0.01225 + 0.04825 + 0.04825 + 0.0382 + 0.00766 = 0.16227
Using Ohm's Law, the fault current can easily be calculated using the formula current = voltage / resistance (I=E/R).
120 volts / 0.16227 ohms = 739.5 amperes
739.5 amperes of fault current.
That's 3.695 times the 200 ampere breaker, which according to a random trip curve I looked up, should trip the breaker between 8-25 seconds. It's 7.395 times the 100 ampere breakers, which would trip between 2-7 seconds.  It's about 37 times the 20 ampere breaker, which is likely beyond the instantaneous trip current level.
In this example, the 20 ampere breaker will trip first (unless any of the other breakers are GFCI breakers).