What is the difference between a GFCI ungrounded circuit and a grounded circuit?
Protection, in theory, no.  The GFCI takes care of the “killing” risk of an outlet, at least, it does as well as grounding does.
However actual grounding does something GFCI does not: provides a place for natural electricity to go, notably ESD from shuffling your feet across the carpet in winter.  With a ground wire, that simply goes into the building’s grounding electrode system.
Without a ground wire, it finds a place the equipment is not insulated for 20,000 volts (many such places), and leaps to the hot or neutral wire, then goes through equipment if necessary to the neutral wire, then via the house’s neutral-ground equipotential bond to grounding electrode.  The equipment might not enjoy this too much.