Why do fresh mandarin oranges get very bitter when baked?
The white, pithy part is a source of the bitterness. Either peel more thoroughly, or trim off with a knife.

The pulp of some oranges (of the navel variety), can turn bitter when exposed to air.
http://www.preparedpantry.com/CitrusQuickGuide.htm