How to wire this back up?
Ok, so you will want to connect the white wires from the yellow, red, green, and the fixture.  Connect all the black wires together except the fixture.

Now the only two wires you have left are the white wire from the switch and the black wire of the fixture.  The white wire will be the "switched hot" and needs to be identified by wrapping a strip of black tape around it.  You don't need to cover up all the white, you just need to have an obvious band of black to mark that wire as hot.

Connect that switched hot white wire to the black wire of the fixture and everything should work fine.  Note that the larger bundles of wires will need to use larger wire nuts (typically red).  Don't try to force more wires into a small yellow wire nut.

Also, I hope the zip ties in the back of that box are not holding the box in place.  The box shouldn't wiggle and should be very securely mounted with strong screws if it's going to hold a fan.