multiplexing 256 led with atmega128 not working properly
Second version is not correct because you are sharing resistor between multiple LEDs in the same moment. The current flowing through single resistor is a sum of currents through Ax, Bx, Cx and Ex (for any fixed x between 0 and 7).

Let's assume F0 is grounded in some moment of time-multiplex. Then, by setting A0, B0, C0 and E0 high you light top left LED in each matrix respectively. If you assume voltage drop over the LED is approximately constant, you have same voltage over resistor and same current through resistor regardless how many of these four LEDs are lit. But this same current is divided between one, two, three or four LEDs, therefore resulting in different brightness depending on how many matrices has this particular position lit.

EDIT: I assume you are running your multiplexer such that you drive low just one of Fx outputs at time and set all Ax, Bx, Cx and Ex outputs accordingly. Should you loop over all Ax, Bx, Cx and Ex such that only one of these 32 outputs is high at each single moment, then the problem described above does not apply.