How is fuel weight calculated?
Fuel doesn't have the same density as water. We're not planning a real flight, so a quick and dirty fuel density figure from Wikipedia tells us:


  In performance calculations, airliner manufacturers use a density of jet fuel around 6.7 lb/USgal or 0.8 kg/l.


So if we take your figure of 320,000 litres (it's a bit unclear quite how much of that is actually usable for flight, but that's a subject for another question), that's about 560,000lbs of fuel or around 254,000kgs.

Which, indeed, is a lot. The fuel is spread throughout the wings (and the trim tank in the tail), which are designed to carry the weight. If you look at What is the weight budget of a fully loaded A380?, you'll see how the fuel capacity interacts with the other weight specifications for the aircraft:


  The maximum fuel weight is the difference between maximum ramp weight and zero fuel weight (which is 562,000−361,000=201,000kg)


Max take-off weight is a bit less than maximum ramp weight, but we'll handwave over taxi fuel for simplicity. The point here is that if you can fill the aircraft with cargo, it would be too heavy if you also filled it with fuel. If you want to take a full load of fuel, you'd have to reduce your payload to keep the total weight low enough. Working out the weight/fuel/range trade-offs for a given flight is what dispatchers and load planners do, using specialized software.