What is the correct current in the datasheet for the LED-Matrix?
So if I wanted to drive an entire column (5 LEDs) I would need 5 times
  the forward current.


There are 5 columns and 7 rows intersecting with each column: -



So if you want to drive a full column at 20 mA per LED the column current in total will be 140 mA.