What value of resistance to put between the output of a 74hc595 and the LEDs?
You are correct that a lower resistance would offer higher brightness. The more current through the LED, the brighter it will be. The current is calculated by dividing the voltage across the resistor by resistance value. The voltage across the resistor would be your 5V - the voltage drop across the LED. this often varies by color. A RED LED might have a 1.2V drop, and GREEN or BLUE would be higher. So to get the same relative brightness in different color LEDS, you will often see a lower resistance used, to get the same current.

Lets say you have a 20mA Red LED, with a 1.2V drop. So (5 - 1.2)/.020 = 190 ohms. If the LED required 10mA, that would be 380 ohms.

Now you DO need to look at the data sheet for the chip driving the LED, and make sure it is up to the task. Can a pin source or sink 10mA? 20mA?