How does an RS-latch work in the presence of propagation delays?
Yes, you are correct. The inputs must be held in a stable state until any change propagates through both gates, and the outputs may be inconsistent during that time.

We usually design circuits with edge-triggered flip-flops, and the corresponding notions are the setup time, hold time, and clock-to-Q delay. The setup and hold time specifications indicate a window of time around the edge of the clock signal where the input must remain stable. The clock-to-Q time is the delay from a clock edge until the output is valid.

When doing a full synchronous system design these requirements combine, along with the logic delay, to determine the minimum clock period. If you absolutely must have output signals that are always valid then you need to use an output register that loads them once they are guaranteed to have settled to their valid values.