Am I using AC relays correctly?
Ok, assuming we're dealing with AC here...

As others have mentioned, you're probably going to want to use relays for this. AC-IN/AC-OUT Solid State Relays are cheap and simple to use. For example this one: https://smile.amazon.com/SSR-25AA-80-250V-24V-380V-Machinery-Control/dp/B01MZ2B0LA

Just hook up a relay in parallel with your controller, so that either the controller or the relay can turn on the RED light. Then hook up the input of the relay to your yellow light signal.

See terrible paint edit below: