Solid state relay Crydom D1D07 closes and remains closed
You are not using the input as specified.
Minimum Turn-On Voltage (6) 3.5 VDC
Must Turn-Off Voltage 1 VDC    ( your circuit is 5-3.4 = 1.6V too high )
Minimum Input Current (for on-state) 10 mA   (unknown)
Maximum Input Current 15 mA    ( your value is unknown)

You can specify your options available to you for driving or someone can tell you how to do it.
This is the input behaviour.




simulate this circuit &ndash; Schematic created using CircuitLab
These are a couple of options of many more.

This is exactly what I expect  Crydom has on their inside design.
Since the current limiter needs 1.5V for IR LED and + 2V for current limiter, This is why you see 3.5V on an open circuit input but there must be enough leakage to trigger the MOSFET.
Using the pull-up resistor causes the input voltage difference to be zero ( both at 5V. ) and this draws little or no current and turns off the IR LED..