Connect to a serial output device with 12V power input with 5V USB-TO-UART
I understood that this is not an issue with the connection scheme, but the problem with DG600F itself.

It is just a fault device. Found another DG600F for a test and it is working great with the same settings.

And I finally managed to connect to NV10 USB+ device using the same connection with CP2102 (but with an addition of TX channel). And I've used SSP protocol, which works through a serial port.