Square wave generator has nigh-useless output
The scope traces are typical of ringing observed when a signal is operated without the designed load.

A simple experiment for the purposes of getting a usable oscilloscope trace would be to add a 50 Ohm resistor to the output end of the cable, and connect the oscilloscope probe across that resistor. A wire-wound resistor would not be ideal, as those come with their characteristic inductance. A carbon resistor is probably easiest to source and acceptable for the experiment.

While the resistor may not have the expected capacitance or inductance of the actual load the device was designed for, this should still reduce or eliminate the signal voltage overshoots you see in the signal trace.