transimpedance op-amp wrong and random output
A photodiode is meant to be used with a bias voltage. This is typically achieved by using the "virtual short-circuit" of the input pins of the opamp in the TIA.

Like so:







simulate this circuit &ndash; Schematic created using CircuitLab

The best way to improve the output signal strength is to get more light on the photodiode.
You can also try increasing the feedback resistance at the expense of stability. If stability is an issue, you can add a feedback capacitance to make it stable again, unfortunately at the expense of bandwidth.