Analog input scaling before ADC sampling
Well, the schematic in principle is correct, and from theory it should work. HOWEVER you didn't check your common mode ranges (both input and output ones).
The TL JFET amplifiers are quite old (still good, however) and designed for dual power supply (typically ±15V, they work with less but not with ±5V).
From the datasheet you'll see that it's input CM range, when powered at ±15V is guaranteed to be only ±11V. So, approximately, if you power it at 12V single supply, you CM range will be 4 to 8V. Clearly you signal in the range 0 to 4V cannot per processed properly.
In the same way, the guaranteed output swing would be ±12V which translates to 3V to 9V. Most of your useful signal range is now out of the guaranteed linear zone of the amplifier.
Luckily you only need to use another opamp and they almost have the same identical pinout. Look for a single supply amp that can go down to 0V both in input and output. For a popular part the LM358 when powered from 12V should fit the bill (unless you really need to go lower than about 5mV, then you need another amp and begin worrying about offset)
EDIT: I read that you had issues with LM358 too. And are using a chemical sensor.
Well, these things tend have horrible output impedances, they get loaded just looking at them. Probably they suggested you the TL since it has a huge input impedance. First of all, check if you circuit is linear giving it a good input signal (from a power supply, from example). Once this is fine is time to attack the sensor itself.
You could try with another FET input amplifier, a CMOS amplifier, an instrumentation amplifier and as last resort, they made special amplifiers just for electrochemical sensors. Well, the last resort is actually a custom designed discrete amplifier…
It could help know the kind of sensor, they have wildly different requirements: some need only a huge input impedance, some have biasing current issues. Often there is a reference schematic to get you started in the sensor manual.