Single-Ended Input operation of Differential ADC
Would this potentially work if I just used the output of the diff amp
  (1-5V) and ground into the +in and -in pins? If not, why not?


What you have said above makes no sense - with both input pins grounded, the device won't work. Probably what you want is reassurance that the device can work single-ended (as per your question title. The data sheet quotes this: -


  For single-ended operation, the > non-inverting input (+IN) of the
  ADC141S626 can be driven with a signal that has a peak-to-peak range
  that is equal to or less than (2x VREF). The inverting input (−IN)
  should be biased at a stable VCM that is halfway between these maximum
  and minimum values. In order to utilize the entire dynamic range of
  the ADC141S626, VREF is limited to VA / 2. This allows +IN a maximum
  swing range of ground to VA. Figure 40 shows the ADC141S626 being
  driven by a full-scale single-ended source.


This is on page 16.