Measuring AC Voltage from Floating Power Supply
Yes it does matter - if the common-mode voltage swing goes outside the isolated supply rails then bad things will happen - at best you won't get a meaningful conversion, at worst it could damage the ADC and op amp.  Somehow or another you should either bias or just connect the neutral to the 0V if you want meaningful results.  As has been pointed out, any circuit that uses AC mains is potentially hazardous, although sometimes there's no getting away from it (e.g. light dimmers).  You could couple the AC live and neutral via some Y-rated capacitors and earth the 0V of your +/-2.5V supply, or use a transformer (this seems like the best idea IMO).  Depending where you live, you may find that your neutral connection is in fact connected to ground at your fusebox.  If this is the case then you should see almost no voltage between neutral and earth unless there's a fault condition, so you could find that you can earth the 2.5V supply and you'll see very little voltage on neutral.  However, do put some protection (series resistor and zener/MOV to ground) between neutral and your ADC input.