How many significant digits should I use for simulating aircraft equations of motion?
From your description, it's not an issue with the aerodynamic coefficients. If as you have described previously that you're modeling F-18, then the airframe should be unstable longitudinally. A 1e-10 deviation (assuming it applies to body rates, body speeds, etc. as well) is well-within typical trim error bound; in fact, a deviation is unavoidable when dealing with computational floating point. But if it's longitudinally unstable, any deviation will magnify over time exponentially.