Effect of One Sample Latency in Audio Application
One sample latency is pretty much unavoidable so this is fairly normal. Please note that you need to look at the total system level latency which includes DA and AD conversion. Many audio applications use Sigma Delta converters which have much higher latency than one sample, so your processing latency wouldn't make much of a difference.
The impact of latency depends very much on the specific application. One sample is very low and the only applications that would be impacted by this is anything with a feedback loop (active noise cancellation, active excursion control) and any application that has a parallel analog path (effects loop in an analog mixer for example)
It's not a problem for UI responsiveness, audio/video sync, play/sing/talk along, multi-speaker synchronization, etc.
The most latency-sensitive audio application is active noise cancellation and many dedicated ANC processors run at a significantly higher sample rate, just to reduce latency and increase bandwidth of the feedback loop.