Rendering issue (JPEG artifact on characters mouth/non-transparent background) how do I fix this?
are you using a jpeg as the mouth's texture, or as render output?

Anyway, jpeg is a really harsh lossy picture format. There is always a quite noticeable loss with it.

I recommend you read the Blender Manual's page about the Supported Graphics Formats. Starting by this very simple yet quite true Hint:


  If you are not interested in technical details, a good rule of thumb for selecting output formats for your project is:
  
  Use OpenEXR
  if you intend to do compositing or color grading on these images.
  Use PNG
  if you intend on-screen output or encoding into multiple video formats.
  Use JPEG
  for on-screen output where file size is a concern and quality loss is acceptable.
  
  All these formats support compression which can be important when rendering out animations.


TL;DR : don't use jpeg if you care about quality.