Speech driven facial animation Is there any way to animate face and lip sync with audio input using machine learning algorithms in Blender ? I mean how can I controll the rigs and give them a value provided by python script for. The idea is to do a dynamic lip synchronization.
I appreciate any help ! thank you
This is a wide area, but I did have one thought about it myself:
The eSpeak speech synthesizer can output phoenomes instead of sound. You could make something to line up shape key keyframes based on this, although timing might be an issue. You might look into either the eSpeak Python api for synthesis timing check on the words, or even try the Descript project: they might provide synchronization data for the layman.