Smoothing 2D points (scattered) in real-time (online smoothing)
Since you want a physics like behavior, you can use a simple physics model. Note all arrays below describe properties of the current state of your dynamics and therefore have the same shape (1, 2).

define forces
a:attraction = (k - p) * scaler

velocity
v:velocity = v + a

positions
p:current position = p + v
k:new dl key point = whatever your dl outputs


You output p to the user. Note, If you want a more natural motion, you can play with the scaler or add additional forces (like a) to v.
Also, to get all points, concatenate p to ps where ps.shape = (n, 2).
I have to work on task of using hand keypoints as pointer (or touchless mouse).
The main problem here is that the (deep learning) hand keypoints are not perfect (sometime under varied of light, skin colors), therefore the chosen key point are scattering, not smoothly moving like the real mouse we use.
How can I smooth them online (in real-time). Not the solution given array of 2D points and then we smooth on this array. This is the case of new point get in one by one and we have to correct them immediately! to avoid user suffer the scattering mouse.
I'm using opencv and python. Please be nice since I'm new to Computer Vision.
Thanks