How do you do a "Video Algorithm"?
This is effectively what every professional editing software in existence does at its core.  Video editing is always done non-destructively in any professional and most consumer contexts.  What that means is that when you are editing a project, you are not actually making changes to the source files, but rather making a series of decisions about what changes should be applied to get the desired output.  When it comes time to make the final rendered output, the instructions are run by the editing software on the original sources and the output is recorded to a new video file.

This series of instructions can be saved off in a variety of different formats such as EDLs (editing decision lists), Final Cut XML files (which are supported by more than just Final Cut) or any number of other formats that include the series of instructions that need to be applied.  These files can be used to move editing projects between completely different editing software.  

It is also possible to change the inputs that go in to these instructions.  As long as they are the same size and placement, then it should carry through consistently for different inputs.  This is why proxy workflows work.  In proxy workflows very large and difficult to work with files (such as 4k RAW video) are replaced with easy to work with files (such as much smaller compressed 1080p video).  This makes it easy to work through the edit and then only swap out with the high quality, difficult to work with footage at the end of the project.

The same principle can be applied to swapping photos or videos in to a particular spot in an overall video template.  All the various effects and edits will be applied to the new source just like they were to the old, you just need a render engine capable of doing the rendering and swap automatically.

There are some commercial options available for this, but much is done with custom solutions.  Depending on the complexity, it may also be possible to do it with an ffmpeg script that could be easily run by a command line, though ffmpeg would be considerably more manual to develop the template for.