Old photographies coloring There is already a succesful question How did they color old black and white movies? on this site. Very general and correspondingly in general answered. Let me please narrow it down to just the photograpy (i.e. no time dimension) and more specified starting point:

What is the algorithm to color the greyscale HD picture given the colors at various points on greyscale original?

Note: Yes, I am aware that the original colors can't be restored any more.
This can be useful to you: Oscar Dalmau Cedeño, Mariano Rivera, Teresa Alarcón: Bayesian Scheme for Interactive Colourization, Recolourization and Image/Video Editing. Comput. Graph. Forum 29(8): 2372-2386, 2010.  ISSN: 0167-7055