An algorithm to find the closest match between 2 arrays of RGB pixel tuples
You can't satisfy both goals.  Suppose $A=[10,20,30]$ and $B=[10,20,30]$; then you presumably want the modified $B$ to be $[10,20,30]$.  Now suppose $A=[10,20,30]$ and $B=[10,30,20]$; then you presumably want the modified $B$ to be $[10,20,30]$.  The original A and the modified B are the same in both cases, so there is no way to tell from them what the original $B$ was (both remain a possibility).