Flag mashup generator Part 2 of the challenge is here

Flags Mashup Bot is a small Twitter bot that generates a new flag based on two random flags and tweets the result every couple of minutes. 

Task

Your task is to replicate what the bot does by writing a script or a function based on the following criteria:


The input is any two lossless images composed of at least two, but no more than eight colours.
To mix up the flags the following algorithm has to be used:


Take the first image
For every colour in the first image assign a colour randomly from the second image
Each colour from the second image has to be used at least once


Unless there are less colours in the first image than colours in the second.
You can choose randomly which colours to drop from the second image in this case.

Each colour from the first image has to be replaced with a different colour


Unless there are not enough colours in the second image.
You can choose randomly which colours will be used multiple times in this case.
However you need to make sure none of the colours are used more than any other colour by more than one
example: if you need to assign 2 colours to a flag of 5 colours you'll need to use one of the colour 3 times, while the other one 2 times. You cannot mix them 1 &amp; 4

If a colour is present in both images you cannot assign that colour with itself you'll need to pick another one

The output is the resulting image


Examples

UK &amp; Hungary



Notes: Both flags have three colours, however both use red and white, so those have to be replaced, hence the first invalid example. The second one doesn't use all three colours, and the third one uses non-existent colours.

South Africa &amp; Sweden



Notes: South Africa's flag has 6 colours, while Sweden's have 2, meaning each of the two colours have to be used exactly 3 times. The invalid ones use the colours in a way which is not compatible with this constraint. Note that although both flags use the colour yellow they are actually not the same shade so are considered different

Poland &amp; Poland



Notes: The only valid solution is switching up the two colours.

Notes


The two images don't need to have the same dimensions
The colour list can be different for the two images, so in total there might be 16 colours
The input can be either a lossless image file (example GIF, PNG, BMP, SVG, etc.), or (especially in case of functions) a two dimensional bitmap array holding the colours of the image in any reasonable format. Any native way of storing a bitmap in your language is also fine.
For both input and output images a two dimensional array holding ASCII characters, where different characters count as different colours, counts as a reasonable format, but solutions that work on "real" bitmaps are preferred
The output and input formats don't need to match
I think the colour replacement rules won't prohibit it to work for any valid two input images, but if yes your program doesn't need to work for them
Colours are considered the same only if they are exactly the same based on their RGB value. E.g. #FF0000 and #FF0001 are considered two different colours.
Standard loopholes, as usual, are prohibited


code-golf, so shortest code by byte count wins and will be accepted. If possible please include example a set of input &amp; output images with your submission
JavaScript (Node.js),  160  159 bytes
I/O format: matrix of 24-bit integers.
Takes input as (a)(b).

a=&gt;F=b=&gt;(g=a=&gt;[...new Set(a.flat())].sort(_=&gt;Math.random()-.5))(a).every((c,i)=&gt;c^=F[c]=B[i%B.length],B=g(b).slice(-g(a).length))?a.map(r=&gt;r.map(c=&gt;F[c])):F(b)

Try it online!
Commented
Helper function
The helper function \$g\$ takes a matrix as input and returns a shuffled list of the distinct values it holds.
g = a =&gt;
  [...new Set(a.flat())]
  .sort(_ =&gt; Math.random() - .5)

Main code
a =&gt; F = b =&gt;                 // a[] = first image, b[] = second image
  g(a).every((c, i) =&gt;        // for each color c at position i in the list of shuffled
                              // colors of the first image:
    c ^=                      //   make sure that c is different from the
      F[c] = B[i % B.length], //     substitute color F[c] picked from the second image
    B =                       //   define B[] as:
      g(b)                    //     the list of shuffled colors from the second image
      .slice(-g(a).length)    //     resized to the number of colors in the first image
  ) ?                         // end of every(); if truthy:
    a.map(r =&gt;                //   return a[] with each original color
      r.map(c =&gt; F[c])        //   replaced with its substitute color
    )                         //
  :                           // else:
    F(b)                      //   try again