Checking for duplicates in array
I think you're checking correctly with {% if variantImage not in productGalleryArray %} (this should check if variantImage isn't already in the array), but your code doesn't seem to set a productGalleryArray array in the global scope. Variables initially set inside for loops are scoped only to that for loop.

So I think you just need to declare an empty array at the top of your code outside and before the first for loop:

{% set productGalleryArray = [] %}