Is there an independent oversight body of Journal Impact Factors?
No.

Academic publishing is not regulated by any oversight body.  There is not even a universal standard for what constitutes a publication.