Any drawbacks of using GIN PostgreSQL index for an integer foreign key?
This seems like a rather esoteric thing, which is probably why it doesn't get recommended all that much.  I do recommend it when the issue comes up, which just isn't all that often.  The size of one index is unlikely to be all that meaningful in the context of an entire database, so making it smaller usually isn't worth worrying about a great deal.

I don't think there are any downsides which are specific to this being a foreign key. The automatically-generated queries that are used to maintain the many-valued side of the constraint can use the GIN index just as well as the Btree index. 

I'd turn "fastupdate" to off, unless you do relevant benchmarking which shows you want it on.  Since the index is over a scalar using "btree_gin", you don't get the explosion in index page writes for each inserted row, like you do with a normal GIN index, so the need for fastupdate is less.

Since this index type is much less used than regular btree indexes, there are more likely to be undiscovered bugs lurking in it (especially if fastpdate = on).  I wouldn't (and don't) let that put me off from using it if it really provided me with something of value, but it keeps me from just blindly using "btree_gin" everywhere I conceivably could.

One thing I've noticed is that the replay of updates/inserts to GIN indexes seem pretty slow (compared to what I naively expected), which could be relevant for PITR, recovery, or streaming replication.  If any of those things are important to you, make sure you include them in your test.