How do I maintain high INSERT-performance on PostgreSQL
There are a few things that might be causing this issue, but I can't be sure any of them are the real problem.  The troubleshooting all involves turning on extra logging in the database, then seeing if the slow parts line up with messages there.  Make sure you put a timestamp in the log_line_prefix setting to have useful logs to look at.  See my tuning intro to get started here:  https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server

Postgres does all of its writes to the operating system cache, then later they head to disk.  You can watch that by turning on log_checkpoints and reading the messages.  When things slow down, it may simply be that all the caches are now full, and all writes are stuck waiting for the slowest part of I/O.  You might improve this by changing the Postgres checkpoint settings.

There's an internal issue with the database people hit sometimes where heavy inserts get stuck waiting for a lock in the database.  Turn on log_lock_waits to see if you're hitting that one.

Sometimes the rate you can do burst inserts at is higher than you can sustain once the system autovacuum process kicks in.  Turn on log_autovacuum to see if the problems are concurrent with when it's happening.

We know that large amount of memory in the database's private shared_buffers cache doesn't work as well on Windows as it does on other operating systems.  There isn't as much visibility into what goes wrong when it happens either.  I would not try to host something that's doing 1000+ inserts a second to a Windows PostgreSQL database.  It's just not a good platform for really heavy writes yet.