How to remove or seal fan hole
If you have easy-ish access to the outside, I'd clean as much of the detritus away, patch the hole with plexiglass (perspex) and copious amounts of silicone caulk. Then install a triangular chunk of exterior plywood (painted both sides and all edges). Flash and caulk as much as possible. Attach as possible through the window frame. 

On the interior, more plexiglass and caulk to seal the interior side. Then a skin of painted (both sides, all edges) exterior plywood (preferably with a smooth face) all the way across the window. Attach as possible to the window frame. Attach a bit of molding at the bottom if there's a gap you dislike. Caulk the heck out of it in an attempt to keep moisture from passing through.

For what it's worth, vent fans like this have a reason for existing. Everything above could be tweaked to allow for the install of a new unit. (Just remember where the hole was.)