How to override thumbnail generation of file type media?
In the media source plugin for this media type you can find the code generating the thumbnail. See the source code of the "File" media type provider.

To summarize the code, you have to put your thumbnails files for each supported MIME type in the folder configured in the "icon_base_uri" configuration (which is stored in the "media.settings.yml" configuration file).

For example, using the default "public://media-icons/generic" folder and inserting a PDF media, you have to store your thumbnail image under the name "public://media-icons/generic/application--pdf.png" for it to be detected and used.