Inserts via Entity Framework on server take about double the time than locally
All of the servers are more powerful than my local machine, so it most
  likely is not a hardware problem.


This is a common misconception.

Single-row inserts, like EF generates, are not expensive in CPU cycles.  Typically bottleneck is writing to the log file.  Your local machine may well give you better log write performance than a powerful server.  Your local machine may have a fast, local SSD.  Or you may have write caching enabled on your local disk.


  What could be the bottleneck here?


My guess is log writes.  If you commit after each row, then this will be much worse.  In EF each call to SaveChanges() uses a separate transaction, so you want to batch 10s to 100s of rows in each call.  You also don't want to have too many rows added to a DbContext instance, as they are all tracked for the lifetime of the DbContext.

But more generally, SQL Server will tell you where the bottleneck is.  You can look at the session wait stats for the session performing the insert in SQL 2016+:

select *
from sys.dm_exec_session_wait_stats
where session_id = 51
order by wait_time_ms desc


Or you can look at the server-wide wait stats while the import is running:

https://blogs.msdn.microsoft.com/dbrowne/2013/01/18/my-favorite-query-for-investigating-sql-server-performance/