In Memory Database in Unit tests, isolate the tests
Generally speaking whenever we are about to write unit tests we should follow the F.I.R.S.T. principles. It is an acronym, which stand for:


Fast: The execution time should be measured in milliseconds. If it takes a second or two then it can be considered as slow and should be revised.
Isolated: Each test is self-contained and does not rely on any other data. They can be run in any order.
Repeatable: The test runs must be repeated over and over again. In each and every run they should either pass every time or always fail.
Self-validating: There is no need for human interpretation whether or not the test succeeded or failed. The test result should be self-explanatory.
Timely: The code and related tests should be written in almost the same time. A new code without relevant unit tests should not be deployed.


Let's examine these ideas for your proposals:

Single database and cleanup


Fast: If for whatever reason a previous cleanup phase missed / failed then your database will have some trash data in it, which might impact the performance of your database operations. Executing the cleanup during setup and teardown might solve the problem, but it will definitely have performance impact.
Isolated: They are sharing the same database, so they are not truly isolated. It might be the case that they can't run in parallel, because ordering might matter.
Repeatable: Because they are using the same database, that's why order might affect the result of your assertions. In case of MSTest you can define ordering but if you need to use them that means your tests are not really isolated.
Self-validating: Because there is a chance for race condition your test results are non-deterministic, which means human-intervention is needed to interpret several resultsets, reproduce the issues (if it is even possible) and fix them. 
Timely: It is irrelevant in our discussion 


Separate database for each test case


Fast: Creating new in-memory database for each test should not impose performance penalty onto tests if there not too many tables and constraints.
Isolated: Separate databases are used for each test, means no shared resource is being used, which helps isolation.
Repeatable: Because each and every time you run your test against a brand new database, there won't be any trash data, which could cause race condition.
Self-validating: By being deterministic, no human intervention is needed to understand the test results.
Timely: It is irrelevant in our discussion




If you don't want to examine the test data manually, then you don't really need use the test name in the database name. You can use any random value:

int jitter = idGenerator.Next();
var condigBuilder = DbContextOptionsBuilder&lt;TestContext&gt;()
    .UseInMemoryDatabase(databaseName: $"TestDb{jitter}") 
    .Options;


or

Guid jitter = Guid.NewGuid();
var condigBuilder = DbContextOptionsBuilder&lt;TestContext&gt;()
    .UseInMemoryDatabase(databaseName: $"TestDb{jitter}") 
    .Options;