CRUD integration test, is it ok?
Adding automated integration tests is a great way to take testing to the next level, and what you've written is really good for a first attempt. Here are some pointers/suggestions that might help you along your way.

It's almost inevitable that your tests will end up effectively testing the same thing multiple times. As you've identified, most of your tests use the GetById method, and in fact all of them Insert a payment. The only potential problem this causes is that if one of those methods stops working you suddenly have most/all of your tests failing and it may not be immediately obvious where the problem is. In the grand scheme of things, this isn't that much of an issue as you'll almost certainly be able to figure out where the problem is by looking at the test output and/or by stepping through the test with a debugger.

Although it seems like your tests are very similar, a lot of the similarities are down to them sharing common setup steps, e.g. inserting a payment so you have something to get/update/delete. It is absolutely fine to have this sort of duplication in unit/integration tests, and again is inevitable in most circumstances. As the number of tests grows, you might find that it makes sense to pull out some of the setup into a separate method that's used by multiple tests. This is also fine to do, as long as it doesn't affect the readability of the test, so avoid using generic method names like Setup() and instead prefer clear (if slightly verbose) names like CreateRepositoryWithSinglePayment(testPayment).

The fluent payment builder makes it hard to know which properties are required and which are optional (if any). I can't advise a great deal on this without seeing the Payment or PaymentBuilder classes, but if all of the properties are required, consider just creating a Payment constructor that assigns parameter values to readonly properties. Then there's no doubt about what values are required to create a valid Payment.

One thing to be aware of when integration testing is that it's often easy to end up with shared state between tests. Here, you are inserting a payment into your database as part of every test and this is not always deleted afterwards (or at least there's no tear-down code included in your post). Since you are actually persisting something in a database, it will therefore exist across tests unless explicitly deleted after each test is run. This could result in some very confusing test behaviour in the future as your tests might not be testing what you intended! You should add a tear-down step by implementing IDisposable and deleting every payment in your test database inside the Dispose() method. This ensures that every test will start from the same fresh state.
On a similar note, be aware that you can get strange behaviour when running integration tests in parallel if they share the same data store(s). You're fine at the moment because by default xUnit runs test cases within the same class in serial, but if you had another test class using the same database these would be run in parallel by default. There are ways to control this behaviour,  but it's more something to be aware of as this shouldn't be an issue given your current test organisation.

You shouldn't need to provide the entire Payment object to delete it from the database. If you can read a payment by its Id using the GetById method then you should also be able to delete a payment by its Id. This should simplify the Delete test case somewhat.

There are plenty of other things you could test here. You've got tests for the 'happy' paths, i.e. the paths where everything goes as expected, but there are certainly lots of edge cases (or unexpected cases) that you could also test. Maybe you've already considered some/all of these, but off the top of my head:

What happens when you insert a payment that already exists in the database?
What happens when you try to read a payment that does not exist in the database?
What happens when trying to update a payment that doesn't exist in the database?
Can you update the payment with a null description? What happens if the description is really long or contains special characters? You'd probably want to use a [Theory] with [InlineData] to enumerate several different test cases if doing this.
What happens when updating properties other than the description?
What happens when trying to delete a payment that doesn't exist in the database?