Generate my own public/private key to secure my bitcoins
As long as you have the wallet dump, you will be able to access your bitcoin.

However, your approach doesn't line up - if you are already at the point of not trusting hardware wallets, trusting a random dedicated server is not a reasonable approach. In fact, a dedicated server may be a worse choice simply because you have no control over the storage drives - it is trivial to recover deleted files from most drives, especially when such a small amount of data is being written.

Additionally, you also are exposed to any compromise on your local machine. The moment you copy over the wallet dump, or even cat it, anyone who has reasonable access to your local machine via a remote access tool or other malware will be able to see the keys.

The entire point of devices like the ledger is to ensure that the private keys are never, ever placed on any computer at all. They exist solely on that device, and on the paper backup of the mnemonic words.

If you want to go a step further, you can use something like the Cold Card (this is not an endorsement, I have no relation to them, and cannot vouch for them). These aren't even connected to a computer at all, and work with PSBTs via SD cards.