How does one create another transaction when all funds are tied up in an unconfirmed previous transaction? If I have only one UTXO and I spend from it that's fine, I send myself back the change and that works how I expect it to. However if I want to create a new transaction in the time that the first transaction hasn't confirmed yet then what do wallets do?

Do they use CPFP/RBF? What's the most reliable way to send a new transaction that depends on the first transaction getting confirmed.
It depends on your wallet software. Some may simply refuse to create the transaction. Others will spend the change output of the unconfirmed transaction. CPFP and RBF typically only occur when you specifically request for it in order to increase the fee of an unconfirmed transaction. Wallets will not do this automatically for you.