What is the most expensive part of mining? From my understanding, miner's take a bunch of transactions and use a nonce, previous block and the transaction data to try and find a hash that beats the difficulty. As soon as this happens, will their block be added, and then 99 blocks later the BTC will be awarded.

Is the solving aspect of the hash the most expensive part, for the computer? Or is gathering the transactions and confirming them more expensive? 

How does one confirm the transactions? Do they have to traverse the entire blockchain, or only find the most recent ledger which shows both user's balances to confirm?
Is the solving aspect of the hash the most expensive part, for the
  computer? Or is gathering the transactions and confirming them more
  expensive?


Finding the nonce that produces the correct hash is by far the most computationally expensive part.


  How does one confirm the transactions? Do they have to traverse the
  entire blockchain, or only find the most recent ledger which shows
  both user's balances to confirm?


The blockchain is the only bitcoin ledger. One confirms transactions by confirming each of them. To confirm a transaction is valid it must satisfy certain rules. One of these rules is that each of transaction's inputs must be correctly signed.

Transaction's input comes from an output of some previous transactions. Bitcoin node that wants to validate the transaction maintains a list of all unspent previous transactions (this list updates regularly and contains only small part of the blockchain), so it can easily find the output that the current transaction's input points to. The output must be in this list of unspent transaction's. An output has all information to verify respective input's digital signature. 

So the node that checks the block, makes the above input verification for each input of each transaction.