What good is to store all the transactions in the history Bitcoin stores all the transactions that happened
in the blockchain.

Are really all the transactions needed? What good is it
to have transactions ten years old stored. Isn't it a waste
of discspace?
The bitcoin blockchain is a record of all transactions. This allows the user to independently validate transactions, and thus verify the authenticity of coins. However, you don't need to store all of the transactions for all time, by running a 'pruned node' you can keep disk-space usage low, while still verifying all transactions. 

Background: bitcoins exist as 'Unspent Transaction Outputs' (UTXOs). Each individual transaction will consume some number of UTXOs as inputs*, and create new UTXO(s) as output(s). 

Under this model, you can trace a UTXO back in time in order to validate it as being an authentic bitcoin transaction. For example, consider some bitcoin you receive today, lets call it 'UTXO-3'. In a simplified manner, you could trace the history of this UTXO-3 (to it's input 'UTXO-2', and so forth), until you find the coinbase transaction that mined the coins.

So ,going back in time, from your UTXO-3:

UTXO-3 &lt;- UTXO-2 &lt;- UTXO-1 &lt;- Newly mined coins

This allows a bitcoin (UTXO) to be traced back to it's origin on the blockchain, which means you can validate it as an authentic bitcoin. Without this feature, you would be unable to verify for yourself that the bitcoin in question was created in a way which follows the network's rules. As mentioned above, bitcoin's design allows you to perform this verification without explicitly storing all transactions for all time (by running a pruned node). 

Additionally, as Pieter mentioned in a comment, a user has the option to use a light-client, which does not require the user to store a copy of the blockchain locally (although this employs a slightly different trust model than running a full-node yourself).

.* (the exception to this rule is a coinbase transaction, which follows special rules to generate coins, without having any inputs)