Where is my bitcoin balance stored in the blockchain? This is possibly a duplicate but, since I am a Bitcoin dummy I cannot find it.

Anyhow; as I understand (and I will make some assumptions here), a block in the blockchain contains one or more transactions.

Transactions are typically bits of information which contains delta's, I mean e.g.: Alice transfers 2 bitcoins to Bert.

So, to determine or verify my bitcoin balance (in order to make a valid transaction), does the whole chain needs to be analysed? If so, is this optimized in any way?

Or am I completely off track here?
The blockchain defines your balance, but it doesn't have a specific field in which your balance is stored.

Yes, it needs to be analyzed to find your balance. However, it doesn't have to be analyzed all over again, just to check for the balance of a new address. Instead, checking the UTXO set (which only has to be created once and then to be updated every time new blocks are received) is sufficient. Furthermore, not the entire UTXO set has to be traversed. There are things like hash maps which make this a lot easier.