What factors could cause a massive spike / drop in bitrate?
Bitrate is an average. As with any average, it is greatly affected by the size of the window it is averaged over. On average the temperature where I live is around 70 degrees F. But some days it’s 30 and some days it’s 120. But the average is 70. If you do daily readings you will see spikes. If you do monthly reading, you will see a sin wave, if you do yearly readings, the line will be pretty flat (not factoring in climate change). 

Individual frame sizes vary massively depending on information density, motion, scene detection, and minimum key frame intervals. It’s a function of how video compression works and completely normal. 

Research how video compression works. Specificity I/P/B frames and it will make sense. 

Also note that available bandwidth changes very rapidly. Especially on wireless networks. And many video conferencing software will adapt to keep the call working.