Issue title: Switching ROMs results in many ways to Segfault the game
 Issue body: While this is probably to be expected with a feature as amusingly nuts as switching ROMs mid execution (seriously, I love this and Yank Game Pak entirely too much), if say, you load Pokemon Red (US) and then switch to Pokemon Blue (U), the games work hilariously fine, but if you do a regular ROM load another ROM after that, mGBA segfaults. Some other ROM comboes segfault off the bat (Donkey Kong (U) -> Kirby's Dream Land (U)), and others have the same "after switch, next regular load crashes" for Pokemon Fire Red (U) -> Pokemon Leaf Green (U)).

Version of mGBA tested: 0.5.0, 0.6-1-40fb750-dirty (built from Arch AUR package)
OS: Arch Linux 64-bit
CPU/GPU: Intel Core m5-6Y54/Intel HD Graphics 515.

 Comments: 
 Comment 0: I totally forgot to test this feature with GB ROMs. It's probably hilariously broken right now.
 Comment 1: Eh, I can reproduce the load bug on GBA games (see the Fire Red/Leaf Green example). But the GB ones are prone to just flat out crashing at times when you do it immediately.
 Comment 2: Partially fixed in 8daa39d21b1dccac49e7dc5ab419819836bbf93d, but GB can still crash any other number of ways.
 Comment 3: I know it's been 4 and a half years, but I finally got around to cleaning up this code path.
{'source': 'GithubIssues', 'repo_name_with_owner': 'mgba-emu/mgba', 'issue_id': 178798482, 'repo_stars': 4264.0}