Adding vbos to a vao once it has been created
So after debugging some more, I've found that yes it is possible. You cam add buffers later on but they will need to point to their own attributes.

My issues came from thinking I could have multiple vbos contributing to a single attribute. That doesn't work. You need multiple vaos for that scenario.