bad camera angle (or trackers?) with aerial footage
Solve mode should not be tripod. Tripod solving is for when the camera is not moving, just rotating from a fixed position. Tripod solve will not result in 3D reconstruction, it will only solve the camera movement, but the trackers will not contain any depth information, they will be projected as in a sphere around the camera.
When the camera moves (as in advances forward of back, or is being displaced sideways or up or down), then use the Default Camera Solve.
As for how to proceed, most of it has been answered here.
Read the following link: get better results for camera tracking
Most problems with solving 3d information come from poor (inaccurate) tracking.  You need at least 8 trackers, but more can help you in this case. Track objects on the field around the buildings as well. Prioritize objects that move fast (in the foreground) and then track a few objects that are very far away.