Why isn't the perspective/focal length correct in this camera track?
Delete the trackers with errors (like the one that tracks a reflection in the window, and one that has only one keyframe).
Then set origin, X and Y, and solve again.
Set the tracking scene again.
You can always fine tune placement manually by grabbing any of the tracking markers and moving them so that they align with the grid (presuming of course that the house is really square, which most of them are not). Keep in mind that the camera solve is a constraint, so moving one element will move everything, including the camera, so there is no danger of having any of the tracked elements be misplaced relative to the camera.

Try to line up markers that belong in the same axis (like a wall, for example).
Then just get a cube in there and try to match the features. Start with the edge of the wall that is closest to the camera, then start moving faces to get the height, width and length.
With a minimum of guesswork and re-orienting the markers manually, will get your camera and geometry in the correct place.
For more detail read:
How to align the camera in a solved motion tracked scene?