What Is the Purpose of Bundle Adjustment?
If all you want is to reconstruct a scene from a pair of images from a pair of calibrated stereo cameras, and your calibration is sufficiently accurate, then you do not need bundle adjustment. 

You do need bundle adjustment if you want to reconstruct a scene from a sequence of images or for a sequence of stereo pairs, where the camera poses in each view are unknown. Search for "structure from motion" and "visual odometry".