1. A method for reconstructing-three-dimensional (3D) lines in a 3D world coordinate system from two-dimensional (2D) lines in a 2D image of scene, comprising: detecting and clustering the 2D lines using vanishing points; generating a graph of vertices and edges, wherein each vertex on the graph represents a line in the 2D image, and wherein each edge connecting two vertexes represents a possibility of a constraint on intersection of two lines corresponding to the two vertices in the 3D world coordinate system; testing jointly the constraints on intersections using orthogonality, collinearity and junction properties of the intersection as a soft penalty for identifying the constraints on intersections that are satisfied on the 3D lines; and reconstructing the 3D lines in the 3D world coordinate system using the identified constraints on intersections, wherein steps of the method are performed by a processor.