Given a set of lines, find only those who are parallel (perspective)
Parallel lines in the image do intersect at a vanishing point. Therefore simply hypothesizing lines (a gradient direction at a point suffices to describe it) and voting (see Hough voting) would suffice to identify this point. One could then record all the lines that casted votes to this very point and identify them. Care must be taken as it is difficult to vote directly on a vanishing point, i.e. the space is not bounded and explodes to infinity. Instead, it is possible to use a Gaussian Sphere as Barnard suggested. 

Vanishing point extraction is a well studied topic and many algorithms exist, some of which are:



  A Novel Line Space Voting Method for Vanishing-Point Detection of
  General Road Images, Zongsheng Wu, Weiping Fu, Ru Xue and Wen Wang
  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4970002/


  Vanishing point detection for road detection, Hui Kong, Jean-Yves
  Audibert, Jean Ponce http://www.di.ens.fr/sierra/pdfs/cvpr09c.pdf


  Vanishing point detection in corridors: using hough transform and
  K-means clustering, R. Ebrahimpour,  R. Rasoolinezhad, Z.
  Hajiabolhasani,  M. Ebrahimi
  http://ieeexplore.ieee.org/document/6135447/


  Hough Transform for Directional Orientation, Ivan Papusha, Matthew
  Ho
  https://stacks.stanford.edu/file/druid:rz261ds9725/Ho_Papusha_RealTimeHoughTransform.pdf



Finally, when multiple lines cause a problem, one could always cluster them using the line parameters $[a,b,c]$. When number of lines are not known, one can use a hierarchical clustering, such as single-linkage or agglomerative clustering.