How to set threshold value for Hough Transform
Well, it really depends on what you expect to find in the image.  The matlab function uses an example threshold of half the largest peak.

What sort of images are you using this on?



So, based on just finding vertical lines in your image, let's make some assumptions.


Your rectangle's minimum vertical side length is $L_{\rm min}$ pixels.
Your edge detection only finds 80% of each edge.


Then the minimum threshold will be 
$$
0.8 L_{\rm min}
$$

You many need to modify this if, for example, your edges tend to be thicker or thinner than the 80% figure.