When suggesting referee names, where do you draw the line for conflict of interest?
The National Science Foundation in the US has clearly articulated guidelines for what constitutes a conflict of interest for reviewers of grant proposals.  These are probably safe — and certainly defensible — guidelines for you to follow.  Specifically (and for the reason you mention), research collaborations are considered to be a conflict only if they occurred within the last 4 years (there are other rules as well, please see the document).