Do referee lists of journals compromise the single-blind peer-review process?
Bear in mind that many papers are reviewed without being published, and that no one knows everyone who could potentially be a reviewer of their paper. So you may guess, but you really can't do so with certainty. 

I think a bigger issue in this regard is the shepherding process done in some conferences, in which one person who served as a reviewer interacts directly with the authors to verify that concerns from the reviews are addressed.  Now you know who a reviewer was, just not which review they wrote.  Such conferences often have 5-7 reviews per accepted paper, but I've seen it done with as few as 3. 

In either case, I think the possible unblinding of a reviewer is a minor concern. You really ought to be writing the public portion of reviews as though the authors know exactly who is reviewing (this came up in another question recently) and some reviewers choose to sign their reviews explicitly as a matter of principle.