Threads and logfiles
I'd use a dictionary (or a HashSet if you don't need the ID information) for coordination.

class UrlLog {
    private Dictionary&lt;string, int&gt; visited = new Dictionary&lt;string, int&gt;();
    public bool HasUrlBeenVisited(string Url) {
        lock(visited) return visited.ContainsKey(Url);
    }
    public void SetUrlVisited(string Url, int threadId) { /* id not strictly necessary */
        lock(visited) visited[Url] = threadId;
    }
}


Then pass one instance of this UrlLog to your threads, and they can use that.

Also, I'm not sure you even need the locks for the checks, but it's better to be safe than sorry. :)