title: Threading Over a List(Of Lists)
body: <p>I have a list object with the following structure </p>
  
  <pre><code>Dim urlMasterList As List(Of List(Of UrlObject) = AggregateUrls()
  </code></pre>
  
  <p><code>AggregateUrls</code> returns lists of domain delimited urls to be consumed by a function called <code>UrlValidator</code>.  The task here is to check whether the urls in these lists are valid and if not add them to a list of broken urls. </p>
  
  <p>Currently I have a nested for each loop to process this list of lists</p>
  
  <pre><code>For Each hostDelimitedUrlList As List(Of UrlObject) In urlMasterList                    
      For Each entry As UrlObject In hostDelimitedUrlList
          If badUrls.Exists(Function(x) x.Url = entry.Url) Then 'badUrls is the master broken list
              badUrls.Add(entry)
          Else
              Try
                  If Not UrlValidator(entry.Url) Then
                      badUrls.Add(entry)
                  End If
              Catch ex As Exception
                  badUrls.Add(entry)
              End Try
          End If
      Next
  Next
  </code></pre>
  
  <p>I am trying to find a good method for threading over this master list.  I have tried a parallel foreach but that yielded some weird results and duplicate entries, even with locking the writing process to <code>badUrls</code></p>
  
  <p>Is this task suited for threading or should I approach it from a different angle?  Ideally I think the approach would be as follows: Have the main thread serve up chunks (5 lists from the master list) to 5 worker threads, when one finishes grab the next resource and serve it up. I cannot find any sort of example or tutorial for this type of issue. </p>