title: C# collection performance: Are Hashset<string> and Dictionary<string, IList<int>> the fastest collections for this purpose?
body: <p>I've parsed a grid of Excel data and built up an object model. There's 32 columns and 100,000 rows. I've been asked to check for rows with duplicate data and report them. For my implementation I'm doing the following:</p>
  
  <ol>
  <li>Using Tasks I'm building an array of Tuple with row id and concatenated cell contents.</li>
  <li>I loop through the resultant array and using a HashSet, I try to insert the concatenated value into the HashSet:</li>
  <li>If HashSet.Add() passes, I create a new entry in my Dictionary> result set for it.</li>
  <li>If HashSet.Add() fails I add that row id to the existing entry in my my Dictionary> result set</li>
  </ol>
  
  <p>Step 1 takes 0.09s, while the rest is taking 822s to process :/ Can anyone spot where I can chop this time down with a more appropriate choice of collections or algorithms?</p>
  
  <p>Code is below:</p>
  
  <pre><code>var results = new Dictionary&lt;string, IList&lt;int&gt;&gt;(numberOfRows);
  var hashSet = new HashSet&lt;string&gt;();
  var duplicateErrors = new List&lt;string&gt;();
  
  for (var row = firstRow; row &lt;= lastRow; row++)
  {
      var row1 = row;
      taskArray[count++] =
      Task&lt;Tuple&lt;int, string&gt;&gt;.Factory.StartNew(() =&gt; GetCompleteRowData(row1, tableRawDataHolders));
  }
  
  foreach (var task in taskArray)
  {
      if (hashSet.Add(task.Result.Item2))
      {
          results.Add(task.Result.Item2, new List&lt;int&gt;() { task.Result.Item1 });
      }
      else
      {
          results[task.Result.Item2].Add(task.Result.Item1);
      }
  }
  </code></pre>
  
  <p>and</p>
  
  <pre><code>public Tuple&lt;int, string&gt; GetCompleteRowData(int row, IEnumerable&lt;ITableRawDataHolder&gt; tableRawDataHolders)
      {
          return new Tuple&lt;int, string&gt;(row, string.Join("",
              tableRawDataHolders.Where(c =&gt; c.Row == row).Select(c =&gt; c.Value).ToArray()));
      }
  </code></pre>
  
  <p>and</p>
  
  <pre><code>public class TableRawDataHolder : ITableRawDataHolder
  {
      public int Row { get; }
      public int Column { get; }
      public string Value { get; }
  
      public TableRawDataHolder(int row, int column, string value)
      {
          Row = row;
          Column = column;
          Value = value;
      }
  }
  </code></pre>