C# dedupe List based on split
<p>I'm having a hard time deduping a list based on a specific delimiter.</p>

<p>For example I have 4 strings like below:</p>

<p>apple|pear|fruit|basket<br>
orange|mango|fruit|turtle<br>
purple|red|black|green<br>
hero|thor|ironman|hulk  </p>

<p>In this example I should want my list to only have unique values in column 3, so it would result in an List that looks like this,</p>

<p>apple|pear|fruit|basket<br>
purple|red|black|green<br>
hero|thor|ironman|hulk </p>

<p>In the above example I would have gotten rid of line 2 because line 1 had the same result in column 3.  Any help would be awesome, deduping is tough in C#.</p>

<p>how i'm testing this:</p>

<pre><code>    static void Main(string[] args)
    {
        BeginListSet = new List&lt;string&gt;();
        startHashSet();
    }


    public static List&lt;string&gt; BeginListSet { get; set; }

    public static void startHashSet()
    {
        string[] BeginFileLine = File.ReadAllLines(@"C:\testit.txt");
        foreach (string begLine in BeginFileLine)
        {

            BeginListSet.Add(begLine);
        }

    }

    public static IEnumerable&lt;string&gt; Dedupe(IEnumerable&lt;string&gt; list, char seperator, int keyIndex)
    {
        var hashset = new HashSet&lt;string&gt;();
        foreach (string item in list)
        {
            var array = item.Split(seperator);
            if (hashset.Add(array[keyIndex]))
                yield return item;
        }
    }
</code></pre>
