<p>Please, now that I've re-written the question, and before it suffers from further <a href="https://stackoverflow.com/questions/56103/fastest-gun-in-the-west-problem">fast-gun answers</a> or premature closure by <a href="https://stackoverflow.com/users/905/keith">eager editors</a> let me point out that this is not a duplicate of <a href="https://stackoverflow.com/questions/9673/remove-duplicates-from-array">this question</a>.  I know how to remove duplicates from an array.</p>

<p>This question is about removing <strong>sequences</strong> from an array, not duplicates in the strict sense.</p>

<p>Consider this sequence of elements in an array;</p>

<pre><code>[0] a
[1] a
[2] b
[3] c
[4] c
[5] a
[6] c
[7] d
[8] c
[9] d
</code></pre>

<p>In this example I want to obtain the following...</p>

<pre><code>[0] a
[1] b
[2] c
[3] a
[4] c
[5] d
</code></pre>

<p>Notice that duplicate elements are retained but that sequences of the same element have been reduced to a single instance of that element.</p>

<p>Further, notice that when two lines repeat they should be reduced to one set (of two lines).</p>

<pre><code>[0] c
[1] d
[2] c
[3] d
</code></pre>

<p>...reduces to...</p>

<pre><code>[0] c
[1] d
</code></pre>

<p>I'm coding in C# but algorithms in any language appreciated.</p>

[{'answer_id': 57013, 'author': 'Bill the Lizard', 'author_id': 1288, 'author_profile': 'https://Stackoverflow.com/users/1288', 'pm_score': 1, 'selected': False, 'text': '<p>I would dump them all into your favorite Set implementation.</p>\n\n<p>EDIT:  Now that I understand the question, your original solution looks like the best way to do this.  Just loop through the array once, keeping an array of flags to mark which elements to keep, plus a counter to keep track to the size of the new array.  Then loop through again to copy all the keepers to a new array.</p>\n'}, {'answer_id': 57022, 'author': 'Terry', 'author_id': 2171, 'author_profile': 'https://Stackoverflow.com/users/2171', 'pm_score': 0, 'selected': False, 'text': "<p>I agree that if you can just dump the strings into a Set, then that might be the easiest solution. </p>\n\n<p>If you don't have access to a Set implementation for some reason, I would just sort the strings alphabetically and then go through once and remove the duplicates. How to sort them and remove duplicates from the list will depend on what language and environment you are running your code. </p>\n\n<p>EDIT: Oh, ick....  I see based on your clarification that you expect that patterns might occur even over separate lines. My approach won't solve your problem. Sorry. Here is a question for you. If I had the following file.</p>\n\n<p>a</p>\n\n<p>a</p>\n\n<p>b</p>\n\n<p>c</p>\n\n<p>c</p>\n\n<p>a</p>\n\n<p>a</p>\n\n<p>b</p>\n\n<p>c</p>\n\n<p>c</p>\n\n<p>Would you expect it to simplify to </p>\n\n<p>a</p>\n\n<p>b</p>\n\n<p>c</p>\n"}, {'answer_id': 57181, 'author': 'nlucaroni', 'author_id': 157, 'author_profile': 'https://Stackoverflow.com/users/157', 'pm_score': 2, 'selected': False, 'text': "<p><strong>EDIT: made some changes and new suggestions</strong></p>\n\n<p>What about a sliding window...</p>\n\n<pre><code>REMOVE LENGTH 2: (no other length has other matches)\n//the lower case letters are the matches\nABCBAbabaBBCbcbcbVbvBCbcbcAB  \n__ABCBABABABBCBCBCBVBVBCBCBCAB\n\nREMOVE LENGTH 1 (duplicate characters):\n//* denote that a string was removed to prevent continual contraction\n//of the string, unless this is what you want.\nABCBA*BbC*V*BC*AB\n_ABCBA*BBC*V*BC*AB\n\nRESULT:\nABCBA*B*C*V*BC*AB == ABCBABCVBCAB\n</code></pre>\n\n<p>This is of course starting with length=2, increase it to L/2 and iterate down. </p>\n\n<p>I'm also thinking of two other approaches:</p>\n\n<ol>\n<li><strong>digraph</strong> - Set a stateful digraph with the data and iterate over it with the string, if a cycle is found you'll have a duplication. I'm not sure how easy it is check check for these cycles... possibly some dynamic programming, so it could be equivlent to method 2 below. I'm going to have to think about this one as well longer.</li>\n<li><strong>distance matrix</strong> - using a levenstein distance matrix you might be able to detect duplication from diagonal movement (off the diagonal) with cost 0. This could indicate duplication of data. I will have to think about this more.</li>\n</ol>\n"}, {'answer_id': 57410, 'author': 'sieben', 'author_id': 1147, 'author_profile': 'https://Stackoverflow.com/users/1147', 'pm_score': 3, 'selected': True, 'text': '<p>Here\'s C# app i wrote that solves this problem.</p>\n\n<p><strong>takes</strong><br>\naabccacdcd  </p>\n\n<p><strong>outputs</strong><br>\nabcacd  </p>\n\n<p>Probably looks pretty messy, took me a bit to get my head around the dynamic pattern length bit.</p>\n\n<pre><code>class Program\n{\n    private static List&lt;string&gt; values;\n    private const int MAX_PATTERN_LENGTH = 4;\n\n    static void Main(string[] args)\n    {\n        values = new List&lt;string&gt;();\n        values.AddRange(new string[] { "a", "b", "c", "c", "a", "c", "d", "c", "d" });\n\n\n        for (int i = MAX_PATTERN_LENGTH; i &gt; 0; i--)\n        {\n            RemoveDuplicatesOfLength(i);\n        }\n\n        foreach (string s in values)\n        {\n            Console.WriteLine(s);\n        }\n    }\n\n    private static void RemoveDuplicatesOfLength(int dupeLength)\n    {\n        for (int i = 0; i &lt; values.Count; i++)\n        {\n            if (i + dupeLength &gt; values.Count)\n                break;\n\n            if (i + dupeLength + dupeLength &gt; values.Count)\n                break;\n\n            var patternA = values.GetRange(i, dupeLength);\n            var patternB = values.GetRange(i + dupeLength, dupeLength);\n\n            bool isPattern = ComparePatterns(patternA, patternB);\n\n            if (isPattern)\n            {\n                values.RemoveRange(i, dupeLength);\n            }\n        }\n    }\n\n    private static bool ComparePatterns(List&lt;string&gt; pattern, List&lt;string&gt; candidate)\n    {\n        for (int i = 0; i &lt; pattern.Count; i++)\n        {\n            if (pattern[i] != candidate[i])\n                return false;\n        }\n\n        return true;\n    }\n}\n</code></pre>\n\n<p><em>fixed the initial values to match the questions values</em></p>\n'}]
['https://Stackoverflow.com/questions/57010', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/4200/']