<p>Ideally the reader has upgraded a native C++ program to Visual Studio 2008, which contains an OpenClipboard() block.  Why not try setting a breakpoint just after getting a successful return-code from OpenClipboard() and step through your code.  According to the Internet it may work on your system, but of course, not on mine, thanks for trying.  </p>

<p>Googling on e.g. (( OpenClipboard 1418 vc6 )) finds articles like "GetClipboardData fails in debugger" and "No Error in VC++6 but Error in VC++ 2005".  Pragmatically for-the-moment, problem solved - I simply cannot set breakpoints within such code, I need to squirrel information and set the breakpoint after the clipboard operations are done.  Error 1418 is "Thread does not have a clipboard open" but it works fine as long as you don't step with VS.NET, or like I say if you keep breakpoints outside of the clipboard-open-close-block.<p>
I would feel better knowing what the exact issue is with the VS.NET debugger.<p>
Being a C++ person I am only dimly aware that you are not supposed to think in terms of threads when doing dot-Net.  Anyway I did not find a guru-quality explanation of what's really going on, whether in-fact the problem is that the dot-Net debugger is subtly interfering with the thread-information somehow, when you single-step thru native C++ code.  <P></p>

<p>System-wise:  about a year old, two dual-core Xeon's, 4 CPU's according to XP-pro.
  I had just finished debugging the code by single-stepping thru it in vc6 under XP-SP2-32-bit.  So I know the code was pretty-much-fine under vc6.  However when I tested with a 10-megabyte CF_TEXT I got exceptions.  I thought to try debugging under the nicer exception model of XP-x64.<p>
Recompiled with visual-studio-2008, I could not get the code to single-step at all.  OpenClipboard worked, but EnumClipboardFormats() did not work, nothing worked when single-stepped.  However, when I set the breakpoint below the complete block of code, everything worked fine.  And <em>YES</em> vc2008 made a pinpoint diagnostic 'stack frame corruption around szBuf.  There is a lot to like about vc2008.  It would be nice if this were somehow merely a clipboard problem - without knowing that I would feel compelled to worry about stepping thru ANYTHING, whether thread-context-issues might be due to the dot-Net-debugger.</p>

[{'answer_id': 79985, 'author': 'Jerub', 'author_id': 14648, 'author_profile': 'https://Stackoverflow.com/users/14648', 'pm_score': 10, 'selected': True, 'text': '<p>You want <code>split</code>, from the built-in <a href="https://docs.python.org/library/shlex.html" rel="noreferrer"><code>shlex</code></a> module.</p>\n<pre><code>&gt;&gt;&gt; import shlex\n&gt;&gt;&gt; shlex.split(\'this is &quot;a test&quot;\')\n[\'this\', \'is\', \'a test\']\n</code></pre>\n<p>This should do exactly what you want.</p>\n<p>If you want to preserve the quotation marks, then you can pass the <code>posix=False</code> kwarg.</p>\n<pre><code>&gt;&gt;&gt; shlex.split(\'this is &quot;a test&quot;\', posix=False)\n[\'this\', \'is\', \'&quot;a test&quot;\']\n</code></pre>\n'}, {'answer_id': 79989, 'author': 'Allen', 'author_id': 6043, 'author_profile': 'https://Stackoverflow.com/users/6043', 'pm_score': 6, 'selected': False, 'text': '<p>Have a look at the <code>shlex</code> module, particularly <code>shlex.split</code>.</p>\n\n<pre><code>&gt;&gt;&gt; import shlex\n&gt;&gt;&gt; shlex.split(\'This is "a test"\')\n[\'This\', \'is\', \'a test\']\n</code></pre>\n'}, {'answer_id': 80015, 'author': 'pjz', 'author_id': 8002, 'author_profile': 'https://Stackoverflow.com/users/8002', 'pm_score': -1, 'selected': False, 'text': '<p>Try this:</p>\n\n<pre><code>  def adamsplit(s):\n    result = []\n    inquotes = False\n    for substring in s.split(\'"\'):\n      if not inquotes:\n        result.extend(substring.split())\n      else:\n        result.append(substring)\n      inquotes = not inquotes\n    return result\n</code></pre>\n\n<p>Some test strings:</p>\n\n<pre><code>\'This is "a test"\' -&gt; [\'This\', \'is\', \'a test\']\n\'"This is \\\'a test\\\'"\' -&gt; ["This is \'a test\'"]\n</code></pre>\n'}, {'answer_id': 80361, 'author': 'Gregory', 'author_id': 14351, 'author_profile': 'https://Stackoverflow.com/users/14351', 'pm_score': -1, 'selected': False, 'text': '<p>If you don\'t care about sub strings than a simple</p>\n\n<pre><code>&gt;&gt;&gt; \'a short sized string with spaces \'.split()\n</code></pre>\n\n<p>Performance:</p>\n\n<pre><code>&gt;&gt;&gt; s = " (\'a short sized string with spaces \'*100).split() "\n&gt;&gt;&gt; t = timeit.Timer(stmt=s)\n&gt;&gt;&gt; print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)\n171.39 usec/pass\n</code></pre>\n\n<p>Or string module</p>\n\n<pre><code>&gt;&gt;&gt; from string import split as stringsplit; \n&gt;&gt;&gt; stringsplit(\'a short sized string with spaces \'*100)\n</code></pre>\n\n<p>Performance: String module seems to perform better than string methods</p>\n\n<pre><code>&gt;&gt;&gt; s = "stringsplit(\'a short sized string with spaces \'*100)"\n&gt;&gt;&gt; t = timeit.Timer(s, "from string import split as stringsplit")\n&gt;&gt;&gt; print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)\n154.88 usec/pass\n</code></pre>\n\n<p>Or you can use RE engine</p>\n\n<pre><code>&gt;&gt;&gt; from re import split as resplit\n&gt;&gt;&gt; regex = \'\\s+\'\n&gt;&gt;&gt; medstring = \'a short sized string with spaces \'*100\n&gt;&gt;&gt; resplit(regex, medstring)\n</code></pre>\n\n<p>Performance</p>\n\n<pre><code>&gt;&gt;&gt; s = "resplit(regex, medstring)"\n&gt;&gt;&gt; t = timeit.Timer(s, "from re import split as resplit; regex=\'\\s+\'; medstring=\'a short sized string with spaces \'*100")\n&gt;&gt;&gt; print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)\n540.21 usec/pass\n</code></pre>\n\n<p>For very long strings you should not load the entire string into memory and instead either split the lines or use an iterative loop</p>\n'}, {'answer_id': 80449, 'author': 'elifiner', 'author_id': 15109, 'author_profile': 'https://Stackoverflow.com/users/15109', 'pm_score': 3, 'selected': False, 'text': '<p>Since this question is tagged with regex, I decided to try a regex approach. I first replace all the spaces in the quotes parts with \\x00, then split by spaces, then replace the \\x00 back to spaces in each part.</p>\n\n<p>Both versions do the same thing, but splitter is a bit more readable then splitter2.</p>\n\n<pre><code>import re\n\ns = \'this is "a test" some text "another test"\'\n\ndef splitter(s):\n    def replacer(m):\n        return m.group(0).replace(" ", "\\x00")\n    parts = re.sub(\'".+?"\', replacer, s).split()\n    parts = [p.replace("\\x00", " ") for p in parts]\n    return parts\n\ndef splitter2(s):\n    return [p.replace("\\x00", " ") for p in re.sub(\'".+?"\', lambda m: m.group(0).replace(" ", "\\x00"), s).split()]\n\nprint splitter2(s)\n</code></pre>\n'}, {'answer_id': 524796, 'author': 'Community', 'author_id': -1, 'author_profile': 'https://Stackoverflow.com/users/-1', 'pm_score': 6, 'selected': False, 'text': '<p>I see regex approaches here that look complex and/or wrong.  This surprises me, because regex syntax can easily describe "whitespace or thing-surrounded-by-quotes", and most regex engines (including Python\'s) can split on a regex.  So if you\'re going to use regexes, why not just say exactly what you mean?:</p>\n\n<pre><code>test = \'this is "a test"\'  # or "this is \'a test\'"\n# pieces = [p for p in re.split("( |[\\\\\\"\'].*[\\\\\\"\'])", test) if p.strip()]\n# From comments, use this:\npieces = [p for p in re.split("( |\\\\\\".*?\\\\\\"|\'.*?\')", test) if p.strip()]\n</code></pre>\n\n<p>Explanation:</p>\n\n<pre><code>[\\\\\\"\'] = double-quote or single-quote\n.* = anything\n( |X) = space or X\n.strip() = remove space and empty-string separators\n</code></pre>\n\n<p>shlex probably provides more features, though.</p>\n'}, {'answer_id': 525011, 'author': 'Ryan Ginstrom', 'author_id': 10658, 'author_profile': 'https://Stackoverflow.com/users/10658', 'pm_score': 5, 'selected': False, 'text': '<p>Depending on your use case, you may also want to check out the <a href="https://docs.python.org/library/csv.html" rel="noreferrer"><code>csv</code></a> module:</p>\n\n<pre><code>import csv\nlines = [\'this is "a string"\', \'and more "stuff"\']\nfor row in csv.reader(lines, delimiter=" "):\n    print(row)\n</code></pre>\n\n<p>Output: </p>\n\n<pre><code>[\'this\', \'is\', \'a string\']\n[\'and\', \'more\', \'stuff\']\n</code></pre>\n'}, {'answer_id': 2159337, 'author': 'Community', 'author_id': -1, 'author_profile': 'https://Stackoverflow.com/users/-1', 'pm_score': 1, 'selected': False, 'text': '<p>Hmm, can\'t seem to find the "Reply" button... anyway, this answer is based on the approach by Kate, but correctly splits strings with substrings containing escaped quotes and also removes the start and end quotes of the substrings:</p>\n\n<pre><code>  [i.strip(\'"\').strip("\'") for i in re.split(r\'(\\s+|(?&lt;!\\\\)".*?(?&lt;!\\\\)"|(?&lt;!\\\\)\\\'.*?(?&lt;!\\\\)\\\')\', string) if i.strip()]\n</code></pre>\n\n<p>This works on strings like <code>\'This is " a \\\\\\"test\\\\\\"\\\\\\\'s substring"\'</code> (the insane markup is unfortunately necessary to keep Python from removing the escapes).</p>\n\n<p>If the resulting escapes in the strings in the returned list are not wanted, you can use this slightly altered version of the function:</p>\n\n<pre><code>[i.strip(\'"\').strip("\'").decode(\'string_escape\') for i in re.split(r\'(\\s+|(?&lt;!\\\\)".*?(?&lt;!\\\\)"|(?&lt;!\\\\)\\\'.*?(?&lt;!\\\\)\\\')\', string) if i.strip()]\n</code></pre>\n'}, {'answer_id': 11194593, 'author': 'moschlar', 'author_id': 1175818, 'author_profile': 'https://Stackoverflow.com/users/1175818', 'pm_score': 1, 'selected': False, 'text': "<p>To get around the unicode issues in some Python 2 versions, I suggest:</p>\n\n<pre><code>from shlex import split as _split\nsplit = lambda a: [b.decode('utf-8') for b in _split(a.encode('utf-8'))]\n</code></pre>\n"}, {'answer_id': 23155180, 'author': 'Daniel Dai', 'author_id': 1089262, 'author_profile': 'https://Stackoverflow.com/users/1089262', 'pm_score': 4, 'selected': False, 'text': '<p>I use shlex.split to process 70,000,000 lines of squid log, it\'s so slow. So I switched to re.</p>\n\n<p>Please try this, if you have performance problem with shlex.</p>\n\n<pre><code>import re\n\ndef line_split(line):\n    return re.findall(r\'[^"\\s]\\S*|".+?"\', line)\n</code></pre>\n'}, {'answer_id': 32480710, 'author': 'hussic', 'author_id': 4111130, 'author_profile': 'https://Stackoverflow.com/users/4111130', 'pm_score': 0, 'selected': False, 'text': '<p>I suggest:</p>\n\n<p>test string:</p>\n\n<pre><code>s = \'abc "ad" \\\'fg\\\' "kk\\\'rdt\\\'" zzz"34"zzz "" \\\'\\\'\'\n</code></pre>\n\n<p>to capture also "" and \'\':</p>\n\n<pre><code>import re\nre.findall(r\'"[^"]*"|\\\'[^\\\']*\\\'|[^"\\\'\\s]+\',s)\n</code></pre>\n\n<p>result:</p>\n\n<pre><code>[\'abc\', \'"ad"\', "\'fg\'", \'"kk\\\'rdt\\\'"\', \'zzz\', \'"34"\', \'zzz\', \'""\', "\'\'"]\n</code></pre>\n\n<p>to ignore empty "" and \'\':</p>\n\n<pre><code>import re\nre.findall(r\'"[^"]+"|\\\'[^\\\']+\\\'|[^"\\\'\\s]+\',s)\n</code></pre>\n\n<p>result:</p>\n\n<pre><code>[\'abc\', \'"ad"\', "\'fg\'", \'"kk\\\'rdt\\\'"\', \'zzz\', \'"34"\', \'zzz\']\n</code></pre>\n'}, {'answer_id': 43035638, 'author': 'THE_MAD_KING', 'author_id': 7771160, 'author_profile': 'https://Stackoverflow.com/users/7771160', 'pm_score': 2, 'selected': False, 'text': '<p>To preserve quotes use this function:</p>\n\n<pre><code>def getArgs(s):\n    args = []\n    cur = \'\'\n    inQuotes = 0\n    for char in s.strip():\n        if char == \' \' and not inQuotes:\n            args.append(cur)\n            cur = \'\'\n        elif char == \'"\' and not inQuotes:\n            inQuotes = 1\n            cur += char\n        elif char == \'"\' and inQuotes:\n            inQuotes = 0\n            cur += char\n        else:\n            cur += char\n    args.append(cur)\n    return args\n</code></pre>\n'}, {'answer_id': 49791573, 'author': 'har777', 'author_id': 1851428, 'author_profile': 'https://Stackoverflow.com/users/1851428', 'pm_score': 3, 'selected': False, 'text': '<p>Speed test of different answers:</p>\n\n<pre><code>import re\nimport shlex\nimport csv\n\nline = \'this is "a test"\'\n\n%timeit [p for p in re.split("( |\\\\\\".*?\\\\\\"|\'.*?\')", line) if p.strip()]\n100000 loops, best of 3: 5.17 µs per loop\n\n%timeit re.findall(r\'[^"\\s]\\S*|".+?"\', line)\n100000 loops, best of 3: 2.88 µs per loop\n\n%timeit list(csv.reader([line], delimiter=" "))\nThe slowest run took 9.62 times longer than the fastest. This could mean that an intermediate result is being cached.\n100000 loops, best of 3: 2.4 µs per loop\n\n%timeit shlex.split(line)\n10000 loops, best of 3: 50.2 µs per loop\n</code></pre>\n'}, {'answer_id': 51560564, 'author': 'Ton van den Heuvel', 'author_id': 79111, 'author_profile': 'https://Stackoverflow.com/users/79111', 'pm_score': 3, 'selected': False, 'text': '<p>The main problem with the accepted <code>shlex</code> approach is that it does not ignore escape characters outside quoted substrings, and gives slightly unexpected results in some corner cases.</p>\n<p>I have the following use case, where I need a split function that splits input strings such that either single-quoted or double-quoted substrings are preserved, with the ability to escape quotes within such a substring. Quotes within an unquoted string should not be treated differently from any other character. Some example test cases with the expected output:</p>\n<pre> input string        | expected output\n===============================================\n \'abc def\'           | [\'abc\', \'def\']\n "abc \\\\s def"       | [\'abc\', \'\\\\s\', \'def\']\n \'"abc def" ghi\'     | [\'abc def\', \'ghi\']\n "\'abc def\' ghi"     | [\'abc def\', \'ghi\']\n \'"abc \\\\" def" ghi\' | [\'abc " def\', \'ghi\']\n "\'abc \\\\\' def\' ghi" | ["abc \' def", \'ghi\']\n "\'abc \\\\s def\' ghi" | [\'abc \\\\s def\', \'ghi\']\n \'"abc \\\\s def" ghi\' | [\'abc \\\\s def\', \'ghi\']\n \'"" test\'           | [\'\', \'test\']\n "\'\' test"           | [\'\', \'test\']\n "abc\'def"           | ["abc\'def"]\n "abc\'def\'"          | ["abc\'def\'"]\n "abc\'def\' ghi"      | ["abc\'def\'", \'ghi\']\n "abc\'def\'ghi"       | ["abc\'def\'ghi"]\n \'abc"def\'           | [\'abc"def\']\n \'abc"def"\'          | [\'abc"def"\']\n \'abc"def" ghi\'      | [\'abc"def"\', \'ghi\']\n \'abc"def"ghi\'       | [\'abc"def"ghi\']\n "r\'AA\' r\'.*_xyz$\'"  | ["r\'AA\'", "r\'.*_xyz$\'"]\n \'abc"def ghi"\'      | [\'abc"def ghi"\']\n \'abc"def ghi""jkl"\' | [\'abc"def ghi""jkl"\']\n \'a"b c"d"e"f"g h"\'  | [\'a"b c"d"e"f"g h"\']\n \'c="ls /" type key\' | [\'c="ls /"\', \'type\', \'key\']\n "abc\'def ghi\'"      | ["abc\'def ghi\'"]\n "c=\'ls /\' type key" | ["c=\'ls /\'", \'type\', \'key\']</pre>\n<p>I ended up with the following function to split a string such that the expected output results for all input strings:</p>\n<pre><code>import re\n\ndef quoted_split(s):\n    def strip_quotes(s):\n        if s and (s[0] == \'"\' or s[0] == "\'") and s[0] == s[-1]:\n            return s[1:-1]\n        return s\n    return [strip_quotes(p).replace(\'\\\\"\', \'"\').replace("\\\\\'", "\'") \\\n            for p in re.findall(r\'(?:[^"\\s]*"(?:\\\\.|[^"])*"[^"\\s]*)+|(?:[^\\\'\\s]*\\\'(?:\\\\.|[^\\\'])*\\\'[^\\\'\\s]*)+|[^\\s]+\', s)]\n</code></pre>\n<p>It ain\'t pretty; but it works. The following test application checks the results of other approaches (<code>shlex</code> and <code>csv</code> for now) and the custom split implementation:</p>\n<pre><code>#!/bin/python2.7\n\nimport csv\nimport re\nimport shlex\n\nfrom timeit import timeit\n\ndef test_case(fn, s, expected):\n    try:\n        if fn(s) == expected:\n            print \'[ OK ] %s -> %s\' % (s, fn(s))\n        else:\n            print \'[FAIL] %s -> %s\' % (s, fn(s))\n    except Exception as e:\n        print \'[FAIL] %s -> exception: %s\' % (s, e)\n\ndef test_case_no_output(fn, s, expected):\n    try:\n        fn(s)\n    except:\n        pass\n\ndef test_split(fn, test_case_fn=test_case):\n    test_case_fn(fn, \'abc def\', [\'abc\', \'def\'])\n    test_case_fn(fn, "abc \\\\s def", [\'abc\', \'\\\\s\', \'def\'])\n    test_case_fn(fn, \'"abc def" ghi\', [\'abc def\', \'ghi\'])\n    test_case_fn(fn, "\'abc def\' ghi", [\'abc def\', \'ghi\'])\n    test_case_fn(fn, \'"abc \\\\" def" ghi\', [\'abc " def\', \'ghi\'])\n    test_case_fn(fn, "\'abc \\\\\' def\' ghi", ["abc \' def", \'ghi\'])\n    test_case_fn(fn, "\'abc \\\\s def\' ghi", [\'abc \\\\s def\', \'ghi\'])\n    test_case_fn(fn, \'"abc \\\\s def" ghi\', [\'abc \\\\s def\', \'ghi\'])\n    test_case_fn(fn, \'"" test\', [\'\', \'test\'])\n    test_case_fn(fn, "\'\' test", [\'\', \'test\'])\n    test_case_fn(fn, "abc\'def", ["abc\'def"])\n    test_case_fn(fn, "abc\'def\'", ["abc\'def\'"])\n    test_case_fn(fn, "abc\'def\' ghi", ["abc\'def\'", \'ghi\'])\n    test_case_fn(fn, "abc\'def\'ghi", ["abc\'def\'ghi"])\n    test_case_fn(fn, \'abc"def\', [\'abc"def\'])\n    test_case_fn(fn, \'abc"def"\', [\'abc"def"\'])\n    test_case_fn(fn, \'abc"def" ghi\', [\'abc"def"\', \'ghi\'])\n    test_case_fn(fn, \'abc"def"ghi\', [\'abc"def"ghi\'])\n    test_case_fn(fn, "r\'AA\' r\'.*_xyz$\'", ["r\'AA\'", "r\'.*_xyz$\'"])\n    test_case_fn(fn, \'abc"def ghi"\', [\'abc"def ghi"\'])\n    test_case_fn(fn, \'abc"def ghi""jkl"\', [\'abc"def ghi""jkl"\'])\n    test_case_fn(fn, \'a"b c"d"e"f"g h"\', [\'a"b c"d"e"f"g h"\'])\n    test_case_fn(fn, \'c="ls /" type key\', [\'c="ls /"\', \'type\', \'key\'])\n    test_case_fn(fn, "abc\'def ghi\'", ["abc\'def ghi\'"])\n    test_case_fn(fn, "c=\'ls /\' type key", ["c=\'ls /\'", \'type\', \'key\'])\n\ndef csv_split(s):\n    return list(csv.reader([s], delimiter=\' \'))[0]\n\ndef re_split(s):\n    def strip_quotes(s):\n        if s and (s[0] == \'"\' or s[0] == "\'") and s[0] == s[-1]:\n            return s[1:-1]\n        return s\n    return [strip_quotes(p).replace(\'\\\\"\', \'"\').replace("\\\\\'", "\'") for p in re.findall(r\'(?:[^"\\s]*"(?:\\\\.|[^"])*"[^"\\s]*)+|(?:[^\\\'\\s]*\\\'(?:\\\\.|[^\\\'])*\\\'[^\\\'\\s]*)+|[^\\s]+\', s)]\n\nif __name__ == \'__main__\':\n    print \'shlex\\n\'\n    test_split(shlex.split)\n    print\n\n    print \'csv\\n\'\n    test_split(csv_split)\n    print\n\n    print \'re\\n\'\n    test_split(re_split)\n    print\n\n    iterations = 100\n    setup = \'from __main__ import test_split, test_case_no_output, csv_split, re_split\\nimport shlex, re\'\n    def benchmark(method, code):\n        print \'%s: %.3fms per iteration\' % (method, (1000 * timeit(code, setup=setup, number=iterations) / iterations))\n    benchmark(\'shlex\', \'test_split(shlex.split, test_case_no_output)\')\n    benchmark(\'csv\', \'test_split(csv_split, test_case_no_output)\')\n    benchmark(\'re\', \'test_split(re_split, test_case_no_output)\')\n</code></pre>\n<p>Output:</p>\n<pre>\nshlex\n\n[ OK ] abc def -> [\'abc\', \'def\']\n[FAIL] abc \\s def -> [\'abc\', \'s\', \'def\']\n[ OK ] "abc def" ghi -> [\'abc def\', \'ghi\']\n[ OK ] \'abc def\' ghi -> [\'abc def\', \'ghi\']\n[ OK ] "abc \\" def" ghi -> [\'abc " def\', \'ghi\']\n[FAIL] \'abc \\\' def\' ghi -> exception: No closing quotation\n[ OK ] \'abc \\s def\' ghi -> [\'abc \\\\s def\', \'ghi\']\n[ OK ] "abc \\s def" ghi -> [\'abc \\\\s def\', \'ghi\']\n[ OK ] "" test -> [\'\', \'test\']\n[ OK ] \'\' test -> [\'\', \'test\']\n[FAIL] abc\'def -> exception: No closing quotation\n[FAIL] abc\'def\' -> [\'abcdef\']\n[FAIL] abc\'def\' ghi -> [\'abcdef\', \'ghi\']\n[FAIL] abc\'def\'ghi -> [\'abcdefghi\']\n[FAIL] abc"def -> exception: No closing quotation\n[FAIL] abc"def" -> [\'abcdef\']\n[FAIL] abc"def" ghi -> [\'abcdef\', \'ghi\']\n[FAIL] abc"def"ghi -> [\'abcdefghi\']\n[FAIL] r\'AA\' r\'.*_xyz$\' -> [\'rAA\', \'r.*_xyz$\']\n[FAIL] abc"def ghi" -> [\'abcdef ghi\']\n[FAIL] abc"def ghi""jkl" -> [\'abcdef ghijkl\']\n[FAIL] a"b c"d"e"f"g h" -> [\'ab cdefg h\']\n[FAIL] c="ls /" type key -> [\'c=ls /\', \'type\', \'key\']\n[FAIL] abc\'def ghi\' -> [\'abcdef ghi\']\n[FAIL] c=\'ls /\' type key -> [\'c=ls /\', \'type\', \'key\']\n\ncsv\n\n[ OK ] abc def -> [\'abc\', \'def\']\n[ OK ] abc \\s def -> [\'abc\', \'\\\\s\', \'def\']\n[ OK ] "abc def" ghi -> [\'abc def\', \'ghi\']\n[FAIL] \'abc def\' ghi -> ["\'abc", "def\'", \'ghi\']\n[FAIL] "abc \\" def" ghi -> [\'abc \\\\\', \'def"\', \'ghi\']\n[FAIL] \'abc \\\' def\' ghi -> ["\'abc", "\\\\\'", "def\'", \'ghi\']\n[FAIL] \'abc \\s def\' ghi -> ["\'abc", \'\\\\s\', "def\'", \'ghi\']\n[ OK ] "abc \\s def" ghi -> [\'abc \\\\s def\', \'ghi\']\n[ OK ] "" test -> [\'\', \'test\']\n[FAIL] \'\' test -> ["\'\'", \'test\']\n[ OK ] abc\'def -> ["abc\'def"]\n[ OK ] abc\'def\' -> ["abc\'def\'"]\n[ OK ] abc\'def\' ghi -> ["abc\'def\'", \'ghi\']\n[ OK ] abc\'def\'ghi -> ["abc\'def\'ghi"]\n[ OK ] abc"def -> [\'abc"def\']\n[ OK ] abc"def" -> [\'abc"def"\']\n[ OK ] abc"def" ghi -> [\'abc"def"\', \'ghi\']\n[ OK ] abc"def"ghi -> [\'abc"def"ghi\']\n[ OK ] r\'AA\' r\'.*_xyz$\' -> ["r\'AA\'", "r\'.*_xyz$\'"]\n[FAIL] abc"def ghi" -> [\'abc"def\', \'ghi"\']\n[FAIL] abc"def ghi""jkl" -> [\'abc"def\', \'ghi""jkl"\']\n[FAIL] a"b c"d"e"f"g h" -> [\'a"b\', \'c"d"e"f"g\', \'h"\']\n[FAIL] c="ls /" type key -> [\'c="ls\', \'/"\', \'type\', \'key\']\n[FAIL] abc\'def ghi\' -> ["abc\'def", "ghi\'"]\n[FAIL] c=\'ls /\' type key -> ["c=\'ls", "/\'", \'type\', \'key\']\n\nre\n\n[ OK ] abc def -> [\'abc\', \'def\']\n[ OK ] abc \\s def -> [\'abc\', \'\\\\s\', \'def\']\n[ OK ] "abc def" ghi -> [\'abc def\', \'ghi\']\n[ OK ] \'abc def\' ghi -> [\'abc def\', \'ghi\']\n[ OK ] "abc \\" def" ghi -> [\'abc " def\', \'ghi\']\n[ OK ] \'abc \\\' def\' ghi -> ["abc \' def", \'ghi\']\n[ OK ] \'abc \\s def\' ghi -> [\'abc \\\\s def\', \'ghi\']\n[ OK ] "abc \\s def" ghi -> [\'abc \\\\s def\', \'ghi\']\n[ OK ] "" test -> [\'\', \'test\']\n[ OK ] \'\' test -> [\'\', \'test\']\n[ OK ] abc\'def -> ["abc\'def"]\n[ OK ] abc\'def\' -> ["abc\'def\'"]\n[ OK ] abc\'def\' ghi -> ["abc\'def\'", \'ghi\']\n[ OK ] abc\'def\'ghi -> ["abc\'def\'ghi"]\n[ OK ] abc"def -> [\'abc"def\']\n[ OK ] abc"def" -> [\'abc"def"\']\n[ OK ] abc"def" ghi -> [\'abc"def"\', \'ghi\']\n[ OK ] abc"def"ghi -> [\'abc"def"ghi\']\n[ OK ] r\'AA\' r\'.*_xyz$\' -> ["r\'AA\'", "r\'.*_xyz$\'"]\n[ OK ] abc"def ghi" -> [\'abc"def ghi"\']\n[ OK ] abc"def ghi""jkl" -> [\'abc"def ghi""jkl"\']\n[ OK ] a"b c"d"e"f"g h" -> [\'a"b c"d"e"f"g h"\']\n[ OK ] c="ls /" type key -> [\'c="ls /"\', \'type\', \'key\']\n[ OK ] abc\'def ghi\' -> ["abc\'def ghi\'"]\n[ OK ] c=\'ls /\' type key -> ["c=\'ls /\'", \'type\', \'key\']\n\nshlex: 0.335ms per iteration\ncsv: 0.036ms per iteration\nre: 0.068ms per iteration\n</pre>\n<p>So performance is much better than <code>shlex</code>, and can be improved further by precompiling the regular expression, in which case it will outperform the <code>csv</code> approach.</p>\n'}, {'answer_id': 53210803, 'author': 'hochl', 'author_id': 589206, 'author_profile': 'https://Stackoverflow.com/users/589206', 'pm_score': 4, 'selected': False, 'text': '<p>It seems that for performance reasons <code>re</code> is faster. Here is my solution using a least greedy operator that preserves the outer quotes:</p>\n\n<pre><code>re.findall("(?:\\".*?\\"|\\S)+", s)\n</code></pre>\n\n<p>Result:</p>\n\n<pre><code>[\'this\', \'is\', \'"a test"\']\n</code></pre>\n\n<p>It leaves constructs like <code>aaa"bla blub"bbb</code> together as these tokens are not separated by spaces. If the string contains escaped characters, you can match like that:</p>\n\n<pre><code>&gt;&gt;&gt; a = "She said \\"He said, \\\\\\"My name is Mark.\\\\\\"\\""\n&gt;&gt;&gt; a\n\'She said "He said, \\\\"My name is Mark.\\\\""\'\n&gt;&gt;&gt; for i in re.findall("(?:\\".*?[^\\\\\\\\]\\"|\\S)+", a): print(i)\n...\nShe\nsaid\n"He said, \\"My name is Mark.\\""\n</code></pre>\n\n<p>Please note that this also matches the empty string <code>""</code> by means of the <code>\\S</code> part of the pattern.</p>\n'}, {'answer_id': 60929888, 'author': 'Mikhail Zakharov', 'author_id': 9127614, 'author_profile': 'https://Stackoverflow.com/users/9127614', 'pm_score': 1, 'selected': False, 'text': '<p>As an option try tssplit:</p>\n\n<pre><code>In [1]: from tssplit import tssplit\nIn [2]: tssplit(\'this is "a test"\', quote=\'"\', delimiter=\'\')\nOut[2]: [\'this\', \'is\', \'a test\']\n</code></pre>\n'}]
['https://Stackoverflow.com/questions/79992', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/10972/']