<p>When using os.system() it's often necessary to escape filenames and other arguments passed as parameters to commands.  How can I do this?  Preferably something that would work on multiple operating systems/shells but in particular for bash.</p>

<p>I'm currently doing the following, but am sure there must be a library function for this, or at least a more elegant/robust/efficient option:</p>

<pre><code>def sh_escape(s):
   return s.replace("(","\\(").replace(")","\\)").replace(" ","\\ ")

os.system("cat %s | grep something | sort &gt; %s" 
          % (sh_escape(in_filename), 
             sh_escape(out_filename)))
</code></pre>

<p><em>Edit:</em> I've accepted the simple answer of using quotes, don't know why I didn't think of that; I guess because I came from Windows where ' and " behave a little differently.</p>

<p>Regarding security, I understand the concern, but, in this case, I'm interested in a quick and easy solution which os.system() provides, and the source of the strings is either not user-generated or at least entered by a trusted user (me).</p>

[{'answer_id': 35831, 'author': 'pauldoo', 'author_id': 755, 'author_profile': 'https://Stackoverflow.com/users/755', 'pm_score': 2, 'selected': False, 'text': "<p>I believe that os.system just invokes whatever command shell is configured for the user, so I don't think you can do it in a platform independent way.  My command shell could be anything from bash, emacs, ruby, or even quake3.  Some of these programs aren't expecting the kind of arguments you are passing to them and even if they did there is no guarantee they do their escaping the same way.</p>\n"}, {'answer_id': 35857, 'author': 'Greg Hewgill', 'author_id': 893, 'author_profile': 'https://Stackoverflow.com/users/893', 'pm_score': 8, 'selected': True, 'text': '<p>This is what I use:</p>\n\n<pre><code>def shellquote(s):\n    return "\'" + s.replace("\'", "\'\\\\\'\'") + "\'"\n</code></pre>\n\n<p>The shell will always accept a quoted filename and remove the surrounding quotes before passing it to the program in question. Notably, this avoids problems with filenames that contain spaces or any other kind of nasty shell metacharacter.</p>\n\n<p><strong>Update</strong>: If you are using Python 3.3 or later, use <a href="http://docs.python.org/dev/library/shlex.html#shlex.quote" rel="noreferrer">shlex.quote</a> instead of rolling your own.</p>\n'}, {'answer_id': 35858, 'author': 'Jamie', 'author_id': 3363, 'author_profile': 'https://Stackoverflow.com/users/3363', 'pm_score': 6, 'selected': False, 'text': '<p>Perhaps you have a specific reason for using <code>os.system()</code>. But if not you should probably be using the <a href="https://docs.python.org/library/subprocess.html" rel="noreferrer"><code>subprocess</code> module</a>. You can specify the pipes directly and avoid using the shell.</p>\n\n<p>The following is from <a href="http://www.python.org/dev/peps/pep-0324/" rel="noreferrer">PEP324</a>:</p>\n\n<blockquote>\n<pre><code>Replacing shell pipe line\n-------------------------\n\noutput=`dmesg | grep hda`\n==&gt;\np1 = Popen(["dmesg"], stdout=PIPE)\np2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)\noutput = p2.communicate()[0]\n</code></pre>\n</blockquote>\n'}, {'answer_id': 847800, 'author': 'pixelbeat', 'author_id': 4421, 'author_profile': 'https://Stackoverflow.com/users/4421', 'pm_score': 7, 'selected': False, 'text': '<p><a href="https://docs.python.org/3/library/shlex.html#shlex.quote" rel="nofollow noreferrer"><code>shlex.quote()</code></a> does what you want since python 3.</p>\n<p>(Use <a href="https://docs.python.org/2/library/pipes.html#pipes.quote" rel="nofollow noreferrer"><code>pipes.quote</code></a> to support both python 2 and python 3,\nthough note that <code>pipes</code> has been deprecated since 3.10\nand slated for removal in 3.13)</p>\n'}, {'answer_id': 1884718, 'author': 'John Wiseman', 'author_id': 122762, 'author_profile': 'https://Stackoverflow.com/users/122762', 'pm_score': 3, 'selected': False, 'text': '<p>Note that pipes.quote is actually broken in Python 2.5 and Python 3.1 and not safe to use--It doesn\'t handle zero-length arguments.</p>\n\n<pre><code>&gt;&gt;&gt; from pipes import quote\n&gt;&gt;&gt; args = [\'arg1\', \'\', \'arg3\']\n&gt;&gt;&gt; print \'mycommand %s\' % (\' \'.join(quote(arg) for arg in args))\nmycommand arg1  arg3\n</code></pre>\n\n<p>See <a href="http://bugs.python.org/issue7476" rel="nofollow noreferrer">Python issue 7476</a>; it has been fixed in Python 2.6 and 3.2 and newer.</p>\n'}, {'answer_id': 3851646, 'author': 'tzot', 'author_id': 6899, 'author_profile': 'https://Stackoverflow.com/users/6899', 'pm_score': 1, 'selected': False, 'text': '<p>The function I use is:</p>\n\n<pre><code>def quote_argument(argument):\n    return \'"%s"\' % (\n        argument\n        .replace(\'\\\\\', \'\\\\\\\\\')\n        .replace(\'"\', \'\\\\"\')\n        .replace(\'$\', \'\\\\$\')\n        .replace(\'`\', \'\\\\`\')\n    )\n</code></pre>\n\n<p>that is: I always enclose the argument in double quotes, and then backslash-quote the only characters special inside double quotes.</p>\n'}, {'answer_id': 10750633, 'author': 'Gary Shi', 'author_id': 626160, 'author_profile': 'https://Stackoverflow.com/users/626160', 'pm_score': 4, 'selected': False, 'text': '<p>Maybe <code>subprocess.list2cmdline</code> is a better shot?</p>\n'}, {'answer_id': 29597408, 'author': 'Rockallite', 'author_id': 2293304, 'author_profile': 'https://Stackoverflow.com/users/2293304', 'pm_score': 2, 'selected': False, 'text': '<p><strong>Notice</strong>: This is an answer for Python 2.7.x.</p>\n\n<p>According to the <a href="https://hg.python.org/cpython/file/2.7/Lib/pipes.py#l262" rel="nofollow noreferrer">source</a>, <code>pipes.quote()</code> is a way to "<em>Reliably quote a string as a single argument for <strong>/bin/sh</em></strong>". (Although it is <a href="https://docs.python.org/2/library/pipes.html#pipes.quote" rel="nofollow noreferrer">deprecated since version 2.7</a> and finally exposed publicly in Python 3.3 as the <code>shlex.quote()</code> function.)</p>\n\n<p>On <a href="https://hg.python.org/cpython/file/2.7/Lib/subprocess.py#l577" rel="nofollow noreferrer">the other hand</a>, <code>subprocess.list2cmdline()</code> is a way to "<em>Translate a sequence of arguments into a command line string, using the same rules as the <strong>MS C runtime</em></strong>".</p>\n\n<p>Here we are, the platform independent way of quoting strings for command lines.</p>\n\n<pre><code>import sys\nmswindows = (sys.platform == "win32")\n\nif mswindows:\n    from subprocess import list2cmdline\n    quote_args = list2cmdline\nelse:\n    # POSIX\n    from pipes import quote\n\n    def quote_args(seq):\n        return \' \'.join(quote(arg) for arg in seq)\n</code></pre>\n\n<p>Usage:</p>\n\n<pre><code># Quote a single argument\nprint quote_args([\'my argument\'])\n\n# Quote multiple arguments\nmy_args = [\'This\', \'is\', \'my arguments\']\nprint quote_args(my_args)\n</code></pre>\n'}, {'answer_id': 70669446, 'author': 'Flimm', 'author_id': 247696, 'author_profile': 'https://Stackoverflow.com/users/247696', 'pm_score': 0, 'selected': False, 'text': "<p>On UNIX shells like Bash, you can use <code>shlex.quote</code> in Python 3 to escape special characters that the shell might interpret, like whitespace and the <code>*</code> character:</p>\n<pre><code>import os\nimport shlex\n\nos.system(&quot;rm &quot; + shlex.quote(filename))\n</code></pre>\n<p>However, this is not enough for security purposes! You still need to be careful that the command argument is not interpreted in unintended ways. For example, what if the filename is actually a path like <code>../../etc/passwd</code>? Running <code>os.system(&quot;rm &quot; + shlex.quote(filename))</code> might delete <code>/etc/passwd</code> when you only expected it to delete filenames found in the current directory! The issue here isn't with the shell interpreting special characters, it's that the filename argument isn't interpreted by the <code>rm</code> as a simple filename, it's actually interpreted as a path.</p>\n<p>Or what if the valid filename starts with a dash, for example, <code>-f</code>? It's not enough to merely pass the escaped filename, you  need to disable options using <code>--</code> or you need to pass a path that doesn't begin with a dash like <code>./-f</code>. The issue here isn't with the shell interpreting special characters, it's that the <code>rm</code> command interprets the argument as a filename <em>or</em> a path <em>or</em> an option if it begins with a dash.</p>\n<p>Here is a safer implementation:</p>\n<pre><code>if os.sep in filename:\n     raise Exception(&quot;Did not expect to find file path separator in file name&quot;)\n\nos.system(&quot;rm -- &quot; + shlex.quote(filename))\n</code></pre>\n"}, {'answer_id': 72750573, 'author': 'Matthew Roberts', 'author_id': 4276963, 'author_profile': 'https://Stackoverflow.com/users/4276963', 'pm_score': 0, 'selected': False, 'text': '<p>I think these answers are a bad idea for escaping command-line arguments on Windows. Based on the results: people are trying to apply a black-list approach to filtering \'bad\' characters, assuming (and hoping) they got them all. Windows is very complex and there could be all manner of characters found in the future that might allow an attacker to hijack command line arguments.</p>\n<p>I\'ve already seen some answers neglect to filter basic meta-characters in Windows (like the semi-colon.) The approach I take is far simpler:</p>\n<ol>\n<li>Make a list of allowed ASCII characters.</li>\n<li>Remove all chars that aren\'t in that list.</li>\n<li>Escape slashes and double-quotes.</li>\n<li>Surround entire command with double quotes so the command argument cannot be maliciously broken and commandeered with spaces.</li>\n</ol>\n<p>A basic example:</p>\n<pre class="lang-py prettyprint-override"><code>\ndef win_arg_escape(arg, allow_vars=0):\n    allowed_list = &quot;&quot;&quot;\'&quot;/\\\\abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-. &quot;&quot;&quot;\n    if allow_vars:\n        allowed_list += &quot;~%$&quot;\n\n    # Filter out anything that isn\'t a\n    # standard character.\n    buf = &quot;&quot;\n    for ch in arg:\n        if ch in allowed_list:\n            buf += ch\n\n    # Escape all slashes.\n    buf = buf.replace(&quot;\\\\&quot;, &quot;\\\\\\\\&quot;)\n\n    # Escape double quotes.\n    buf = buf.replace(\'&quot;\', \'&quot;&quot;\')\n\n    # Surround entire arg with quotes.\n    # This avoids spaces breaking a command.\n    buf = \'&quot;%s&quot;\' % (buf)\n\n    return buf\n\n</code></pre>\n<p>The function has an option to enable use of environmental variables and other shell variables. Enabling this poses more risk so its disabled by default.</p>\n'}]
['https://Stackoverflow.com/questions/35817', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/3715/']