Read a python variable in a shell script?
Question: my python file has these 2 variables:

    
    
    week_date = "01/03/16-01/09/16"
    cust_id = "12345"
    

how can i read this into a shell script that takes in these 2 variables?

my current shell script requires manual editing of "dt" and "id". I want to
read the python variables into the shell script so i can just edit my python
parameter file and not so many files.

shell file:

    
    
    #!/bin/sh
    
    dt="01/03/16-01/09/16" 
    cust_id="12345"
    

In a new python file i could just import the parameter python file.


Answer: Consider something akin to the following:

    
    
    #!/bin/bash
    #      ^^^^ NOT /bin/sh, which doesn't have process substitution available.
    
    python_script='
    import sys
    d = {}                                    # create a context for variables
    exec(open(sys.argv[1], "r").read()) in d  # execute the Python code in that context
    for k in sys.argv[2:]:
      print "%s\0" % str(d[k]).split("\0")[0] # ...and extract your strings NUL-delimited
    '
    
    read_python_vars() {
      local python_file=$1; shift
      local varname
      for varname; do
        IFS= read -r -d '' "${varname#*:}"
      done < <(python -c "$python_script" "$python_file" "${@%%:*}")
    }
    

You might then use this as:

    
    
    read_python_vars config.py week_date:dt cust_id:id
    echo "Customer id is $id; date range is $dt"
    

...or, if you didn't want to rename the variables as they were read, simply:

    
    
    read_python_vars config.py week_date cust_id
    echo "Customer id is $cust_id; date range is $week_date"
    

* * *

Advantages:

  * Unlike a naive regex-based solution (which would have trouble with some of the details of Python parsing -- try teaching `sed` to handle both raw and regular strings, and both single and triple quotes without making it into a hairball!) or a similar approach that used newline-delimited output from the Python subprocess, this will correctly handle any object for which `str()` gives a representation with no NUL characters that your shell script can use.
  * Running content through the Python interpreter also means you can determine values programmatically -- for instance, you could have some Python code that asks your version control system for the last-change-date of relevant content.

Think about scenarios such as this one:

    
        start_date = '01/03/16'
    end_date = '01/09/16'
    week_date = '%s-%s' % (start_date, end_date)
    

...using a Python interpreter to parse Python means you aren't restricting how
people can update/modify your Python config file in the future.

Now, let's talk caveats:

  * If your Python code has side effects, those side effects will obviously take effect (just as they would if you chose to `import` the file as a module in Python). Don't use this to extract configuration from a file whose contents you don't trust.
  * Python strings are Pascal-style: They can contain literal NULs. Strings in shell languages are C-style: They're terminated by the first NUL character. Thus, some variables can exist in Python than cannot be represented in shell without nonliteral escaping. To prevent an object whose `str()` representation contains NULs from spilling forward into other assignments, this code terminates strings at their first NUL.

* * *

Now, let's talk about implementation details.

  * `${@%%:*}` is an expansion of `$@` which trims all content after and including the first `:` in each argument, thus passing only the Python variable names to the interpreter. Similarly, `${varname#*:}` is an expansion which trims everything up to and including the first `:` from the variable name passed to `read`. See [the bash-hackers page on parameter expansion](http://wiki.bash-hackers.org/syntax/pe).
  * Using `<(python ...)` is process substitution syntax: The `<(...)` expression evaluates to a filename which, when read, will provide output of that command. Using `< <(...)` redirects output from that file, and thus that command (the first `<` is a redirection, whereas the second is part of the `<(` token that starts a process substitution). Using this form to get output into a `while read` loop avoids the bug mentioned in [BashFAQ #24 ("I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?")](http://mywiki.wooledge.org/BashFAQ/024).
  * The `IFS= read -r -d ''` construct has a series of components, each of which makes the behavior of `read` more true to the original content:

    * Clearing `IFS` for the duration of the command prevents whitespace from being trimmed from the end of the variable's content.
    * Using `-r` prevents literal backslashes from being consumed by `read` itself rather than represented in the output.
    * Using `-d ''` sets the first character of the empty string `''` to be the record delimiter. Since C strings are NUL-terminated and the shell uses C strings, that character is a NUL. This ensures that variables' content can contain any non-NUL value, including literal newlines.

See [BashFAQ #001 ("How can I read a file (data stream, variable) line-by-line
(and/or field-by-field)?")](http://mywiki.wooledge.org/BashFAQ/001) for more
on the process of reading record-oriented data from a string in bash.

