%sosdict

"""
Explanation: Using SoS with iPython
Using sos magic within iPython allows you to run sos within an iPython session. It does not provide a full-blown SoS system but on the other hand you can use all the iPython features that you are familiar with.
Setting up ipython
sos magic is installed by default when you install sos. A profile has been created for you so that you can use it by command
ipython --profile sos
You can also load the extension using command %load_ext sos_magic after ipython starts in its default profile, or edit ~/.ipython/profile_default/ipython_config.py and add sos_magic to c.InteractiveShellApp.extensions so that the extension is loaded automatically to your default profile.
SoS ipython magics
Magic sosdict
Let us starts with the basic. When you start an ipython interactive shell through ipython command or ipython/Jupyter notebook, you are given a cell that you can enter python expression or statements, or iPython magic. There are two kinds of magics

Line magic that starts with % or starts at the beginning of line without % if automagic is set to True (default). Line magic takes the words after it as parameter.
Cell magic that starts with %% that takes both the words and lines after the magic word.

If the sos_magic extension is loaded, it already has a SoS dictionary. You can get the dictionary using line magic %sosdict, for example
End of explanation
"""


%sos a = 1

%sosdict

"""
Explanation: The dictionary is empty because we have not assigned anything to it. Let us run a sos statement
End of explanation
"""


d = %sosdict

d.keys()

"""
Explanation: and you can see the sos dictionary contains one item. There are other usages of the %sosdict magic, you can get the dictionary by assigning the dictionary to a variable
End of explanation
"""


d = %sosdict a
d.keys()

"""
Explanation: If you are interested in only a subset of variables, you can list them after %sosdict
End of explanation
"""


%sosdict --keys

"""
Explanation: You can get the keys of the dictionary easier using
End of explanation
"""


%sosdict --reset
%sosdict

"""
Explanation: If after a while you would like to reset the dictionary and run a SoS script from fresh, you can reset the dictionary using option --reset
End of explanation
"""


%sosdict --keys --all

"""
Explanation: The SoS dictionary actually contains other items such as all the SoS actions and functions. If you would like to see all of them, use option --all. For example,
End of explanation
"""


%sos a=10

%sos "a + 100 = ${{a+100}}"

"""
Explanation: In summary, the %sosdict magic accepts
%sosdict [-a|-all] [-k|--keys] [-r|--reset] [var1] [var2] ...
where

var1, var2 etc are name of variables. All variables will be displayed if no variable is specified.
-a|-all: list all dictionary keys, including SoS functions and variables.
-k|--keys: list only keys, not their values
-r|--reset: reset the dictionary to its original content (with only SoS internal values)

Magic sos
Magic sos can be either a line magic or cell magic. Using sos as a line magic, it simply executes the SoS (python) expression or statement in SoS. For example,
End of explanation
"""


%sos_options sigil='` `'

%sos "a + 100 = `1+100`"

%sos b=['file1.txt', 'file2.txt']

%sos "`b!r,`"

"""
Explanation: Here we use SoS string interpolation to evaluate an expression a+100 and return its string representation. The sigil is supposed to be ${ } but it has to be used as ${{ }} because iPython has its own interpolation system that tries to evaluate anything in { }. This can be annoying but you can try to change the SoS default sigil using a %sos_options magic
End of explanation
"""


%sos_options sigil='${ }'

%sos name = 'Bob Kenny'
%sos "My name is ${name}"

"""
Explanation: Note that iPython will leave { } alone if it does not understand the content inside, so you are usually ok if the variable is not recognizable by iPython
End of explanation
"""


%%sos
run:
    echo "something"

"""
Explanation: If you would like to execute multi-line SoS statements or scripts, you will need to use the magic in cell mode (with %% prefix), for example,
End of explanation
"""


%%sos --rep 10
resource   = '~/resource'
ref_genome ="${resource}/hg19"
parameter: rep = 5

%sos rep

"""
Explanation: runs a shell script within iPython. Similarly, you can run arbitrary shell, R, perl, python code in ipython/SoS, with string interpolation. Note that ipython already has a magic called %%script that allows you to execute scripts in a cell, while SoS provides a lot more features.
For example, the %%sos treats its content as a complete SoS script and accepts command line arguments.
End of explanation
"""


%sos a = 20

%sos b = "a**2 = `a**2`"

%sosget a b

a

b

b = 'something else'

%sosput b

%sosdict b

"""
Explanation: Because rep is defined as a command line parameter, the command line option --rep 10 overrides its default value 5.
Magic sospaste
If you are using a terminal based iPython session (instead of the Jupyter notebook with iPython kernel that you are seeing right now), you will soon notice a problem with the cell magic %%sos in that it does not accept blank new lines, which is problematic for large piece of code. In this case, you can use line magic %sospaste to read the content directly from clipboard. Using the same example, you can select the text (with newline), and run
```python
In [5]: sospaste --rep 10
resource   = '~/resources'
ref_genome = '${resource}/hg19'
parameter: rep = 5
-- End pasted text --
```
This is the most convenient way to execute pieces of SoS script in iPython and is used most frequently.
Magics sosget and sosput
Magics sosget and sosput are used to exchange variables between ipython and sos. For example, if you defined a variable in SoS and would like to use it in ipython, you can use magic %sosget var1 var2 to copy the value of var1 and var2 from SoS dictionary to ipython namespace. Similarly, %sosput copies variables from the ipython namespace to SoS dictionary. For example,
End of explanation
"""


%sosset -v 3

"""
Explanation: Magic sosset
If you are tired of entering certain SoS options after magic %%sos or %sospaste, you can set these options persistently using magic sosset. For example, if you set
End of explanation
"""


%sosset

"""
Explanation: You can run the magic %sosset without option to get a list of acceptable options.
End of explanation
"""


%sos_options sigil='${ }'

%sosset -v1

"""
Explanation: A complete example
Let us reset the default sigil and try to use iPython sos magics to debug a SoS script.
End of explanation
"""


%%sos
resource   = '~/resources'
ref_genome = '${resource}/hg19'

"""
Explanation: The script currently looks like
```python
resource   = '~/resources'
ref_genome = '${resource}/hg19'
parameter: rep = 5
[1]
print(rep)
seq = range(rep)
input:  for_each='seq'
python:
  import time
  print('sleep {} seconds.'.format(_seq))
  time.sleep(_seq)
```
To check the global definition, you can copy and paste the definitions to ipython as
End of explanation
"""


%sos ref_genome

"""
Explanation: The statements are executed and you can check the result using
End of explanation
"""


%%sos --rep 3

resource   = '~/resources'
ref_genome = "${resource}/hg19"

parameter: rep = 5

%sos ref_genome

%sos rep

"""
Explanation: resource is not interpolated because the expression is quoted using single quotes. To use SoS string interpolation, we will need to quote the string in double quotes
End of explanation
"""


%%sos
[1]
print(rep)
seq = range(rep)
input:  for_each='seq'

python:
  import time
  print('sleep {} seconds.'.format(_seq))
  time.sleep(_seq)

"""
Explanation: Now, let us continue, copy/paste or select the next step, run sospaste and ....  a big block of errors!
End of explanation
"""


!cat .sos/default_1_0.py

"""
Explanation: Here 10 is printed so rep is valid. This means the rep we set last time is available and correct, then what might be the problem? Let us see what is saved in .sos/default_1.py using a bit magic of ipython
End of explanation
"""


%%sos
[1]
print(rep)
seq = range(rep)
input:  for_each='seq'

python:
  import time
  print('sleep ${_seq} seconds.')
  time.sleep(${_seq})

"""
Explanation: The error seems to be obvious, we need to use ${_rep} for the value to be passed through string interpolation. Let us make some changes to the script and run
End of explanation
"""


