title: Create new column in Pyspark DataFrame by taking ratio of existing columns
body: <p>I have two columns in a PySpark DataFrame and I want to take ratio of these two columns after filling null values (not inplace). Currently my DataFrame looks like as follows:</p>
  
  <pre><code>+----+----+---+----+----+----+----+
  |Acct| M1D|M1C| M2D| M2C| M3D| M3C|
  +----+----+---+----+----+----+----+
  |   B|  10|200|null|null|  20|null|
  |   C|1000|100|  10|null|null|null|
  |   A| 100|200| 200| 200| 300|  10|
  +----+----+---+----+----+----+----+
  </code></pre>
  
  <p>My desired output would look as follows:</p>
  
  <pre><code>+------+------+-----+------+------+------+------+-------+
  | Acct |  M1D | M1C |  M2D |  M2C |  M3D |  M3C | Ratio |
  +------+------+-----+------+------+------+------+-------+
  |    B |   10 | 200 | null | null | 20   | null |     0 |
  |    C | 1000 | 100 | 10   | null | null | null |    10 |
  |    A |  100 | 200 | 200  | 200  | 300  | 10   |    20 |
  +------+------+-----+------+------+------+------+-------+
  </code></pre>
  
  <p>I want to take ratio of <code>M2D</code> with <code>M3C</code> to create new column <code>Ratio</code>. Before taking ratio I want to fill <code>M2D</code> with <code>0</code> and <code>M3C</code> with <code>1</code> which would be performed on the fly in order to avoid nulls and to avoid replacing values inplace.</p>
  
  <p>I tried doing this by using following code.</p>
  
  <pre><code>df = df.withColumn('Ratio', col('M2D').fillna(0, subset=['M2D']) / col('M3C').fillna(1, subset=['M3C']))
  </code></pre>
  
  <p>The above code gave me the following error</p>
  
  <pre><code>TypeError: 'Column' object is not callable
  </code></pre>
  
  <p>As stated by above error in order to avoid TypeError I tried the following line of code. Instead of column now I am using DataFrame.</p>
  
  <pre><code>df = df.withColumn('Ratio', df.select('M2D').fillna(0, subset=['M2D']) / df.select('M3C').fillna(1, subset=['M3C']))
  </code></pre>
  
  <p>The above code led to following error.</p>
  
  <pre><code>TypeError: unsupported operand type(s) for /: 'DataFrame' and 'DataFrame'
  </code></pre>
  
  <p>How can I achieve my desired output?</p>