title: PySpark: modify column values when another column value satisfies a condition
body: <p>I have a PySpark Dataframe with two columns:</p>
  <pre><code>+---+----+
  | Id|Rank|
  +---+----+
  |  a|   5|
  |  b|   7|
  |  c|   8|
  |  d|   1|
  +---+----+
  </code></pre>
  <p>For each row, I'm looking to replace <code>Id</code> column with &quot;other&quot; if <code>Rank</code> column is larger than 5.</p>
  <p>If I use pseudocode to explain:</p>
  <pre><code>For row in df:
    if row.Rank &gt; 5:
       then replace(row.Id, &quot;other&quot;)
  </code></pre>
  <p>The result should look like this:</p>
  <pre><code>+-----+----+
  |   Id|Rank|
  +-----+----+
  |    a|   5|
  |other|   7|
  |other|   8|
  |    d|   1|
  +-----+----+
  </code></pre>
  <p>Any clue how to achieve this? Thanks!!!</p>
  <hr />
  <p>To create this Dataframe:</p>
  <pre><code>df = spark.createDataFrame([('a', 5), ('b', 7), ('c', 8), ('d', 1)], ['Id', 'Rank'])
  </code></pre>