title: RDD creation and variable binding
body: <p>I have a very simple code:</p>
  
  <pre class="lang-py prettyprint-override"><code>def fun(x, n):
      return (x, n)
  
  rdds = []
  for i in range(2):
      rdd = sc.parallelize(range(5*i, 5*(i+1)))
      rdd = rdd.map(lambda x: fun(x, i))
      rdds.append(rdd)
  
  a = sc.union(rdds)
  print a.collect()
  </code></pre>
  
  <p>I had expected the output to be the following:</p>
  
  <pre><code>[(0, 0), (1, 0), (2, 0), (3, 0), (4, 0), (5, 1), (6, 1), (7, 1), (8, 1), (9, 1)]
  </code></pre>
  
  <p>However, the output is the following:</p>
  
  <pre><code>[(0, 1), (1, 1), (2, 1), (3, 1), (4, 1), (5, 1), (6, 1), (7, 1), (8, 1), (9, 1)]
  </code></pre>
  
  <p>This is bewildering, to say the least.</p>
  
  <p>It seems, due to lazy evaluation of RDDs, the value of <code>i</code> that is being used to create RDDs is the one it bears when <code>collect()</code> is called, which is 1 (from the last run of the <code>for</code> loop).</p>
  
  <p>Now, both elements of the tuple are derived from <code>i</code>.</p>
  
  <p>But it seems, for the first element of the tuple, <code>i</code> bears values 0 and 1  while for the second element of the tuple <code>i</code> bears the value 2.</p>
  
  <p>Can somebody please explain what's happening?</p>
  
  <p>Thanks.</p>