title: Questions about Python 'yield' keyword that I have not found answers elsewhere, and its specific use in a code I am working on
body: <p>I am facing a python script that was handed over to me that works. I understand the purpose of that script and its role in the big picture of how it interacts with other modules, as well as its internal architecture pretty well in most places. However, I have to do a major overhaul of that script, essentially to remove some old classes and add plenty of new subclasses so that it provides more functionality that we need. My question comes largely from what I have seen to be some unexplained discrepancy between some functions returning a list with that object in it vs <code>yield</code>ing that object back to itself.</p>
  
  <pre><code># These functions are methods that belong to a class. 
  # There is a top level script that instantiates that class and calls
  # these methods on that class, and depending on the `self.mode` variable located in the instance namespace, it invokes the different subsequent methods, 
  # which are either generateHybridSim() or generatePureHwSim()
  # It is worth pointing out here that HybridSimStep and ShangHWSimStep
  # are both classes themselves and that they will be instantiated later on as
  # I will describe after this chunk of code
  
  def generateSubTests(self) :
    if self.mode == TestStep.HybridSim :
      return self.generateHybridSim()
    elif self.mode == TestStep.PureHWSim or self.mode == TestStep.AlteraSyn \
       or self.mode == TestStep.AlteraNls or self.mode == TestStep.XilinxSyn  :
    return self.generatePureHWSim()
  
  return []
  
  def generateHybridSim(self) :
    return [ HybridSimStep(self) ]
  
  def generatePureHWSim(self) :
    yield ShangHWSimStep(self)
    num_iter = self.max_scheduling_iterations
    if num_iter &gt; 1 :
      for i in range(1) :
        sim_step = ShangHWSimStep(self)
        sim_step.option = self.option.copy()
        sim_step.hls_base_dir = os.path.join(sim_step.hls_base_dir, str(i))
        sim_step.rtl_output = os.path.join(sim_step.hls_base_dir, sim_step.test_name + ".sv")
        sim_step.option['max_scheduling_iterations'] = i + 1
        yield sim_step
  </code></pre>
  
  <p>Ultimately, regardless of whether the <code>generateHybridSim()</code> or <code>generatePureHwSim()</code> methods are invoked, they all get called in another module in the exact same way:</p>
  
  <pre><code># The 'job' that is in front of generateSubTests() is the instance's
  # variable name, and you can see that prepareTest() and runTest()
  # methods get called on the subtest iterable object, which so happens
  # to be class's instance.
  # So in short, these two methods are not defined within generateSubTests() method, but
  # rather under the classes that the generateHybridSim() and 
  # generatePureHWSim() methods had returned or yielded respectively.
  
  for subtest in job.generateSubTests() :
        subtest.prepareTest()
        subtest.runTest()
        time.sleep(1)
        next_active_jobs.append(subtest)
  </code></pre>
  
  <p>I'm really confused here now, and I don't know what's the significance of the use of <code>yield</code> here vs <code>return</code>, and I need to figure out why the previous programmer who wrote this script did that. This is because I'll be implementing new subclasses that must themselves contain their own <code>generateSubTests()</code> methods and must adhere to the same function call. The fact that he did <code>for subtest in job.generateSubTests</code> means that I am restricted to only returning a list with the class in it, or yielding the class itself, otherwise it wouldn't fit the python for loop iteration protocol. I have tried testing the code by modifying the <code>yield</code> statements in <code>generatePureHWSim()</code> to <code>return</code> ones like in <code>generateHybridSim()</code> and it seems to run fine, although I can't be sure if there's any subtle bugs that has introduced. However, I don't know if I'm missing out something here. Did the previous programmer wanted to facilitate concurrency <a href="http://www.dabeaz.com/coroutines/index.html" rel="nofollow noreferrer">http://www.dabeaz.com/coroutines/index.html</a> by turning the function into a generator using <code>yield</code>?</p>
  
  <p>He has since left our lab entirely and so I'm not able to consult him for his help. </p>
  
  <p>Also, I've read up on <code>yield</code> from various sources including the post:
  <a href="https://stackoverflow.com/questions/231767/what-does-the-yield-keyword-do-in-python">What does the &quot;yield&quot; keyword do in Python?</a>  ; although they have helped me understand what yield does, I still don't understand how using it here helps us in our context. In fact, I don't even understand why the previous programmer wanted to implement a loop with <code>for subtest in job.generateSubTests() :</code> and force the <code>generatePureHWSim()</code> and <code>generateHybridSim()</code>  methods to have to be generators themselves, so that we can have a loop just to call the other methods of <code>prepareTest()</code> and <code>runTest()</code> on the instance. Why couldn't he have just returned the class directly and called those methods??? </p>
  
  <p>This is really tripping me up. I would greatly greatly appreciate any help here!!! Thank you.</p>
  
  <p>PS: one more question - I noticed that in general, if you have a function that you defined as:</p>
  
  <pre><code>def a():
    return b
    print "Now we return c"
    return c
  </code></pre>
  
  <p>It seems like whenever the first statement within is executed, and <code>b</code> is returned, then the function completes execution and <code>c</code> is never returned because that statment that comes after <code>return b</code> will never be touched. Try adding the print statment, and you'll see that it will never be printed.</p>
  
  <p>However, when it comes to <code>yield</code>:</p>
  
  <pre><code>def x():
    yield y
    print "Now we yield z"
    yield z
  </code></pre>
  
  <p>I noticed that even after the first <code>yield y</code> statement has been executed, the subsequent <code>yield z</code> will get executed. Try adding the print statement, and you'll see that it gets printed out. This is something I observed as I was debugging the above code, and I don't understand this difference in behavior between <code>yield</code> and <code>return</code>. Can someone please enlighten me on it?</p>
  
  <p>Thank you.</p>