Python Mock Process for Unit Testing
Question: **Background:**  
I am currently writing a process monitoring tool (Windows and Linux) in Python
and implementing unit test coverage. The process monitor hooks into the
Windows API function [EnumProcesses](http://msdn.microsoft.com/en-
us/library/windows/desktop/ms682629%28v=vs.85%29.aspx) on Windows and monitors
the /proc directory on Linux to find current processes. The process names and
process IDs are then written to a log which is accessible to the unit tests.

**Question:**  
When I unit test the monitoring behavior I need a process to start and
terminate. I would love if there would be a (cross-platform?) way to start and
terminate a fake system process that I could uniquely name (and track its
creation in a unit test).

**Initial ideas:**  

  * I could use [subprocess.Popen()](http://docs.python.org/2/library/subprocess.html#popen-objects) to open any system process but this runs into some issues. The unit tests could falsely pass if the process I'm using to test is run by the system as well. Also, the unit tests are run from the command line and any Linux process I can think of suspends the terminal (nano, etc.).
  * I could start a process and track it by its process ID but I'm not exactly sure how to do this without suspending the terminal.

These are just thoughts and observations from initial testing and I would love
it if someone could prove me wrong on either of these points.

I am using Python 2.6.6.

**Edit:**  
Get all Linux process IDs:

    
    
    try:
        processDirectories = os.listdir(self.PROCESS_DIRECTORY)
    except IOError:
        return []
    return [pid for pid in processDirectories if pid.isdigit()]
    

Get all Windows process IDs:

    
    
    import ctypes, ctypes.wintypes
    
    Psapi = ctypes.WinDLL('Psapi.dll')
    EnumProcesses = self.Psapi.EnumProcesses
    EnumProcesses.restype = ctypes.wintypes.BOOL
    
    count = 50
    while True:
        # Build arguments to EnumProcesses
        processIds = (ctypes.wintypes.DWORD*count)()
        size = ctypes.sizeof(processIds)
        bytes_returned = ctypes.wintypes.DWORD()
        # Call enum processes to find all processes
        if self.EnumProcesses(ctypes.byref(processIds), size, ctypes.byref(bytes_returned)):
            if bytes_returned.value &lt size:
                return processIds
           else:
                # We weren't able to get all the processes so double our size and try again
                count *= 2
        else:
            print "EnumProcesses failed"
            sys.exit()
    

[Windows code is from here](http://stackoverflow.com/questions/6980246/how-
can-i-find-a-process-by-name-and-kill-using-ctypes)


Answer: **edit: this answer is getting long :), but some of my original answer still
applies, so I leave it in :)**

Your code is not so different from my original answer. Some of my ideas still
apply.

When you are writing Unit Test, you want to only test _your_ logic. When you
use code that interacts with the operating system, you usually want to mock
that part out. The reason being that you don't have much control over the
output of those libraries, as you found out. So it's easier to mock those
calls.

In this case, there are two libraries that are interacting with the sytem:
`os.listdir` and `EnumProcesses`. Since you didn't write them, we can easily
fake them to return what we need. Which in this case is a list.

But wait, in your comment you mentioned:

> "The issue I'm having with it however is that it really doesn't test that my
> code is seeing new processes on the system but rather that the code is
> correctly monitoring new items in a list."

The thing is, we don't _need_ to test the code that actually **monitors the
processes on the system** , because it's a third party code. What we _need_ to
test is that your **code logic handles the returned processes**. Because
that's the code you wrote. The reason why we are testing over a list, is
because that's what your logic is doing. `os.listir` and `EniumProcesses`
return a list of pids (numeric strings and integers, respectively) and your
code acts on that list.

I'm assuming your code is inside a Class (you are using `self` in your code).
I'm also assuming that they are isolated inside their own methods (you are
using `return`). So this will be sort of what I suggested originally, except
with actual code :) Idk if they are in the same class or different classes,
but it doesn't really matter.

## Linux method

Now, testing your Linux process function is not that difficult. You can patch
`os.listdir` to return a list of pids.

    
    
    def getLinuxProcess(self):
        try:
            processDirectories = os.listdir(self.PROCESS_DIRECTORY)
        except IOError:
            return []
        return [pid for pid in processDirectories if pid.isdigit()]
    

Now for the test.

    
    
    import unittest
    from fudge import patched_context
    import os
    import LinuxProcessClass # class that contains getLinuxProcess method
    
    def test_LinuxProcess(self):
        """Test the logic of our getLinuxProcess.
    
           We patch os.listdir and return our own list, because os.listdir
           returns a list. We do this so that we can control the output 
           (we test *our* logic, not a built-in library's functionality).
        """
    
        # Test we can parse our pdis
        fakeProcessIds = ['1', '2', '3']
        with patched_context(os, 'listdir', lamba x: fakeProcessIds):
            myClass = LinuxProcessClass()
            ....
            result = myClass.getLinuxProcess()
    
            expected = [1, 2, 3]
            self.assertEqual(result, expected)
    
        # Test we can handle IOERROR
        with patched_context(os, 'listdir', lamba x: raise IOError):
            myClass = LinuxProcessClass()
            ....
            result = myClass.getLinuxProcess()
    
            expected = []
            self.assertEqual(result, expected)
    
        # Test we only get pids
        fakeProcessIds = ['1', '2', '3', 'do', 'not', 'parse']
        .....
    

## Windows method

Testing your Window's method is a little trickier. What I would do is the
following:

    
    
    def prepareWindowsObjects(self):
        """Create and set up objects needed to get the windows process"
        ...
        Psapi = ctypes.WinDLL('Psapi.dll')
        EnumProcesses = self.Psapi.EnumProcesses
        EnumProcesses.restype = ctypes.wintypes.BOOL
    
        self.EnumProcessses = EnumProcess
        ...
    
    def getWindowsProcess(self):
    
        count = 50
        while True:
           .... # Build arguments to EnumProcesses and call enun process
           if self.EnumProcesses(ctypes.byref(processIds),...
           ..
           else:
               return []
    

I separated the code into two methods to make it easier to read (I believe you
are already doing this). Here is the tricky part, `EnumProcesses` is using
pointers and they are not easy to play with. Another thing is, that I don't
know how to work with pointers in Python, so I couldn't tell you of an easy
way to mock that out =P

What I _can_ tell you is to simply not test it. Your logic there is very
minimal. Besides increasing the size of `count`, everything else in that
function is creating the space `EnumProcesses` pointers will use. Maybe you
can add a limit to the count size but other than that, this method is short
and sweet. It returns the windows processes and nothing more. Just what I was
asking for in my original comment :)

So leave that method alone. Don't test it. Make sure though, that anything
that uses `getWindowsProcess` and `getLinuxProcess` get's mocked out as per my
original suggestion.

Hopefully this makes more sense :) If it doesn't let me know and maybe we can
have a chat session or do a video call or something.

**original answer**

I'm not exactly sure how to do what you are asking, but whenever I need to
test code that depends on some outside force (external libraries, popen or in
this case processes) I mock out those parts.

Now, I don't know how your code is structured, but maybe you can do something
like this:

    
    
    def getWindowsProcesses(self, ...):
       '''Call Windows API function EnumProcesses and
          return the list of processes
       '''
       # ... call EnumProcesses ...
       return listOfProcesses
    
    def getLinuxProcesses(self, ...):
       '''Look in /proc dir and return list of processes'''
       # ... look in /proc ...
       return listOfProcessses
    

These two methods **only do one thing** , get the list of processes. For
Windows, it might just be a call to that API and for Linux just reading the
/proc dir. That's all, nothing more. The logic for handling the processes will
go somewhere else. This makes these methods extremely easy to mock out since
their implementations are just API calls that return a list.

Your code can then easy call them:

    
    
    def getProcesses(...):
       '''Get the processes running.'''
       isLinux = # ... logic for determining OS ...
       if isLinux:
          processes = getLinuxProcesses(...)
       else:
          processes = getWindowsProcesses(...)
       # ... do something with processes, write to log file, etc ...
    

In your test, you can then use a mocking library such as
[Fudge](http://farmdev.com/projects/fudge/api/fudge.html). You mock out these
two methods to return what you **expect** them to return.

This way you'll be testing _your_ logic since you can control what the result
will be.

    
    
    from fudge import patched_context
    ...
    
    def test_getProcesses(self, ...):
    
         monitor = MonitorTool(..)
    
         # Patch the method that gets the processes. Whenever it gets called, return
         # our predetermined list.
         originalProcesses = [....pids...]
         with patched_context(monitor, "getLinuxProcesses", lamba x: originalProcesses):
             monitor.getProcesses()
             # ... assert logic is right ...
    
    
         # Let's "add" some new processes and test that our logic realizes new 
         # processes were added.
         newProcesses = [...]
         updatedProcesses = originalProcessses + (newProcesses) 
         with patched_context(monitor, "getLinuxProcesses", lamba x: updatedProcesses):
             monitor.getProcesses()
             # ... assert logic caught new processes ...
    
    
         # Let's "kill" our new processes and test that our logic can handle it
         with patched_context(monitor, "getLinuxProcesses", lamba x: originalProcesses):
             monitor.getProcesses()
             # ... assert logic caught processes were 'killed' ...
    

Keep in mind that if you test your code this way, you won't get 100% code
coverage (since your mocked methods won't be run), but this is fine. You're
testing your code and not third party's, which is what matters.

Hopefully this might be able to help you. I know it doesn't answer your
question, but maybe you can use this to figure out the best way to test your
code.

