Does my test automation strategy sound ludicrous?
Question: I am developing an automation testing framework for testing a web service. The
web service is SOAP and implemented in Java (via Apache Axis2), however, our
tests are implemented in Python and uses the suds library to issue requests to
the server. The tests are high level tests that tests scenarios such as
backing up a user's data.

Now this web service is complicated in that certain methods require passing
lots of different types of objects and often require calling other methods to
acquire certain objects. For example, we have a call for backing up a user
whose pre-reqs looks like this :

  1. call method getUser() to retrieve object User
  2. call method getDataset() to retrieve object Dataset
  3. call method getXService() to retrieve object XService
  4. call method doBackup(User, Dataset, XService) to begin the backup 

This is just a simple example of one of our calls that have many other pre-
requisite calls before the primary call can be made. Since these scenarios
need to be executed often in the tests, I want to abstract the set of calls so
that to perform the above backup, I just need to call one method that just
calls the other methods for me.

My question is, should I do this in an object-oriented fashion and pretty much
create a 1-to-1 mapping of Python classes that map to the Java version of the
objects? So my testing framework would just include classes so I could do:

    
    
    # User, XService, and Dataset are classes that correspond with 
    # types implemented in the SOAP web service
    from lib import XService, Dataset 
    
    class User():
      def __init__(self, **kwargs):
        self.id = kwargs.get('id', None)
    
      def create(self):
        soap_client.call('createClient', self.id)
    
      def backup(self):
        dataset = Dataset.get(1234)
        service = XService.getInstance() 
        soap_client.call('doBackup', self, dataset, service)
    

So all I have to do is call backup() rather than issue 3 different methods
everytime.

The downside of this is that I have to create an object for every Type on the
server. Moreover, the objects on the testing side may get stale since they
don't sync with data on the server.

My other idea was not to go the Object Oriented route and just create a module
with just methods like backupUser() or restoreBackupToUser() and just feed
them actual data objects returned from the server. This approach would solve
the stale data issue but also would create massive modules that would be hard
to maintain.

Can anyone who has encountered this problem give some tips or critique on my
approaches. Perhaps I'm thinking too much and there is a better way to go
about testing the web service methods.


Answer: I had to create an automated test suite for testing a web service a while
back. I wrote the whole thing object orientated as there were a lot of methods
that each test would use in common. Also this saved a lot of time when it came
to negative testing etc. If you have the time writing you're test this way
will be earier to maintain and once you have wrote all the initial methods to
be called you'll save a lot of time as you're tests will just be several
function calls with very little to no logic. Here is an example of something
we done:

    
    
    import suds, unittest, pexpect, re, os,time, sys, random
    from random import randrange
    from setauth import Authentication
    
    ####################################################################
    #
    #                 Add/Get Profile Function Tests
    #
    ####################################################################
    
    class TestAddGetProfile(unittest.TestCase):
    
        def setUp(self):
            project = testvars[0]
            media_address = testvars[5]
            mgmt_address = testvars[4]
            self.profile = testvars[7]
    
            # connects to WSDL file and stores location in variable 'client'
            self.client = testvars[15]
            self.client.options.location = media_address
            self.mgmtclient = testvars[14]
            self.mgmtclient.options.location = mgmt_address
    
            self.getProfileToken = self.client.factory.create('ns4:ReferenceToken')
            self.getProfileToken.value = self.profile
    
            self.createProfileToken = self.client.factory.create('ns4:ReferenceToken')
            self.createProfileToken.value = self.profile
    
            self.createProfileName = self.client.factory.create('ns4:Name')
            self.createProfileName.value = self.profile
    
            self.deleteProfileToken = self.client.factory.create('ns4:ReferenceToken')
            self.deleteProfileToken.value = self.profile
    
            self.auth = Authentication()
            self.auth.setAuth('Administrator', self.client, self.mgmtclient)
            self.auth.makeUsers(self.mgmtclient)
    
    
        def sudsCreateProfile(self):
    
            create_result = self.client.service.CreateProfile(self.createProfileName, self.createProfileToken)
    
        def tearDown(self):
            self.auth.deleteAllUsers(self.mgmtclient)
            self.client = None
    
    class AdminCreateProfile(TestAddGetProfile):
    
        def runTest(self):
            self.auth.setAuth('Administrator', self.client, self.mgmtclient)
            try:
                self.sudsGetProfile()
                self.sudsDeleteProfile()
            except:
                pass
            try:
                self.sudsCreateProfile()
                self.assertTrue(True)
            except:
                self.assertFalse(True)
    

We have several other script's here for authentication and a test wrapper to
run all the tests. But you should get the general idea here.

Hope this help's you're decision.

