You are here: Home / IT elements / What’s the big deal about Testing? It’s what you do and how you do it.
Testing can often seem like the unglamorous Cinderella of the software development world, not as influential as the Customer-facing roles in project teams or as creative as design or coding (development). But Testing as a discipline is just as important, and in some cases, more important, to the overall project and positive outcomes than any other activity, bar none. Ok, without building or otherwise acquiring the product or service there is nothing to test, but if the thing that you have produced is not acceptable to the paying Customer, is not safe, usable (by ‘Users’) or fit-for-purpose, then the cost and impact of inadequate testing can be a lot worse than doing nothing at all.
In this shortish introduction to a big topic I will try to debunk some of the perceived problems with testing, explain some of the jargon, and hopefully encourage you to think a bit more about the pursuit of quality as the rationale for testing, rather than the negative connotations of finding fault. For clarification here, I am interested in testing designed and man-made artefacts (per the definition in the Glossary of IT Terms), so not, for example, testing a hypothesis, marketing (A/B type testing), gauging opinions, or taking measurements in the natural or human worlds.
‘Bug’ in the sense of any unexpected behaviour, software glitch or problem with interpretation of the requirements or design specifications. All of these things are important, but not the end in itself. It can be misleading to measure the value and effectiveness of testing as a simple headcount of bugs found and killed, i.e. problem resolved.
Non-functional testing (against non-functional requirements ‘NFRs’ if they exist) goes beyond ‘what’ the software does to ‘how’ it does it.
In both cases there may be positive and negative test scenarios. The latter may involve trying to ‘break’ the system…better that you find any problems before a clumsy User or some malicious outsider does!
To answer questions about how the system performs, ‘under stress’, for example, a peak number of Users or transactions, large file sizes, or environmental conditions that may be at the limits of normal operation.
A sub-set of the non-functional testing. Sometimes it is difficult to articulate and test how easy it is to use a product, a website or software system without letting real users ‘play with it’ possibly in a sand-box, either the finished product (alpha or beta test) or an early proof-of-concept or prototype.
Test how well the software or application copes with external threats, potential malice-ware and service attacks. Unfortunately this is an increasing issue for all organisations that need to protect themselves, their income, sensitive data, or reputation.
An early confidence booster; how does the whole system operate when everything is connected together, possibly involving components from different sources, and connecting software from different suppliers.
Each piece of code (unit test), sub-system, component and interface should already be proven in advance, so that there are none or fewer surprises at this stage!
Normally a formal and fairly late positive test that the whole system is acceptable to the Customer prior to ‘hand-over’. This may include some regression testing, to ensure that existing operational systems or components are not adversely affected.
Lastly, analogous to the discussion about gathering requirements, bugs tend not to be lying around waiting to be found. The testing process needs planning, preparation of testing tools and data (test scenarios/test cases).
There is a mantra in IT and technology projects, ‘test early, test often’. Even without a finished product to look at and kick the wheels, you can validate requirement models or design prototypes, and verify that each finished part of the whole works independently, especially if you are approaching the project in an ‘agile’ iterative and incremental fashion. It’s always cheaper and quicker to modify or rework the earlier the better.
Ideally testing, whether by dedicated testers or not, should be fully integrated within the development cycle and collaborative project team. However, in a tradition waterfall-type project this is not possible, so a hand-off of the software product, documentation and acceptance criteria is needed. The testers may be a separate off-shore team in a different continent, which can add to the challenges of testing efficiently and effectively.
The sobering thought is that for the majority of software systems and applications – so called ‘variable’ software – the testing can never be 100% complete. There is always the unexpected usage scenario, changes to the external environment, or the addition of unpredictable human users! So, assuming there is never enough time, the amount of testing becomes an exercise in prioritisation, budget constraints, and risk management.
As above, unless it is a trivial exercise, where failure has no financial or reputational cost, some testing is better that no testing. But, the $64,000 question is, how much testing is enough?
It eventually comes down to a juggling act between quality, cost and timescales – the so-called quality triangle – where acceptable quality could be very subjective and mean ‘good enough’ to meet the Customer’s needs and expectations.
Thank you for reading this post, and for your comments and feedback.
As the wife of a developer I am quite familiar with these terms and your explanations are great.
You really managed to translate “tech” into “normal human” language, Tony. Well done.
I’ll definitely let my husband read it too. He might be quite interested in reading your blog.