The Importance of Testing
6th January 2014
GCD Technologies is a software company that creates it’s own software as well as writing software for clients. It doesn’t matter who we write software for, it always gets tested. Unfortunately the level of testing can vary from project to project and correlates closely to budget. We perform most of the testing ourselves, but we also encourage our customers to become involved as much as possible.
In an ideal world all software would be tested to the same comprehensive level, but in the real world limited budget and timescales can impact on the amount of testing that can be performed. The aim of this post is to give an insight into the level of testing that goes into our work, and an understanding of how important it is to factor it into your budget and timescales.
Types of Testing
Testing software takes a number of forms and takes place at many points throughout the project life cycle.
Unit testing is performed at an extremely low level at the time the software is being written. As a result it is rarely observed by clients and so doesn’t get appropriate recognition of it’s importance. Writing tests for the small chunks of code (the units in question) allows us to quickly detect when a subsequent change is made that has negative repercussions. This is an invaluable early warning system throughout the initial development of an application, but is also a benefit when it comes round to working on newer versions. A good suite of unit tests means that we can get stuck into your version 2.0 without breaking the features already released in version 1.0.
We carry out functional testing from an early point in our projects. As soon as it is ready to be used in a meaningful manner we start putting it through it’s paces. This is actually carried out in a number of ways:
- Before committing code to our source code repository, we send it to our fellow developers for review – the code is examined, built and tested. Features that work are committed, features that don’t work, or even just don’t work quite right, are pushed back to the original developer for re-work.
- Once a feature or function has passed the approval of a fellow developer, it gets picked up by someone a bit more critical. Developers can be exceptionally critical (especially of important real-world stuff like continuity errors in movies) but have a soft spot for each other and a tendency to let some stuff through. Letting someone else perform functional testing means that there’s more chance of picking up an issue that makes it through a developer blind spot.
- When we’re internally happy with how features are progressing we’ll release the application in a controlled manner to our customers so that they can try the software out. This is a great way to ensure that their vision matches our implementation.
User interface testing
User interface testing can take place at a number of points throughout the project. Our designers use tools that allow them to quickly mock up proposed interfaces and allow the users and clients to try them out. Early feedback from users can help shape the product and allows us to get a feel for common usage patterns.
When the interfaces are more mature we perform additional testing to make sure that the product is stable in use, and that the developers have implemented the vision of the designers correctly. This could be as simple as catching graphical glitches, or the implementation of animations that users will find jarring or confusing.
System testing allows us to start connecting the discrete components of the project together and start testing them in an integrated manner. Many of the projects we work on are client-server systems where the client and server components are developed in parallel by different developers. End-to-end system testing allows us to check that the components interact with each other according to the specification that they were developed to.
System testing is also an opportunity to test integration with APIs, or with third party software components. During initial development it can be limiting to rely on external interfaces or libraries. These may not even be ready for consumption or they may be unavailable for periods of time. Developing against substitute (or mock) APIs or components allows us to develop at our own pace, and can provide a static target to work with.
Towards the end of the development cycle we will start to test the system under load to see how it will perform in the real world. This is an essential testing phase – load (or performance) testing can identify bottlenecks that may not be uncovered in simpler functional testing. Identifying these before putting a product into consumption can prevent costly downtime and analysis while your customers are asking why the product is unable to cope with demand.
This is the bit that makes us nervous. It’s the software development equivalent of waiting to see what the Man From Del Monte thinks of your months of hard work. Acceptance testing is your opportunity as a customer to be satisfied that the product you are about to sign-off on is the product you asked for in the first place.
It’s worth bearing in mind that acceptance testing is not an opportunity to introduce wholesale changes to your website or application. The acceptance tests should be based on the original specification and requirements. If, during the process of acceptance testing, you find something you want to change, then there is still time to put in a change request.
User Testing & A/B testing
There is no better test of a system than watching it being used by real users. This can be achieved in a number of ways including using online services such as usertesting.com which allows us to capture screen and audio recordings of real users sitting in the comfort of their own home while performing a set of key activities on your website or app. This can often give eye-opening insight to how different parts of a system work (or don’t work) in the wild. This type of user testing can be carried out both during the development process and after your system goes live.
A/B testing is a great way to test key parts your system, typically done after a system goes live, A/B testing is a way of splitting you users into two random groups (Group A & Group B). Group A will alway see the original version of one part of your site (e.g. the sign up page) at the same time Group B will see a newly designed version that we believe may work better. By running the A/B test for a set period and measuring how many signups you receive from each variation, you can measure the effectiveness of a proposed change. Companies like Google, Facebook and Twitter are constantly running A/B (or even more complex multivariate) testing in order to constantly improve their solutions.
Why Bother Testing?
Not all of our customers see the benefits of testing, and some feel that the costs in terms of time and budget do not justify such luxuries. Properly tested software costs less – the debate about this ended a long time ago. Studies have shown that the further along the development process that bugs are discovered, the more they cost to rectify. It can take just minutes for a developer to fix a breaking unit test, but it can cost days of lost revenue if a production system goes down. Understanding the importance of allocating time and budget to adequately test your system can mean a smoother launch, reduced maintenance, and easier development of features in the future. All things that cost less in the long run.