There is a saying that I always come back to when thinking about testing, “Don’t let perfect be the enemy of good.”
When we test software, it’s not our job to find bugs. It’s not our job to humiliate developers who missed something you think is a simple defect (it rarely is). Nor is it to fight with project managers about how much time QA has (it’s rarely enough). It is, however, to ensure the system under test is of a high enough quality by assessing critical components and flows through the application and designing test cases to exercise such areas.
The members of the QA team are not the only ones who have this responsibility, but we focus on that more than any other group. We want to make sure that whatever gets out to production can be counted on to show the end users that they made the right decision to use our product.
And that can cause problems.
By trying our best to ensure the quality of the product, we can sometimes fall into the trap of over-testing, leading us to focus on bug finding instead of quality. This can lead to testing improbable user journeys when our focus should be on proving that the system meets the needs of the users who are going to interact with it. It’s not to say that those sorts of tests have no place in a test run, but they should compliment everyday use cases, not come at the expense of them.
To put it another way, if we had a simple form with four drop-downs on it each with ten possible selections and we wanted to test every possible combination of choices on that form, testing without any breaks for 7.5 hours a day, five days a week would still take over a month to complete. No one has time for that, nor would it be a good use of your day even if you did.
Exhaustive testing is impossible. Bugs exist in the wild and always will. It’s not our responsibility to find all of them. It’s our responsibility to help release high-quality software to the people who want to use it.
So when thinking about how to test something, don’t let perfect be the enemy of good.
