Software development as a discipline is experiencing considerable growth these days. This expansion is evident not only in the development process itself, numerous conferences and the number of employed software developers (estimated at 20 million or more), but also in the amount of companies that rely heavily on IT infrastructure and services. However, the more specific needs a company has, the more specialised software it requires. In other words, more work has to be put into the development of such a product. Simply put, there are fewer ready, ‘plug and play’ tools available that a company could simply purchase and implement. In this case the only option is to finance custom software. Eventually, the company becomes an owner of an IT product that addresses its unique needs. The software development market may be growing, but it is also constantly changing. The company also has to evolve and so does its software. Introducing changes to a live software product involves a certain risk which can not be ignored because breaking any functionality would entail financial losses. After all, the company relies on its software. As a result, a lot of effort must be put into quality assurance and testing software to avoid problems. Of course this requires additional expenses to either hire more testers or outsource more work. To make things worse, the complexity of software projects grows exponentially over time if they are actively developed, which makes maintenance and testing more expensive. Does it always have to be like this? The short answer is: no. This is where test automation plays its role.
What is automated testing?
According to the simplest definition, automated testing amounts to autonomously checking whether software works as expected. These expectations are driven by actual business requirements and as a general rule cover all critical paths. It means that we interact with the application, checking functionalities that either directly earn money or whose failure means losses, financial or otherwise.
Preparing a test suite usually involves writing extra code that serves one purpose - making sure that the system under test behaves as desired. Basically it means that dealing with automated testing requires a certain amount of programming skills. On the other hand, one may encounter tools for test automation that have a nice graphical interface and are user friendly. There are several such products on the market, for example Selenium IDE, iMacros or TestComplete, just to name a few. However, many of them are limited to testing simple web pages. Additionally, adopting any tool takes additional effort to be put in integrating a one-size-fits-all product to specific needs.
Acceptance and end to end tests vs. unit and integration tests
The aforementioned definition may suggest that the only kind of automated testing is carried out on the layer of an UI. For a web application it would effectively mean a web browser. But that is just one type of a test. Depending on the nomenclature, criteria and additional conditions, testing through UI may be called acceptance or end to end testing. A simple summary of these:
- acceptance tests – determine whether the requirements of a specification are fulfilled
- end-to-end tests – test the whole application through all its layers, starting from the UI up to underlying external services
Conducting such tests is the exclusive responsibility of testers and quality assurance engineers. They leverage their programming skills to automate their work, but in contrast to software developers their work requires considerably less knowledge and experience to be effective.
Instead, the most important thing is to choose the right tools and testing strategy. These may vary depending on the project and stakeholders requirements. For example, the stakeholders may like to have detailed reports of testing scenarios, or maybe even participate in the process of creating test suites. In the latter case, behaviour driven design might be the answer.
There are more types of automated tests:
- unit tests – focuses on testing the smallest possible unit of code, such as a single class or function
- integration tests – checks whether a few components work well together,
In contrast to acceptance and end to end testing, unit and integration tests are usually conducted and maintained by software developers (especially unit tests), because they require detailed and extensive knowledge about the particular piece software that can’t always be directly interpreted in terms of business requirements.
This is just the tip of the iceberg. There are many more types of tests that can be automated, but this article focuses on the first one - acceptance / end to end tests, since they bring the greatest value - if done right.
The advantages of automated testing
Without further ado, let's get right to it—here are the advantages of automated testing.
1. Scaling better than manual testing
As I’ve mentioned before, the complexity of software projects grows over time. This means that testing the whole application would require more and more testers. It’s much cheaper to automate as many paths as possible instead of employing new people for doing repetitive work. In the case of insufficient speed of automated test suites it’s still easier and more affordable to configure tests to run in parallel or add another machine than to hire more quality assurance engineers.
2. Reliability
It was already said that conducting the same testing scenarios over and over again is a laborious and repetitive task. People are terrible at this kind of work - with time they get used to certain things and fall into routine. They may just have a bad day or be tired. It is hard to resist the temptation of omitting a few seemingly irrelevant checks, especially if the next day will look just the same.
Human nature becomes an obstacle here, so to minimize risks more people are involved in the process - eventually at least one person will find a bug. Computers, on the other hand, excel in doing the same thing around the clock. They will not omit a single test case unless explicitly instructed to do so. Any deviation from expected test results will be reported automatically. Finally, every run of automated tests will do exactly the same things.
3. Motivation boost
Freeing people from repetitive work enables them to spend more time on creative activities. This is not only beneficial to the employee, but also to employer. Happier and motivated staff is more willing to learn and stay in a company that has such empowering atmosphere.
4. Easy reporting
Automated tests make reporting easy and accessible, because usually in the end we get an aggregated report of the tests conducted in a single run. Using this information one is able to either monitor the stability of a product in real time or create more traditional, ‘offline’ reports containing information about a given period of time.
5. Shortened feedback loop
If a developer accidentally breaks some less known and used functionality, how long does it take to detect such a situation? Provided we have automated tests in place, it may be a matter of minutes. Without them, it might take days, even weeks. Early bug detection is one thing - it protects the project. On the other hand, a short feedback loop empowers developers and encourages them to take big steps, so eventually they become more productive. They are less worried and spend little time on defensive programming which amounts to avoiding breaking anything at all costs. Software developers gain confidence and a motivation boost.
6. Opening doors to higher team velocity
Automated tests form a foundation for modern software development approaches such as Continuous Integration, Continuous Delivery and Continuous Deployment. Their goal is to provide lower risk of release, shortening the time of getting the software on market and improving quality. Discussing them in detail is beyond scope of this article, but it is enough to say that automated testing opens another door to faster software development.
When is test automation especially beneficial?
Test automation has the greatest return on investment in software projects that already have been around for a while, are running in production and earning revenue.
They are equally adequate for a project that has not reached release stage yet, but already has at least established functionalities. What may be not obvious, test automation should be added in the earliest possible project stage with at least one basic test scenario.
The reason is simple - it’s much cheaper in the beginning. A protective coat of automated tests grows with the software product itself. This directly impacts the product, which is designed to be testable from the start. This is manifested in many ways, for example the presence of repeatable procedures of setting up new instances of an application on a staging server, the developers’ machines or the testers’ computers.
Under what circumstances you may not benefit from automated testing?
There are two cases when automating tests is not worth the effort:
- Small projects with a short lifespan that do not change at all after release beside tiny adjustments may not benefit from automated testing because manual testing may simply be done faster than writing tests. Also there is no profit from periodic checks because project isn’t being developed anymore.
- The second category is very specific. It contains projects whose business value is not well known, so it has to be discovered. In other words, the basic functionality and even purpose of a project changes relatively often. There is no established set of functionalities which automated tests might be checking. It would require an enormous effort to keep up with changing requirements.
Conclusion
With all the pros of automated testing in mind, one may confidently say that they are worth the effort in most cases. A great deal of software projects may benefit right from the start from automation, if only stakeholders decide to allocate a certain amount of funds on hiring people designated to establishing, development and maintenance of test automation.
In the end, test automation is a sound investment into the future reliability of your software product. Find a way to implement it early, and it will save you significant stress and costs in the future.
Of course, if your goal is to create world-class software, automated tests are only part of the process. On the business side, you may also look into product validation with techniques such as impact mapping to ensure the profitability of your product. And if you need to keep tabs on the productivity of your developers, burndown charts may be a worthwhile addition to your toolset.
If you have more testing-related questions after reading this article, make sure to use our shiny comments section and ask away. We check our comments regularly and we’ll be sure to get back to you with an answer.
And if you know anyone who might want to know more about automated testing, why not share the article using the neat buttons below?
