Can QA testers carry out unit tests

Software Maintainability - Part 2: Tests

Easily maintainable software minimizes the effort required for troubleshooting, adaptive maintenance and subsequent change requests. But what distinguishes good maintainability and how can we achieve it? Now that we have dealt with the documentation in the first part of the “Maintainability of Software” series, we want to go into more detail about the tests in Part 2. To this end, we deal with the different types of tests, with automation and test environments, and make clear the need for a high level of test coverage.

introduction

The maintenance team will have the task of reproducing misconduct reported by the customer as quickly as possible in an environment similar to the production system (integration or test stage). Based on the following error analysis, the source code will then have to be corrected. The source code will also be adapted for technical changes (change requests) that have to be implemented in the maintenance and operation phase. But code adjustments can have side effects, especially if they are done in a central location.

In most cases, the maintenance team consists of fewer people than the team responsible for the original development. To the same extent, the time available and the budget are often significantly smaller. Nonetheless, the customer is rightly expected to understand the complex technicalities and technology of the software from the first troubleshooting step and that further code adjustments are made with the same high quality as in the previous new development.

In order to give the maintenance team security that the changes made will not have any undesirable side effects on other use cases, the highest possible test coverage through the various automated test methods is essential.

If the test coverage of technical use cases were low or, in the worst case, nonexistent, the maintenance team would have to know all the technical aspects when correcting an error. In addition, after a code change, a complete manual system test would have to be repeated to the same extent as was carried out for acceptance. This is the only way to ensure that the change made is correct, comprehensive and free of side effects.

Comprehensive unit tests, automated integration tests and regular load and performance tests are very valuable for the maintenance team in maintaining quality. In order for the maintenance team to be able to use these tests as quickly as possible, they must be automated in a test environment and the content and execution must be well documented.

Unit tests

As already described, it is essential that unit tests (module or component tests) are available at all. This requirement applies not only to backend code (e.g. Java) but also to the proportion of frontend code (e.g. Javascript, Angular) that has grown again in the web environment in recent years.

The source code of the unit tests should be of the same quality as the application code itself in terms of readability and robustness. The unit test should be written in such a way that the technical application implemented in the component is not only checked technically, but also the desired behavior of the component can be read from the source code.

As part of continuous integration, all existing unit tests must be automated in such a way that they deliver a result as quickly as possible after every change (every push). This means that a tool for code analysis (e.g. SonarQube) is provided and configured. This can collect the metrics for test coverage immediately after a push and check them against a quality target (quality gate) defined for the project. If the check fails, the development team can be informed automatically and promptly by email.

But how high should the test coverage through unit tests be? This question is much debated and this article is not intended to provide a conclusive answer for every project. However, experience from projects that have already been successfully taken over into maintenance shows that with good planning, a high priority for the tests and appropriate configuration of the code analysis tool (e.g. ignoring automatically generated code), a high test coverage beyond 80% with reasonable effort can be achieved.

It is important that the time and budget are allocated for setting up, managing, configuring and regularly readjusting the automated code analysis. The work packages required for this must already be planned in the preparation of the offer and calculation for the new development project.

Right at the beginning of the project, it must be clarified which tool and which metrics the code analysis for the project should be based on. If such a tool is already provided centrally in your own company, it should also be used together with a standard set of metrics (e.g. condition coverage), as this saves effort and budget in the project and promotes a largely uniform quality of the tests in the company. Any deviation from the standard tool and set must be clearly documented in the project documentation.

The following graphic shows an example of the code coverage (coverage) of a unit test determined with the EclEmma tool in the Eclipse development environment:

With all the activities described here, however, one should never forget what actually characterizes a good unit test. To say that the test is green and has a code coverage of over 80% meets the requirement at first glance, but would by far not be sufficient. A good unit test checks the technical requirements in all aspects, including borderline cases, and is easy to read and understand for third parties. The answer to the following question often helps to evaluate the quality of a unit test:

Does the test fail as soon as the application code to be tested is changed in such a way that the technical requirement is no longer correctly implemented afterwards?

Integration and interface tests

While unit tests ensure the correct functionality of the smallest units, namely individual, self-contained components of the software, integration tests check the functionality when several such components work together. In addition, interface tests check the correctness of the data exchanged between the components. From the maintenance point of view, the requirement applies here that corresponding tests are already implemented with a high level of test coverage during the new development of the software and, if possible, are carried out automatically and regularly.

Test setup

For an integration test, some individual components that have already been tested by unit tests are combined to form a new component and the interaction of the individual components is tested. The resulting new “larger” component can then in turn be tested in combination with other components. As a result, the entire system is broken down into components. In addition, the data that is exchanged between the components via the interfaces must then be checked using suitable interface tests.

For example, a simple web application consists of a front-end and a back-end component. Their professional interaction is to be tested through integration tests (e.g. with Selenium). The front-end and back-end consist of different individual components, the functionality of which has already been ensured by appropriate unit tests. There is an interface between the front end and the back end, for which a corresponding interface test (e.g. with SoapUI or Postman) must be implemented.

Test coverage

Now to the question of how high the test coverage must be through technical integration tests. At least all core functionalities should be covered. However, since it always makes sense to consider costs and benefits for the rest, the list of all use cases can be divided into 3 categories by weighting:

Category A includes all business-critical core processes of the application. They are in permanent use and must not fail. At least 80% should be covered with one test for the positive case and two tests for the negative case.

Category B. includes all standard processes of the application. They are used regularly and cause severe degradation if they fail. At least 70% should be covered with a test for positive cases and a test for negative cases.

Category C includes all other application processes. They are rarely used and / or have little impact if they fail. At least 50% should be covered with a test for the positive case.

Mocks

For an integration test, it is often necessary to simulate interfaces to components that are not directly tested in this test but are required. The same applies to interfaces to peripheral systems, which sometimes cannot be made available in a test environment. Mock services (e.g. with SoapUI) can be implemented for this purpose. Their configuration, functionality and professional behavior must be documented in such a way that the maintenance team can understand them, set them up and, if necessary, adapt them to changed requirements.

Test data

For integration and interface tests, test data are created during the redevelopment of the software. Sometimes real data is also provided anonymously as test data by the customer. These should be structured in such a way that they come as close as possible to the subject matter. It must be documented how the test data is structured and which steps are necessary in order to correctly import them into the database at the time of a test run. In the best case, this is done automatically as part of a continuous integration (e.g. in a database in the Docker container).

Load and performance tests

In order to ensure that newly developed software in productive operation meets the required, non-functional requirements, appropriate load and performance tests must be implemented. These tests are to be implemented in such a way that, on the one hand, they simulate the maximum expected user behavior (the expected load) and, on the other hand, can make a statement about the expected resource requirements and the necessary scaling.

To carry out the load or performance tests, a corresponding tool (e.g. Gatling, JMeter) can be configured in such a way that it automatically executes use cases with the desired frequency, parallelism and load changes (increase, decrease) on a test system specially provided for this purpose. The test system must be as similar as possible to the later productive system. This applies not only to the hardware used and the connection of peripheral systems, but also to the quantity and quality of the test data to be prepared on the test system. For this purpose, the numbers of entities, aggregates, sessions, users, etc. to be expected in productive operation should be used. In addition, it is essential to pay attention to the technical quality, coherence and consistency of the test data.

But which use cases should be selected for a load or performance test? First, at least the core application processes that are most frequently used must be examined in detail. Second, a test should be implemented that executes a mix of different use cases at the same time. And thirdly, those processes must be identified and checked that are highly likely to have a negative impact on performance. These include processes that work on a lot of data and / or implement a complex algorithm (e.g. long-running reports).

In the project, time and budget must be provided in advance in order to be able to plan the different tests and to define the key figures to be determined as well as the performance target (limits). Then, at the start of the project, the test system must be set up and configured, the continuous integration must be expanded and, of course, the tests must be implemented as early as possible.

The following graphic shows an example of the temporal progression of the memory requirement of a software under load:

In order to be able to recognize a change in performance at an early stage, the load and performance tests must be carried out at regular intervals during the development phase (e.g. once per sprint or once a month). The prerequisite for this, however, is that the functional tests (unit, integration and interface tests) have been carried out successfully in advance, i.e. that there are no errors in the functional requirements. After the test has been carried out, the key figures determined must be compiled and checked against the performance target. A test should always fail if one of the key figures exceeds or falls below the previously defined limits (the performance target).

Due to the regular activities related to load and performance tests, it makes sense to automate these as well and integrate them into a continuous integration pipeline. In the best case scenario, a pipeline build (job) fails because the performance target for the current version (sprint) has not been achieved.

In any case, it must be documented at a central point which load and performance tests are available, how these are to be carried out, which test data are to be used, which key figures are the basis and how the performance target is defined. In addition, the test results of the individual test runs must be recorded in a suitable place.

Some possible measuring points and values ​​are listed here as examples, which can be determined as key figures depending on the type of software and the type of non-functional requirements (e.g. using JavaMelody):

  • Response time (average, percentile, min, max)
  • Maximum achievable number of simultaneous users or use cases per user
  • RAM requirement (min, max, throughput, garbage collection)
  • CPU usage, number of parallel threads
  • Runtime of database queries (min, max, depending on the amount of data)

Test environments

While unit tests are carried out at the build time with the current code status in the context of the build environment, for the other test types (integration, interface, load and performance tests) the finished software is run on a specially designed configured environment rolled out and configured. For this purpose, several suitable test environments (test stages) have to be provided, which in the best case are configured in such a way that they come as close as possible to the later productive environment in every aspect. This applies to the following points, for example:

  • Resources (CPU, memory, hard disk)
  • Software (operating system, Java version, ...)
  • Runtime environment (application or web server)
  • Peripheral systems (interfaces, test systems of the customer)

The generation and configuration of the test stages should be automated as far as possible (e.g. by Docker, Ansible). For this purpose, the required test environment can be neatly set up and configured as part of the continuous integration through a step prior to the test execution. Then the following test run is not only based on the current version of the software, current test data and a possibly changed configuration of the environment (e.g. Tomcat version) are also used.

One of the test environments can be provided explicitly so that the QA team can carry out manual tests on an ongoing basis during development and the full system test at the latest when the software is accepted. In addition, this environment can regularly be used to present the current state of development (e.g. in the sprint review) and can also be used to reproduce errors that occur.

Conclusion

Automated tests of different types, granularity and coverage are intended to increase security and speed for code adjustments to be implemented after go-live. We have presented criteria that are to be observed as the basis for the later transfer of software to application management (maintenance). It is important that in order to fulfill these criteria activities are necessary that have to be planned in the first phases of a software development project!

outlook

In the third and last part of this series, we will finally deal with code quality as well as continuous integration and deployment.

Part 3 - Quality and Automation

Missed the first part?

Part 1 - Documentation