Regression Testing

Regression testing is one of the most frequently used objectives of testing. It focuses on the application retesting after changes are implemented. That changes can be local, - like new features or bug-fixes, or global, - like new business logic implementation.

Software must be checked for bugs every time the changes are applied, and regression testing is intended to make sure that the new version of the software is as stable as the previous one. The part of errors that occur after the code was changed is more likely to be from 50% to 80% of all errors that are discovered. Regression testing reduces two risks:

• a change that was intended to fix a bug failed;
• a change had a side effect, as a result a new bug was discovered or an old one appeared.

Types of Regression Tests

Regression testing combines different testing methods that have one common trait: tests are executed in regression manner. That is why the term “regression testing” may contextually mean:

• Verification test that consists from build verification test and bug verification test:
  - Build verification test(build acceptance test, quick check, smoke test) - is a small set of functionality tests that are intended to check general product features, this test is executed on each new build.
  - Bug verification test - re-executes the failed test cases to check that the bugs were fixed.
• Old fix regression test - validates the “verified” (fixed) bugs. This test is intended to cover the possibility that the new bug-fix affects the old ones or a developer mistakenly broke an old bug-fix.
• General functional regression testing - verifies that new functionality or code changes do not destabilize or regress the code. QA specialists should retest the project including the parts that worked in previous versions.
• Port or conversion testing - checks that the program works correctly after porting to another platform
• Configuration testing - validates that the program code runs correctly on a new version of the operation system or with a new device or in combination with a new application. Configuration testing and conversion testing are very much alike, the difference is that the program code is not changed.
• Localization testing - verifies that the software interface is readable and understandable for the people who speak other languages and/or have different set of cultural rules. Localization testing can involve the modified test cases, and the test cases that were written just for this purpose. New localization test cases may become regression test cases if it is required.

Regression Testing Strategy or Effective Regression Testing

Testing process must be completed in a very compressed time frame. There are some recommendations how to set the testing priorities and gain the most effective results:

1. Start regression testing from build verification test.
2. Verify the installation process if the application requires it.
3. Execute a short test of main and new functionality.
4. Run bug verification test to check that new bugs were fixed.
5. Run general function regression tests of the parts of the system that were changed. This testing is implementing over test cases that had status “passed” during previous test run.
6. Run old fix regression tests over the changed parts of the system to check the old bug-fixes.
7. Run the tests that were passed twice (they are marked as “closed”) if there are any changes in the code.

The last regression test of an application before its acceptance testing or release is called final regression test. Final regression test verifies the most complicated parts of the system. The recommended order of the final regression testing is:

1. Run the test cases that were failed more than 2 times, including the test cases that were sent for rework;
2. Run the test cases that were failed once and then passed;
3. Run the test cases that was passed only once and were never executed again;
4. Run all other test cases that are included into the testing plan.

Benefits of Automated Regression Testing

Usually regression tests are not changed greatly from version to version and can be automated. It is recommended to automate the main use cases, if the regression testing is executed more than 3-5 times.

The “pros” for tests automation are:

• automated tests increase testing speed without influence on a general testing process;
• automated tests save human and time resources, because the scripts are coded once and can be executed as long as the tests remain relevant;
• automated tests are clear and transparent, all the tests are formalized because of written scenarios;
• automated tests can be scheduled and executed without human control, test results are available anytime to anyone who is interested in.

The “cons” for automation testing are:

• automated tests must remain up to date to be effective;
• some errors cannot be discovered by automated tests;
• automated tests can not define exact error, just display that some scripts are failed.

There are lots of automation tools and development teams may select the ones they like.

The Advantages of Regression Testing

Regression testing is important part of the development and effective way of support, because it helps to:

• reduce 50-80% of defects that were caused by the code changes intended to fix a bug or create a new feature;
• keep documentation up to date: description of test cases, test results, software specification;
• support the stable functioning of the system and the clients’ confidence in the software quality;
• reduce the costs of technical support by decreasing the number of emergency situations during the commercial operation.