Having experienced test automation and also having done it myself for various BPM solutions, I have come across some repeating issues.

 Automating test scenarios is usually aimed at regression testing to ensure that the solution is still working as it was before some minor changes were made to the solution. Test automation has the advantage of being able to quickly execute pre-defined scenarios which otherwise would have to be executed manually. If the solution has undergone signs of changes, the automated scenarios would have to be changed before automated regression testing. Another possibility would be to keep in mind that the scenarios, which comprise changes, are supposed to fail. Scenarios may also comprise testing that has already been tested by your software supplier and you may not want to repeat that again.

There are two issues that will be addressed today. I will first describe the issues separately and subsequently I will propose an approach for each solution.

1. Test Coverage and Timing

How much of the solution should be tested automatically and when? Depending on the coverage of the scenarios, one can achieve more or less certainty regarding possible regression of the solution under test. The ultimate goal should neither be low nor high coverage. The problem with high coverage is that tests are more likely to fail because of minor changes; worse yet, maintaining automated test scenarios will be rather costly and time-consuming. In order to make efficient use of resources, I would advise to limit automated test scenarios to functional testing. If you include UI testing, creating and maintaining the automated tests will be costly and time-consuming. Moreover, it would require a lot of effort to maintain them.

2. Product versus Solution Testing

When designing automated test scenarios for a BPM solution, I would advise to limit oneself to solution testing. Beware of testing the product, BPM tool, itself. If you use standard functionalities from the product, for example a calendar widget, it is unnecessary to test that again, because the supplier has done that already. But if the functionality is critical to your solution, you should include it in your test. Keep in mind this is the exception and not the rule.

The above-mentioned issues are only a small selection of issues one can come across in automated testing. Usually the issues you have to deal with are the same as for manual testing but then only more extensive.

 

About Jessica

Jessica Otto Senior BPM Test BPMCompanyI'm a BPM consultant focussing on QA and testing. I have been involved in several projects in the financial sector. Until recently I worked for some software companies. Having experienced diverging project approaches and software in different environments, I decided to become a BPM consultant.
In addition to my technical experience, I have a legal background and perform research in legal knowledge management.