Testing Strategies to Deal with the Demanding Dozen-Part 3

Over the last two installments we have outlined 12 challenges that financial organizations face. In this post, we move from analysis to action so that you and your team can thrive rather than just survive in the current environment.

Identify the threats and weaknesses then prioritize.
Prioritize your test needs based on various risk factors. Identify events that would have the most impact on your organization, and evaluate the probability of those events occurring. The resources and staff-hours you assign to address these risks must be proportionate to the perceived risk. Identify a high-risk or high-loss situation in which improved testing can substantially reduce risks or losses. A few weeks of effort in planning and implementing a comprehensive test strategy can pay big dividends in increased system confidence, reduced time to market for new payment types, and lowered blood pressures at your next meeting with your system auditor.

Determine where automation can help improve your test procedures.
The goal is to capitalize on any investment in test tools by automating as much of your testing as possible. Automation is the most efficient choice for repetitive tasks, such as regression testing or stress testing, while manual testing may be more appropriate for user acceptance testing. Thorough regression testing is difficult to achieve if testers are required to manually enter each transaction standing at an ATM. Instead, suggest that testers use a script-based tool to automate this effort—saving hours (and aching feet) each time they test.

Additionally, note that regression testing is critical but is not sufficient to guarantee your implementation. Applications may stand up well to regression testing, but fail in production when transaction volumes cause bottlenecks in your application or operating environment. Automated tools are needed to test volume and performance as well. However, avoid tools that force you to spend more time “automating” than you do testing.

Automate testing too.
Automating tests facilitates wider application coverage, improves test depth, and introduces fewer errors into the test pool. For example, if your organization uses test-driven development (a process that has developers write very specific unit tests prior to development to test their changes to the code), then suggest that the developers create additional tests to verify the integration of their code into the system. Those integration tests could then be passed on to QA for use during regression testing and to production support for use when testing fixes.

Share across department lines and build a collaborative team.
Testing is done throughout your organization by developers, QA personnel, system analysts, and production support. Take an inventory of the test tools used by each department and examine the similarities between your test goals and those of other areas. Make the best use of test cases by sharing and reusing them.

Talk to industry peers and discuss their test approach and the tools they use.
If you already use tools, ask your vendor to refer you to other users who may have valuable insights on how to get more out of the tools, or who may be able to recommend other processes or tools. If you currently don’t use automated test tools, you may be able to obtain tool recommendations from other financial institutions, software development companies that you have used, or your ATM or EFT vendors.

Communicate actions and results while identifying possible solutions.
Increase the visibility of the testing already occurring across your organization. Alert management to the threats you’ve identified and how proper testing can thwart them. Create a sense of urgency. Hundreds of thousands of dollars can be lost when a single obscure change is not properly regression tested, or when fraudulent transactions are not trapped, even though an alert was posted, because an effective test case could not be constructed. Make certain managers understand that the right tools and tests can save your financial institution from loss of credibility as well as financial losses. Provide an analysis of the tools used by your organization, as well as any tools you want to acquire, including a comparison of their features and their suitability for the testing you have targeted.

Acquire best of breed additional tools and thoroughly train your team in them.
Ensure you make complete use of these solutions throughout your organization.

To deliver error-free code on time and within budget, you must have tools for:

Unit testing to validate individual software components or modules against the design specifications.

Conformance testing to verify the implementation’s conformance to industry standards.

Functional testing to validate that the application meets the requirement specifications.

Performance testing using benchmarks to compare the system to past performance, or to determine the system’s ability to scale to anticipated processing rates (as in stress testing).

Stress testing to evaluate a system at or beyond the limits of standard processing rates.

Integration testing to verify that any updated modules perform correctly when integrated with other modules.

User acceptance testing during which the system is tested by prospective users and is evaluated based on whether or not the customer will accept the new system.

Regression testing to verify by consistent, repeatable testing that any additions or changes made to the system have not caused the failure of any previously provided functions.

End-to-End testing to verify that every step in the payments value chain is functioning properly.

We hope these recommendations are useful. Post a comment or two below to share your opinion or any suggestions you have that might help all of us dedicated to being the best at financial services testing. Good luck – but don’t depend on it in your testing, right?