This is a guest post by Peter G. Walen.
The software your company makes and possibly uses, is different from all other software. You have built a custom solution for how your company works and operates. Your software is unlike any other. Unfortunately, your test problems are not unique.
This is particularly true when it comes to payment system test management. It goes beyond defining test cases and loading them into Jira or MS ADO or any other tool. IT requires a slight difference in thinking that is focused on risk and exposure to risk that many organizations simply don’t consider.
Even if your software is using a third-party provider to handle payments, you must approach this as it is your company that is on the line. If there are problems impacting your customers, they won’t care if the problem was with a third party or not. If there is a breach or leak of cards or transaction data, it will be your company making the news.
Business Need & Requirements
Test management starts with doing a careful review and analysis of the business problem to be solved and how the requirements express those needs. We need to understand what the intent of the project or change is. We need to close gaps in our business case. This will help close gaps in expectations and defined requirements.
There are always gaps. There are always presumptions that everyone understands what the needs are or how the current system works. Sometimes, these gaps will not be identified until late in development. Once in a while, they get found after the software is deployed and in use.
A patch might help close this in most situations. In payment systems, the gaps are often only found when there is a breach. By then it is too late. We need to identify gaps and risks throughout the cycle.
I remember one instance when I worked with payment card processing software where we found a gap in testing, a week before the scheduled deployment and delivery date. This resulted in a complete revision of how my team did the test design. We changed everything.
From that point on, test design was done before software design. We tested the requirements by building decision trees and mind maps where we identified holes in the system. These holes went back to the Business Analysts with an explanation of what we found and a request for clarification.
It did not make the test team popular with the sales team for a time. What they quickly realized was that subsequent projects were ready for delivery sooner because any possible design problems had been accounted for before “design” began.
Managing the Process
The problem is often a presumption that things will simply work. It is human nature, I suspect, to believe that a good development team, or test team, will always work without errors or problems. The gap between needs and requirements is only one aspect.
In payment systems, the challenge is to know what gaps to look for. Managing the Process around testing is related to managing test case design and development. When money is involved, the need to test rigorously seems obvious. Accounts need to balance and transactions all need to be accounted for.
The more obscure questions come from the form of a payment system being exercised. Are you developing for an online retailer? What about a mixed online and brick-and-mortar store? Do they need to handle the same payment methods? Are some methods accepted one way and not the other? What about capacity? Do they limit the cards or payment methods for some transactions and not others?
While these are “requirements” of the system, they need special consideration for testing. Repeating similar steps with minor variations between them is usually seen as waste, or at least unnecessary. When exercising payment methods and cards, this repetition may be precisely what is needed.
One other consideration is there. This speaks to challenges when it comes to any kind of payment process. While most financial transactions look for immediate balancing of transactions, to make sure money is credited to the correct accounts, with payment transactions, it is perhaps more important to prevent future problems by scrutinizing how the transactions themselves get handled.
From storing and tracking transaction data, including payment account information, to monitoring how this information is transmitted, there is one thing that must be at the forefront of testing with payment systems. It is the thing that will reduce the chances of your company making headlines in ways you never want: Security.
Security
Organizations can use OWASP guidelines for their systems. This is a great start. All software development groups should be aware of and apply these ideas. When it comes to payment transactions, PCI compliance is a huge issue.
Both of these go well beyond “nice to have” or “next release.” Many will know about OWASP. PCI is an area a fair number of organizations don’t worry about, trusting their vendor to handle that. Except if there is a problem around payment handling, your customers will be coming to you for solutions and redress.
You must be aware of what these requirements entail. Then you need to figure out how to test for them. Then you need to make sure that every change to an area around payments is tested for compliance with PCI.
Then you need to test it. You need to test it over and over again. Each release needs this testing. I suggest each build, but without a robust automation framework that can handle variability in scenarios while testing the “same” script repeatedly, this may be a great deal of work. Still, you must test it.
The Testing
One objection people often raise with this idea, it is wasteful and consumes a lot of resources. It almost certainly consumes a lot of resources. However, is it wasteful to look for vulnerabilities in payment systems? Possibly.
I know of no instance where a system breach did not cost the company far, far more than testing would have. The point of automating these tests, particularly tests for the more common threats, is to allow your people to dig deep into the software and see what they have not considered before. Then these can be tested, Once they have been exercised and mapped out, they too can be automated to allow your test teams to look for other vulnerabilities.
I cannot recall a single case where the savings of not testing for these issues was greater than the costs. From legal fees to damaged, if not ruined reputations, the costs are extraordinary.
What I have found, is that by incorporating OWASP guidelines and testing for PCI compliance a regular part of your testing helps keep security concerns in the minds of the test and development teams. The need for constant vigilance becomes a shared marker for your organization.
The rewards are simply that you have not made headlines for a payment card breach.
Yet.
