Last year, I spent a lot of time working on projects that expanded my skills outside of testing. This year, I’m hoping to focus my time on developing testing skills.
I came to this decision after reflecting on some problems that I ran into during my annual review. I described the first annual goal to come from this meeting in my last diary entry, Reprioritizing Testing. Now, I’d like to add two more goals: revamping our test plans and developing a strategy for security testing.
Get TestRail FREE for 30 days!
Rethink Test Plans
One of the problems I want to work on comes from a recent internal audit. The auditor asked to see our test plan. I demonstrated that our testing consists primarily of testing the features and defects that are included in the release.
Each feature and defect in the management tool is accompanied by a test charter that shows how it was tested. Then I showed her where our non-functional testing charters are kept, including post-build smoke tours, usability and performance charters, etc. Regression is typically planned using a risk-based approach, but this product had not been released yet, so there was no planned regression testing.
The auditor said this was unacceptable. She had expected a formal document that included an explicit test approach statement, a list of tests to be completed, a schedule, lists of tools to be used, lists of people who would work on the project, and more. She considered our informal needs-based test planning to be a “non-conformity.”
Create a Test Plan, Not a Test Plan Document
The auditor encouraged us to conform to the test plan template another group uses. Their template seemed long and rigid, but there were parts of it that could be useful. They address environment as a part of the test plan. Our products generally need to be tested in three environments, and we consider that throughout the test process. We have sometimes taken those expected environments for granted, and that results in a surprise when we discover that a customer will be using an environment that we didn’t test against.
I asked the auditor to change the official wording to “collaborate with” other groups in order to create a test plan that would be acceptable. I believe that by talking to other test groups to see what their test planning looks like and by researching alternative test plan formats, we can find something that will satisfy both our auditors and our needs without creating excess work.
Develop a Security Testing Strategy
During the same audit, the auditor asked us about our security testing strategy. Until very recently, our products were exclusively client-server systems hosted on customer domains. For us, security meant simply testing that login works, that password restrictions were obeyed, and that the internal permission-based features were enforced. We focused less on intruder-access and more on making sure the users could perform the right tasks for their roles.
The auditor asked to see our test cases for security testing. We had a few, but the security system was so simple and had seen so few changes that we rarely ran them. In the few regression tests where we had included the security tests, we never found a problem. For the expected environment of a customer domain, the product was highly secure, and we were able to take that for granted throughout the lifecycle of the product.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
Evolving Technology Means Evolving Security
As our product portfolio evolves, I realize we can no longer rely on our customers’ security systems to keep our software secure. We are beginning to develop products that are served up on a web browser. For now, we still expect them to be served on a customer’s domain, but we have included technology that will allow the servers to be hosted on the cloud. This means our idea of security testing will become broader and will require more technical skills. We’re going to need to learn about authentication, authorization, penetration testing and much more.
My game plan for this year is to collect resources about security testing, work with a developer to gain a better understanding of it and share that information with the other testers on my team. I’ve already taken a hands-on workshop to learn more about examining the messages sent between the browser and the server using a web proxy server, and the problems that can be found there.
One of the developers on my team is part of a group developing security products for teams across the company to use. I’m hoping to work with him using the materials we’ve been collecting to create a curriculum that can be used by my team to help us catch up with the current state of security testing for the web. If it’s successful, I may be able to share it with other teams in the early stages of moving toward testing for the web.
Sometimes, Less Is More
Normally when I plan my year, it’s difficult to narrow down my goals. I often end up with five to seven annual goals because my interests are so varied and there’s so much I want to learn and do.
By reducing my number of goals this year to three, all centered around testing, I hope to accomplish more and focus more deeply on expanding my testing skills. By spending time on developing some expertise, I may also be better equipped to collaborate with my team and other teams in my company. I think it’s going to be a good year.
This is a guest posting by Carol Brands. Carol is a Software Tester at DNV GL Software. Originally from New Orleans, she is now based in Oregon and has lived there for about 13 years. Carol is also a volunteer at the Association for Software Testing.
Test Automation – Anywhere, Anytime
