Why We Need a Test Strategy

So today I was asked, by a major project, to review their test plan and test cases. I asked ‘can I see the test strategy’. There were lots of blank faces around the virtual room and, after a long pause, the Project manager suggested that a strategy was not needed.

NFR Non test Strategy - ad banner

So it began. The project was delivering a policy based data loss security service that would sit in line and therefore had the potential to be massively disruptive. There would be in excess of 50 rules and each rule had multiple permutations of keywords, data types and regular expressions to match upon.

My first two follow up questions were ‘are you testing every rule and every permutation of every rule’. Answers from around the project team ranged from no we are sampling, through we will take a risk-based approach, to yes we are testing everything.

My next question was ‘what is the scope of the the test all functional areas, non-functional requirements, all intergetrations, unit, system, integration etc’. Again there was no consensus in the room.

I labour the point but without a test strategy, it would have been easy to make a case to either approve or reject the test plan.

Parts of the Test Strategy

Feeling helpful and wanting my project colleagues to success, I agreed to sit with them and help to create the test approach. We went through each part of the test strategy to define:

Scope

For the scope, we agree the extent of the functional and non-fucntional testing to execute and, as importantly, what would not be tested. We agreed that each policy would be tested but not every permutation of the rules, in effect adopting a risk-based approach (more on that in a later post).

Approach

We agreed the roles and responsibilities for each party in the creation of the test data and test cases. Who would create, approve and execute the test plan. How evidence would be captured and defects recorded. We agreed how the test cases would be linked back to the strategy to ensure coverage.

Test Environment

We agreed the environment in which the tests would be executed. Unusually we were having to test in production as there were no lower environments for the security tooling. To avoid production disruption we were able to scope the tools to only apply to a small number of test users.

Test Tools

As well as the tools used to execute and automate testing, we also discussed where the test plan would be stored, how each test case was evidenced and the platform used for defect recording and tracking to closure.

Exit Criteria

Finally we recorded what would be deemed a successful test and allow the project to move onto the next phase in the SDLC.

Test Strategy Begets the Test Plan

With the test strategy agreed it was an easy ask to review the plan and individual test cases to ensure that we had complete coverage of the test strategy objectives. On first pass we identified gaps that might well have otherwise been missed.

Continue Reading

As an Amazon Associate I may be paid commission for qualifying purchases