Software Testing strategy

We are doing it wrong if our software testing is slow , manual , repetitive , time consuming.

We are doing it wrong if there is not enough low level testing(unit , integration) and too many tests run through GUI and even a larger number of manual tests.

We are doing it wrong if our testing strategy is this.

The remedy is automation. Automate your tests so that you can change your codebase without the fear of breaking things.

In his book “Succeeding with Agile”, Mike Cohn describes the concept of a test automation pyramid, describing three levels of test automation, their relation and their relative importance.

For modern microservice application this pyramid may seems overly simplistic but the essence is to write tests with different granularity. The more high-level you get the fewer tests you should have.

What are the different granular levels ? What are the low level tests ? What are the high level tests?

Start with the Unit test…..

Unit tests

This should be the foundation of the testsuite. Each test should have the narrowest scope. It should make sure a particular unit of the code base works as expected. The number of unit tests should largely outnumber all other types of test in the test suite. Start with one test class per production class. Unit test should cover all the public interface of the class.

According to Jay Fields' Working Effectively with Unit Tests, there are two types of unit tests

solitary unit tests for tests that stub all dependencies and sociable unit tests for tests that allow talking to real dependencies Whatever the approach, at the end of the day automated tests are more important.

Integration Tests

Any modern application integrates with some other parts . Unit tests leave out these parts to make better isolation. But these need to be tested and Integration testing is there to help. It is relatively slower than unit tests.

Integration testing need to be done narrowly testing one integration point at a time replacing other dependencies with test doubles

Test one integration point at a time.

Contract Tests

Modern software applications are developed in parallel by many teams as microservices. Splitting your system into many small services often means the services need to communicate with each other via certain interfaces. The interfaces can be REST, gRPC or Queues.

The consumer of an interface writes tests to check its format. The consumer team then publishes these tests so that the publishing team can easily execute them.

Contract tests focus on the contract of the external service and not necessarily on the data.

UI Tests

This test looks to test the User Interface of the application. A user action in the UI component should trigger the right action with the right data and the right expected state change.

Modern UI frameworks (react, vue.js, angular, etc) often include their own tools and helpers that make it easy to thoroughly test these interactions at a very low level (unit tests).

For server side rendering applications, Selenium based tests will be useful.

End-to-End Tests

UI tests driven by web drivers are a good example of end-to-end tests. The end-to-end tests come with their own set of problems. They are notoriously unstable and fail for unexpected and unforeseeable reasons. It is terribly slow and requires a lot of maintenance.

You should reserve End to End testing for the most important transactions within your application. Automate your most important steps of the user journey into end to end tests.

Rest API end to end tests are less flaky compared to UI tests.

Acceptance Tests

Acceptance tests are about User perspective and it's not about technical perspective. Acceptance tests are better approached in BDD fashion. It can be a nice trick to shift your mindset from implementation details to users' needs by writing BDD-style tests.

Exploratory Testing

Use your creativity here. Go with the destructive mindset. Issues found in these tests are gaps in the low level tests suite and it should be fixed in the lower level. Your delivery process should not be slowed by exploratory tests.

Ref :

https://alisterbscott.com/kb/testing-pyramids/

https://www.softwaretestingmagazine.com/knowledge/the-software-testing-cupcake-anti-pattern/

https://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid

https://fabiopereira.me/blog/2010/05/27/ttdd-tautological-test-driven-development-anti-pattern/

https://martinfowler.com/articles/practical-test-pyramid.html