Contract-based testing 1: Introduction
Modern software architecture promotes building tightly scoped, reusable applications. Because of this, big organisations have hundreds of applications working together at any given time. These applications talk to each other via their interfaces. These complex networks of applications can cause hard-to-trace bugs which are often difficult to test.
Contract-based testing is a fundamental way to approach testing the interfaces between applications. It gives you the confidence that two applications can talk to each other with no need for an end-to-end environment.
In this series of articles, I will talk about what contract-based testing is, why you should use it, and how its processes work.
Definition
First things first. A definition by pact.io:
[Contract-based testing is ensuring] that … applications will work correctly together by checking each application in isolation … [against] a shared understanding that is documented in a contract.
So we are testing with full applications here. This is generally considered a job for end-to-end tests on an expensive end-to-end environment. When contract-based testing, we’re testing the full application in isolation instead. Because of this, we don’t need a test environment, we can execute these tests on our own computer.
Testing pyramid
For contract tests, we’re not using an end-to-end environment, but are still testing full applications. This means we can place contract tests just below end-to-end tests in the testing pyramid.
Contracts
As the name of the paradigm implies, this approach to testing relies heavily on contracts. A contract is the documentation of an application interface that serves as the shared understanding between the interface provider and its consumers.
By using this shared understanding, we can do all kinds of interesting things. Some of them done by humans and others by computers. The contract — our shared understanding — is only useful when it adheres to all the following principles.
A contract …
- Is the single source of truth for the interface
- Contains everything you need to understand and use the interface
- Contains all potential states of the interface
- Is both human-readable and machine-readable
The first three principles might sound familiar to those working with formalized requirement documents. Nothing new there. It’s the fourth principle that opens a lot of interesting doors for us. It makes sure that our interface requirements are readable for everyone and everything. This allows us to bridge the gap between business requirements and test automation.
Bridging the gap between the business and the application development cycle is great for gaining speed and accuracy in developing new features. Contract-based testing does this by incorporating the single-source-of-truth-requirement-document as a fundamental part of the tests for both the interface provider and their consumers.
REST context
Most people discuss contract-based testing in the context of REST APIs, but it’s not limited to REST. It does not distinguish between protocols. Contract-based testing is a way of thinking about ensuring the quality of integration between applications. Or, in simpler words: Contract-based testing is a test paradigm.
That being said, I will only give REST API based examples to simplify these articles. When I’m talking about REST things, please remember that the principles of contract-based testing work equally well for other integration protocols like SOAP, Kafka, etc.
Approaches
There are two ways to approach contract-based testing. They distinguish themselves by who writes the contract. I will elaborate on both approaches in part 3 and part 4, but below is a sneak peek of how these approaches work.
Provider-driven contract-based testing
In the provider-driven approach, the provider writes the interface contract. The provider makes sure their interface implementation matches the contract before sharing the contract with their consumers. The interface consumers then use the contract to upgrade their tests.
Consumer-driven contract-based testing
In the consumer-driven approach, all interface consumers write an interface contract. This way the consumers are dictating the features of the interface. The consumers make sure their application can handle all the features dictated in the contract before sharing the contract with the provider. The interface provider will then use all contracts to make sure that the interface can deliver all consumer-defined features.
Conclusion
Contract-based testing is a test paradigm for testing full applications with no need for a dedicated test environment. It relies heavily on human-readable and machine-readable requirement documents called contracts. This allows the paradigm to bridge the gap between the business and test automation.
There are two approaches to contract-based testing. I will explain the details of how these approaches work in part 3 and part 4.
But first, in part 2, I want to take a step back to talk about why we should test as little as possible on end-to-end environments. I’ll also talk about some issues that pop up when we move away from end-to-end testing.
