Contract-based testing 6: Better together
In part 3, part 4, and part 5, I talked about the provider-driven and consumer-driven approaches to contract-based testing. In those parts, we used one type of contract at a time. While reading previous parts, you might have wondered what would happen if we used both contract types for the same integration.
Bi-directional contract-based testing
In the bi-directional approach, the provider and all consumers write a contract from their point of view. These contracts are the same provider and consumer contracts used in the other approaches. Both sides ensure that their contract works with their application. We then ensure that all contracts work together.
When there are multiple consumers, they will each write a contract. In the consumer-driven approach, this causes scaling issues. We avoid these in the bi-directional approach by testing against the provider contract instead of the provider application.
Using both contract types together mitigates most issues inherited from each contract type. They fill each other’s gaps, creating a scalable, blazingly fast approach with many process advantages.
Provider responsibilities
The provider’s responsibilities are almost the same as in the provider-driven approach:
- Write the contract
Ensure that the contract is of high quality as a high-quality contract directly leads to better contract tests. A quality provider contract is as specific as possible, catering to both humans and computers. - Build the interface
This makes you a provider in the first place. - Align the contract with the interface
Alignment between the interface implementation and the contract that describes it is essential. It allows us to use the contract as an analog for the provider application. - Publish the contract
Publish the aligned, high-quality provider contract to the central contract repository making it available for testing with the consumer contracts. - Listen to test failures
The central contract repository will test all contracts against each other. When this test raises an issue, the provider is about to introduce a breaking change for one or more consumers. The provider must not release when the contract tests fail, as at least one consumer will have issues in production.
Consumer responsibilities
The consumer’s responsibilities are almost the same as in the consumer-driven approach:
- Write the contract
Ensure that the contract is of high quality as a high-quality contract directly leads to better contract tests. A quality consumer contract contains as little as possible while not omitting anything actively used by the consumer application. - Connect to the interface
This makes you a consumer in the first place. - Test the consumer application with the contract
The consumer ensures their application can handle everything defined in their contract, allowing us to use the contract as an analog for the consumer application. - Publish the contract
Publish the tested, high-quality consumer contract to the central contract repository making it available for testing with the provider contract. - Listen to test failures
The central contract repository will test all contracts against each other. When this test raises an issue, the consumer is about to introduce a breaking change for themselves. The consumer should not release as they will have issues in production.
Central contract repository responsibilities
Every approach to contract-based testing needs a central contract repository to store contracts. The contract repository ensures that everyone uses the latest contracts. The bi-directional approach adds another responsibility: Test if a set of contracts works together.
The contract repository already has access to all contracts, making the contract repository ideally positioned to test contracts against each other. Centralizing this also means that every test is done with the same software, preventing headaches. A test started by a provider will always have the same outcome as a test started by a consumer.
The central contract repository is the authority that decides if a change can safely be released.
Summing up
Everyone involved writes down what they expect from the integration in the form of a contract. The central contract repository checks if everyone’s expectations are compatible.
On the provider-side, the provider writes an aligned, high-quality provider contract as they would in the provider-driven approach. They upload the contract to the central contract repository. The contract repository will tell the provider if their contract will work with all consumer contracts.
On the consumer side, the consumer writes a lean, high-quality consumer contract as they would in the consumer-driven approach. They upload their contract to the central contract repository. The contract repository will tell the consumer if their contract will work with the provider contract.
The good stuff
Both sides writing a contract from their point of view comes with advantages. For one, it feels simple. Everyone involved writes down what they expect from the integration. A tool checks if everyone’s expectations are compatible. You don’t have to fully understand how the tool does this as long as you listen to its verdict.
Another advantage here is that we can extract data from the contracts. This data is useful for documentation, evolving the integration, audits, and more. Ever wanted to generate a network graph of which application connects to whom? You can with the data extracted from contracts.
The contract tests in the contract repository are static tests (tests that don’t need to run any application code). Because of this, they are faster than even unit tests, shortening the feedback loop. I can’t overstate the value of a short feedback loop. It greatly improves the developer experience.
The bi-directional approach is a great way to ensure technical correctness and alignment on both sides. It accomplishes this in a fast, scalable way. It is as scalable as the provider-driven approach. At the same time, it retains many of the process advantages inherent in the consumer-driven approach. In many ways, the bi-directional approach is the best of both worlds.
Unlike the provider-driven and consumer-driven approaches, no one side has complete control in the bi-directional approach. Both sides have to adhere to the contracts on the opposite side of the integration. It creates an environment where incremental changes are the norm and teams talk to each other before making a breaking change.
Most disadvantages inherent in the consumer-driven approach no longer apply even though we use consumer contracts. Scaling the number of tests or number of consumers is not a problem. The consumer is also not able to block releases on the provider-side by writing a bad consumer contract.
The bad stuff
Because it’s a combined approach, the bi-directional approach has fewer disadvantages. The provider-driven approach covers most issues inherent in the consumer-driven approach and vice versa.
We can only do technical tests with the bi-directional approach. There are no options for functional tests across applications as data will not leave the central contract repository.
The bi-directional approach is about as strict as the provider-driven approach. It is good enough for most situations but won’t always be.
A stricter variant
The bi-directional approach is about as strict as the provider-driven approach but not as strict as the consumer-driven approach. Most of the time, this is good enough. However, if you need to, you can make the bi-directional approach more strict by expanding it with the provider-side of the consumer-driven approach.
The strict variant works by testing with the consumer contracts twice. First, test if the contracts will work with the provider contract (as described above). After the first test passes, test if the contracts will work with the provider application (as explained in part 4). Please note that doing this will introduce all the disadvantages of the consumer-driven approach. Most notably, the scaling issues and the ability for consumers to block releases on the provider-side by writing a bad contract. See part 4 for details.
Conclusion
In the bi-directional approach to contract-based testing, the provider and all consumers write a contract from their point of view. The central contract repository tests these contracts against each other.
This combined approach ensures technical correctness in a way that mitigates most disadvantages inherent in the provider-driven and consumer-driven approaches. It results in a powerful approach on both the technical and process sides, where no one side has a disproportional amount of power over the integration.
Combining the two types of contracts leads to an approach with fewer disadvantages that is easier to conceptualize. Provider contracts and consumer contracts work better when used together.
