Contract Testing

• A technique for testing an integration point by checking each application in isolation to ensure the messages it sends or receives conforms to a shared understanding that is documented in a “contract”.
• A form of testing, where you test the "contract" between the services/systems communicating to each other.

The "Contract"

• A specification of what data Consumer request and get from the Provider

  • e.g. When consumer calls GET /api/promotions, the Provider would return JSON with array of promotions

• The contract looks just like how we mock HTTP request with Nock, then what’s the difference?

  nock('http://www.example.com')
    .get('/api/promotions')
    .reply(200, { promotions: [{ ... }] })
  nock('http://www.example.com')
    .get('/api/promotions')
    .reply(200, { promotions: [{ ... }] })

Contract vs Mocking

• When mocking requests, we tend to use it for 3rd party APIs, we don’t care how the endpoint will behave if we send data like query strings, since we don’t have any control of them.

• But since we are building both of the services calling each other (Consumer & Provider) the Contract works on both sides:

  • On consumer side, the contract emulates the provider (i.e. as a HTTP mock server) which receives request and returns data
  • On provider side, the contract emulates the consumer (i.e. cURL, Postman) which make the request and verify the returning data

Who defines the contract?

• Provider -> "Provider-driven Contract Testing"

  • Pros : Good when we have many consumers or consumers don’t understand the server and/or provide meaningful contracts
  • Cons : The workflow will be one-way, and longer or no feedback loop from consumer will happen (https://pactflow.io/blog/the-curious-case-for-the-provider-driven-contract)

• Consumer -> "Consumer-driven Contract Testing"

  • Faster feedback loop
  • More practical
  • https://pact.io

Pact

• Multi-language support (JS, Ruby, Python, Go, etc.)
• Supports most integrations
• Supports CI/CD with Pactflow or self-hosted

Usage

• Start from consumer side (Consumer-driven)
• Provider verifies the contract

Consumer

Write unit/integration tests with Pact mocks

1. Setup Pact (It will create mocked Provider)

Consumer

Write unit/integration tests with Pact mocks

2. Setup mock request & response with Interaction

Consumer

Write unit/integration tests with Pact mocks

3. Write tests as usual

Consumer

Run the contract test

• When the test runs, it will create a contract as JSON file

Consumer

Share the contract

• Then we share the contract with the provider to use
  • Send the file directly
  • Publish to the "Broker" pact_broker (Free), Pactflow (Paid)

Provider

Get the contract

• Setup Pact to fetch the contract from the file, or the broker

Provider

• Start the provider test server, then verify the contract

Provider

• Start the provider test server, then verify the contract

Provider

• If using Pactflow, it will support detecting the branch, version and the contract result
  • 

Provider

• Success means that the contract test on Consumer version X and Provider version Y is passed and is safe to be deployed
• Use can-i-deploy CLI tool on CI to check the status against consumer & provider version
  • 

Workflow

Workflow

Summary

• Contract testing
  • Increase confidence
  • Fast
  • Run independently
• Caveats
  • Not "replacing" e2e tests, nor integration tests

References

• https://pactflow.io/how-pact-works
• https://www.slideshare.net/sebrose/contract-testing-and-pact
• https://slides.com/mariedrake/contract-testing-101
• https://www.youtube.com/playlist?list=PLwy9Bnco-IpfZ72VQ7hce8GicVZs7nm0i