| layout | title | nav_exclude |
|---|---|---|
default |
Comparison with Pact |
true |
| Specmatic.in | Pact.io | Pactflow.io Paid | |
|---|---|---|---|
| Approach | Contract Driven Development (CDD) | Code First, Consumer Driven Contract Testing | Bi-Directional Contract Testing (BDCT) |
| OpenAPI Support | YES | NO OpenAPIs and Pact are designed with different ends in mind | PARTIAL , only supports OAS as Provider Spec), Several Limitations - No Remote Refs / Single document approach, Need to dereference "$Ref", and more |
| Suitable for Consumer First Development Style (Frontend Before Backend) | YES Consumer Team can author the API Specification, leverage it as mock / stub and then share it with Provider team as a guideline | YES Pact requires starting with Consumer development first by mocking the Provider and then leveraging these mocks to generate the Contract for the Provider | YES same as Pact OSS |
| Suitable for Provider First Development Style (Backend Before Frontend, public facing APIs, ec.) | YES Provider Team can author the API Specification, leverage as tests and then share it with Consumer team to stub the Provider | NO Pact is better suited for Consumer First Development Style | YES Provider has to publish their OAS to PactFlow server, they will only get feedback after Consumer has publish their Contract and PactFlow has completed Cross Contract Verification |
| Suitable for API First Development Style | YES Consumer and Provider Teams collaboratively author API Specification and only then being development | NO It does not support Contract / API Spec first Development Style | YES Consumers still cannot use the API Specification as Stub, they will only have to Cross Contract validate with the Specification |
| Parallel Development of Consumer and Provider | YES This is one of the primary capabilities of Specmatic, to allow teams to independently develop and confidently deploy microservices | NO Consumer development has to be ahead of Provider development | YES |
| Backward Compatibility Verification #NOCODE | YES This is one of the flagship capabilities in Specmatic. Contract vs Contract comparison, no need to implement the changes in Provider or Consumer Code | NO Compatibility can only be possibly detected after an actual change happens in Provider code, two version of Pact JSONs cannot be compared to see if the changes are compatible | NO Same as Pact OSS |
| API Specifications as Test #NOCODE | YES Specmatic can generate and run tests, both happy path and boundary conditions, all with just your OpenAPI specifications | NO Pact JSONs are the ones that are run as test, OpenAPI cannot be run as tests | YES WITH CAVEAT - External Tool Dependency with Bring Your Own testing tool such as Dredd - Also several limitations / considerations |
| API Specifications as Stub #NOCODE | YES OpenAPI is all you need to run a stub server, no need to define a stub, just the expectation data | NO Pact DSL necessary to define Stubs | NO When leveraging Bring Your Own mocks, the frameworks such as Mountebank, Wiremock, etc. have to be setup |
| Smart Mocks (Stale Stub Detection / Expectation Validation against API Specification) | YES Immediate Feedback, expectations that are not as per API Spec are immediately rejected with proper error message | NO / Not Applicable. Contracts are generated based on the expectation, stale stub detection can only happen when pact json is run as test against Provider | NO / Not Applicable. PactFlow allows Bring Your Own (BYO) mocks, even with Mountebank the approach has no immediate verification of stub data with Specification. Mountebank imposters have to be converted to Pact Files which are then uploaded to PactFlow where it is compared with Provider's OAS. Almost 3 levels of indirection before you know whether the stubs are stale. |
| Unfragmented Devex / Minimal learning curve | YES You just need API Specification of your choice such as OpenAPI and Specmatic - No additional DSLs | NO Need to pick up several tools to achieve a basic level of coverage - Pact, Pact DSL for Mocking, Pact Broker, API Specification such as OpenAPI, Tool to run API Specification as Test. Has significant learning curve. | NO The Bring Your Own tools approach adds to the list of Pact OSS - PactFlow, Pact DSL for Mocking AND / OR other tools like Mountebank, API Specification such as OpenAPI, Ready API, Dredd, etc. to run API Specification as Test, PactFlow Broker Server. If teams within an organization are leveraging a mix and match of these tools that makes it hard for people to move across teams. |
| Can all stakeholders participate / author Contracts / API Specifications | YES Devs, architects, test engineers, practically anyone who can author an OpenAPI Spec. | NO Developer/SDET only because they have to generate to Pact JSON / Contract only after they have written a mock with Pact DSL | PARTIAL Only Provider Spec / OAS can be written by anyone. Consumer Contract has to be written / generated by Consumer Developer / Testers. |
| Suitable for adoption in Greenfield Projects | YES Specmatic helps team become productive quickly with Contract Driven Development | YES | NO Better suited for "Retrofitting contract-tests onto existing systems" |
| Suitable for adoption in Brownfield Projects | YES Specmatic can immediately provide value on existing projects and helps the team slowly adopt Contract Driven Development | NO Better suited for "Greenfield projects" | YES WITH CAVEAT - Involves Guarantee Tradeoffs |
| Minimal / No Leakage for Compatibility Issues | YES Specmatic keeps a tight check on both the Consumer and Provider and immediately provides feedback where they deviate from API Specification thereby preventing compatibility Issues | YES While Code First, Consumer Driven Contract has a narrow set of use cases where it is suitable, within that narrow area it is able to prevent / reduce compatibility issues. | NO BiDirectional Contract Testing has lower guarantees in comparison to Consumer Driven Contract / Pact OSS |
| Low Cost of Maintenance | YES Since Specmatic is leveraging your API Specifications there is no maintenance overhead. You just need to maintain your Stub / Expectation data. | NO In comparison to Specmatic which only needs API Specifications, you need to maintain the Pact JSONs. | NO Has higher cost of maintenance in comparison with Pact OSS |
| Open Source | YES License - MIT | YES | NO - Involves Proprietary Code |
| Language and Platform Agnostic | YES Is able to work completely as a standalone executable | NO Pact DSL is language specific. While Pact supports several languages, if your language is not supported, you will have to try alternate approaches. | YES WITH CAVEAT - With Bring Your Own Tools approach this should be possible, however it comes at additional effort. Example: If you are using Mountebank instead of Pact Mocking, imposters have to be converted to Pact Files |
| Unlimited Team Size | YES | YES | NO Team sizes vary based on the Pricing Plan |
| Broker Less | YES Requires no server setup | NO Pact Broker has to be run on a server where Pacts are uploaded, requires maintenance. | NO PactFlow involves cloud based Pact Broker, on-prem options seem to be available with enterprise pricing plan |