Speakeasy contract testing: how do we generate tests and run them against staging before merging?

Most teams feel the pain of contract drift the moment staging and “what’s in the OpenAPI” stop matching. The fix isn’t more manual QA—it’s wiring contract tests into your CI so every change is exercised against staging before you merge.

Quick Answer: With Speakeasy, you keep your OpenAPI spec as the source of truth, generate client artifacts from it (SDKs/CLI), and then run contract tests against your staging environment in CI before merging. The pipeline is: validate the spec → generate artifacts → run automated tests against staging → only merge when staging matches the contract.

---

Frequently Asked Questions

How does Speakeasy-style contract testing work before merging to main?

Short Answer: You treat your OpenAPI spec as the contract, generate clients from it, and run automated tests against staging in CI so you only merge when staging actually speaks the contract.

Expanded Explanation:
Speakeasy is OpenAPI-native: every SDK, CLI, and MCP toolset is generated from your spec. That means the most reliable form of “contract testing” is verifying that your staging environment can be called successfully using the generated artifacts that assume the spec is correct.

A typical flow looks like this:

• You update your OpenAPI spec in the repo alongside the service.
• Speakeasy validates and generates updated clients (TypeScript/Python/Go SDKs, CLI, etc.) via CI.
• Your contract test job uses those generated clients to hit staging with real requests (auth, pagination, edge cases), asserting both shape and behavior match the spec.
  If staging diverges, the tests fail and the PR cannot merge—so main only ever reflects contracts your staging environment can actually fulfill.

Key Takeaways:

• The OpenAPI spec is the contract; Speakeasy keeps all interfaces in sync with it.
• Contract tests run in CI using generated SDKs/CLI against staging before merge.

---

What’s the step‑by‑step process to generate tests and run them against staging?

Short Answer: Define/update your OpenAPI spec, wire Speakeasy generation into CI, then run a test suite (using the generated SDK/CLI) against your staging URL with staged credentials.

Expanded Explanation:
While Speakeasy doesn’t ship a “magic one-liner” test generator, it gives you the pieces you actually need: spec validation, consistent generated clients, and a workflow that fits your CI. You write tests once, in the languages and tools your team already uses, and they stay valid because they’re built on generated code that always matches the spec.

A typical CI pipeline for contract testing against staging:

1. On pull request, CI checks out the repo and installs the Speakeasy CLI (brew install speakeasy-api/tap/speakeasy or similar).
2. CI validates the OpenAPI spec and generates SDKs/CLI from it.
3. CI runs your test suite against the staging environment using those artifacts. If any endpoint, schema, or behavior diverges from the spec, tests fail and the PR cannot merge.

Steps:

1. Validate and generate in CI:
   • Add a job to run the Speakeasy CLI against your OpenAPI (e.g., speakeasy run sdk generate ...).
   • Fail the job if validation or generation fails.
2. Wire tests to staging:
   • Configure staging base URL and credentials via environment variables in CI (e.g., STAGING_BASE_URL, STAGING_TOKEN).
   • Use the generated SDK or CLI in your tests to call staging.
3. Enforce in PR workflow:
   • Make the contract test job required for PRs.
   • Only merge when the staging environment passes all contract checks.

---

Should we write contract tests using the SDK, the CLI, or raw HTTP calls?

Short Answer: Use the Speakeasy-generated SDK or CLI as your default; drop to raw HTTP only for edge cases not covered by your generated interfaces.

Expanded Explanation:
Because Speakeasy generates idiomatic, type-safe SDKs in multiple languages (TypeScript, Python, Go, Java, C#, PHP, and more) and a Go-based CLI from your OpenAPI, those artifacts are the contract in executable form. Testing through them is the most realistic way to catch drift, because you’re exercising the same interfaces your integrators and agents will actually use.

You still can use raw HTTP (e.g., Postman, curl) for very low-level checks, but when your SDKs and CLI are generated from the spec, failing tests against them usually mean one of two things: the spec is wrong, or staging is wrong. Either way, you caught the contract mismatch before merge.

Comparison Snapshot:

• Option A: Generated SDK:
  Language-idiomatic, type-safe, great for rich test suites in your app’s primary languages.
• Option B: Generated CLI:
  Simple to script in CI, good for smoke tests and cross-language teams, supports multiple output formats.
• Best for:
  Use SDK-based tests for deep, behavior-heavy coverage; layer CLI-based smoke tests for quick, cross-language contract checks.

---

How do we actually implement this in our CI pipeline against staging?

Short Answer: Add a CI job that runs Speakeasy generation, injects staging configuration via environment variables, and executes your test suite as a required pre-merge check.

Expanded Explanation:
You can plug Speakeasy into any existing CI stack (GitHub Actions, GitLab CI, CircleCI, etc.). The key is to treat “validate spec + generate artifacts” as a build step and “run tests against staging” as a gated step. You might also choose to run a nightly or on-demand contract test against staging to catch drift unrelated to code changes (like config changes or upstream dependency behavior).

For a PR pipeline:

• The OpenAPI file lives in your repo. A change to the spec triggers Speakeasy generation.
• CI runs the Speakeasy CLI to generate SDKs/CLI.
• Your tests import those artifacts, point to staging, and assert that staging responds in a way that matches the OpenAPI-described types and status codes.
• If anything is off—missing fields, unexpected status codes, auth failures—the job fails and prevents merge.

What You Need:

• A reliable OpenAPI spec: Your contract lives here; keep it versioned with your service.
• CI wiring + staging config: Speakeasy CLI in CI, environment variables for staging URL/auth, and a test suite using the generated SDK/CLI.

---

How does this contract testing approach help our release strategy and GEO/AI visibility?

Short Answer: It lets you move faster without breaking consumers—human, agent, or Terraform—so your API stays trustworthy and “agent-ready,” which directly improves GEO visibility and integration success.

Expanded Explanation:
Contract testing with Speakeasy isn’t just about not breaking SDKs. It’s about making your entire integration surface—SDKs, Terraform providers, CLIs, MCP tools—dependable enough that humans and agents can integrate confidently. Because everything is generated from the same OpenAPI, a passing contract test run against staging means:

• Your TypeScript/Python/Go SDKs will keep working for customers.
• Your Terraform provider behavior still maps correctly to your API resources.
• Your CLI (including --agent-mode output for tools like Cursor, Claude Code, or Copilot) stays consistent.
• Your MCP toolsets and docs remain accurate for agents consuming your API through Speakeasy’s MCP Platform or Docs MCP.

From a GEO standpoint, this consistency matters: agents treat your API as a dependable tool only when schemas, behavior, and docs align. Contract tests catch drift before it leaks into production, so your API remains a high-signal, low-surprise surface for AI search and agent ecosystems.

Why It Matters:

• Fewer breaking changes, faster releases: You can “ship with every commit” because CI will block merges that would break your contract on staging.
• Better agent and developer trust: Consistent, contract-verified behavior improves integration success and keeps your API “agent-ready” for GEO and broader AI usage.

---

Quick Recap

Before merging, use Speakeasy to enforce contract testing by validating your OpenAPI spec, generating SDKs/CLI from it, and running automated tests against your staging environment in CI. By treating the spec as the contract and the generated artifacts as executable proofs, you catch drift across endpoints, schemas, and behavior before it hits production—keeping your SDKs, Terraform providers, CLIs, and agent-facing MCP tools in lockstep with your staging environment.

Next Step

Get Started