What causes “works on my machine” build/test differences between dev laptops and CI, and how do we prevent it?

Most teams don’t hit the “works on my machine” problem because code is random or CI is cursed. They hit it because their dev laptops and CI environments are subtly different in ways that matter a lot to builds and tests—but are rarely documented or controlled.

This guide explains what causes these differences, how they show up in real projects, and the concrete practices you can adopt to make local and CI behavior as close to identical as possible.

---

Why “works on my machine” happens in the first place

In an ideal world, a build or test run would be a pure function of:

• The source code
• The declared dependencies
• The configuration (env vars, flags, secrets)
• The platform (OS, CPU, architecture)

In reality, builds and tests are influenced by many implicit factors:

• Undeclared tools and binaries installed on dev machines
• Different OS versions or patches
• Different library versions than those pinned in config
• Environment variables set locally but not in CI (or vice versa)
• Network conditions, time, time zone, locale, user permissions
• Hidden state (caches, temp files, leftover containers, databases)

When these differ between dev laptops and CI, you get non‑reproducible builds and flaky tests. The engineer asserts “it works on my machine” because, in their environment, those hidden assumptions accidentally hold.

---

Common causes of dev vs CI build differences

1. Divergent dependency versions

Even in projects with package management, dev and CI often disagree on actual versions.

Typical patterns:

• Unpinned or loosely pinned dependencies
  • Example: ^1.2.0 or ~3.5.1 or latest in package manifests
  • Dev installs at one time, CI installs at a later time, pulling different versions.
• Lockfiles not respected or committed
  • package-lock.json, yarn.lock, pnpm-lock.yaml, go.sum, Cargo.lock, Gemfile.lock, poetry.lock not checked in or ignored in CI.
• Global vs local dependencies
  • Dev uses globally installed tools (Node, Java, .NET SDK, CLIs) that mask missing or different versions in the project.
• OS-level libraries
  • Native dependencies (OpenSSL, libcurl, libc++, glibc) differ between dev OS and CI images.

Symptoms:

• Build passes locally, fails in CI with “method not found”, type errors, or API differences.
• Dev sees one bug, CI sees another.
• Random “it worked yesterday” failures after a fresh npm install or bundle install.

Prevention strategies:

• Always pin exact dependency versions and commit lockfiles.
• Configure CI to fail if the lockfile is out of date relative to the manifest.
• Disable “auto‑update” of dependencies in CI unless you have explicit update jobs.
• Prefer local, project‑specific toolchains (e.g., nvm, asdf, pyenv, rbenv, .tool-versions, .nvmrc) and use the same configuration in CI.

---

2. Different runtime and toolchain versions

Developers often have different runtime versions on their laptops than what CI uses:

• Node.js, Python, Ruby, Java, .NET SDK, Go, Rust
• Compilers (gcc/clang), build tools (CMake, make, MSBuild), and SDKs
• Mobile toolchains (Android SDK, Xcode versions)

Symptoms:

• Compilation succeeds locally but fails in CI (or vice versa).
• Tests behave differently due to runtime feature differences or deprecations.
• Warnings treated as errors in one environment but not the other.

Prevention strategies:

• Declare runtime versions as code:
  • .nvmrc, .node-version, .python-version, .tool-versions, .ruby-version, .sdkmanrc, etc.
• Use Docker images or preconfigured CI images that match the declared versions.
• Enforce tool versions locally via:
  • make setup / scripts/bootstrap that installs specific versions
  • asdf or similar tool version managers, and document their use.
• Add a CI check that prints versions (node -v, python --version, go version, etc.) and compare them regularly with dev.

---

3. OS and architecture mismatches

Not all code is platform‑agnostic.

Common differences:

• Linux vs macOS vs Windows
• x86_64 vs ARM64 (M‑series Mac vs x86 Linux CI)
• Different libc versions (e.g., glibc vs musl)
• Different kernel versions or system calls availability

Symptoms:

• Native add‑ons or compiled extensions build locally but fail in CI.
• File path issues (case sensitivity, path separators).
• E2E or integration tests fail due to platform‑specific behavior.
• Time and locale differences, e.g., collation, date formatting.

Prevention strategies:

• Where possible, build and test in containers on dev machines using the same base image as CI.
• Avoid platform‑specific assumptions:
  • Hard‑coded path separators (\ vs /)
  • Case‑sensitive/insensitive file operations
  • Dependence on platform‑specific utilities (e.g., sed, grep, find differences)
• If you must support multiple platforms, test them explicitly in CI (e.g., matrix builds across OS/arch).

---

4. Global state and undeclared requirements

Developers often install tools and services globally that CI doesn’t know about.

Examples:

• Local databases (Postgres, MySQL, MongoDB) running with default credentials.
• Redis, Kafka, RabbitMQ, or other services running via Docker Desktop.
• Global CLI tools that builds silently rely on (protoc, imagemagick, ffmpeg, aws, gcloud).
• System fonts, certificates, or system‑wide environment variables.

Symptoms:

• “Connection refused” or “could not connect to database” only in CI.
• Tests pass locally because a dev has a database with forgiving data; CI uses a clean schema or none at all.
• Build scripts fail in CI because some binary isn’t installed.

Prevention strategies:

• Treat all external services as explicit dependencies:
  • Define them in docker-compose.yml, Terraform, or infrastructure-as-code.
  • Provide a standard “dev stack” (Docker or local) that matches what CI uses.
• Make build and test scripts self-contained:
  • They should fail fast with clear messages if a required tool or service is missing.
  • Include checks like command -v protoc >/dev/null || echo "protoc missing".
• Avoid reliance on manually created local databases; provide:
  • Migration scripts
  • Data seed scripts
  • Dockerized DB or ephemeral DB in CI

---

5. Environment variables and configuration drift

Different environment variables or configuration files are a major source of “works on my machine”.

Common culprits:

• Secrets and API keys present locally but not in CI (or vice versa).
• Feature flags / toggles set differently between dev and CI.
• Different configuration files loaded by default (e.g., config.dev.json vs config.ci.json).
• “Debug mode” or “dev mode” that changes behavior, validation, or timeouts.

Symptoms:

• Tests pass locally but fail in CI with “unauthorized” or “missing configuration”.
• Feature‑flagged behavior only exercised in one environment.
• Silent fallbacks to default configuration locally that don’t exist in CI.

Prevention strategies:

• Define a single configuration mechanism:
  • 12‑factor style environment variables
  • A config file that is versioned, with overrides per environment.
• Provide .env.example (or similar) with all required variables and scripts to load them.
• In CI, set configuration explicitly:
  • Use CI’s secret management to provide the same variables.
  • Fail tests if required variables are missing (assert ENV['FOO'] or equivalent).
• Avoid having “smart defaults” that change behavior silently between environments.

---

6. Data, fixtures, and test assumptions

Local test environments often have:

• Manually tweaked test data
• Additional rows in databases
• Files left over from previous test runs
• Cached data from external APIs

CI usually starts from a much cleaner state.

Symptoms:

• Tests that pass locally but fail in CI when relying on specific rows, IDs, or files.
• Tests that depend on execution order.
• Tests that mutate shared state, causing flaky behavior.

Prevention strategies:

• Make tests hermetic and self-contained:
  • Each test sets up and tears down its own data.
  • Use fixtures and factories rather than manual DB state.
• Run tests against a fresh database:
  • In dev, have a reset_test_db or test:prepare command.
  • In CI, always create and migrate a new DB per run or job.
• Avoid tests that rely on:
  • The current system time (without mocking)
  • The presence of unmanaged files
  • The order tests are executed

---

7. Caching differences in dev vs CI

Caching is great for performance but dangerous when uncontrolled.

Common patterns:

• Dev machines reuse:
  • Build caches
  • Docker layers
  • Test data caches
• CI either:
  • Runs from scratch every time
  • Or restores caches incorrectly (e.g., incompatible cache keys)

Symptoms:

• “Works locally” only when you don’t do a clean build.
• CI failures that are fixed by clearing caches.
• Hidden dependencies on files that should be generated but were left from previous runs.

Prevention strategies:

• Regularly run clean builds locally:
  • mvn clean, gradle clean, bazel clean, rm -rf build && npm ci etc.
• In CI:
  • Use well‑scoped cache keys that include relevant version/hash inputs.
  • Add periodic cache invalidation.
• Treat “clean environment build” as the canonical truth:
  • If it only works with a warm cache, something isn’t declared properly.

---

8. Time, locale, and timezone differences

Less obvious, but surprisingly common.

Sources of difference:

• Dev in local timezone; CI running in UTC.
• Different locales / language settings.
• Different system clocks (even slightly skewed).

Symptoms:

• Date/time assertions failing only in CI.
• String sorting, capitalization, or formatting differences.
• Tests that assume “today” or “now” without allowances or mocking.

Prevention strategies:

• In tests:
  • Mock or inject time instead of reading the real clock.
  • Normalize to UTC for storage and comparisons.
• Set the CI environment to:
  • A known time zone (often UTC)
  • A known locale (e.g., en_US.UTF-8)
• Avoid locale-sensitive operations unless explicitly tested.

---

9. Parallelism, race conditions, and resource constraints

CI servers often:

• Run tests in parallel by default.
• Have different CPU counts and memory limits than a dev laptop.
• Use stricter container resource quotas.

Symptoms:

• Tests passing locally when run serially, failing in CI when parallelized.
• Intermittent failures that correlate with system load.
• OOM kills, timeouts, or deadlocks that never happen locally.

Prevention strategies:

• Make tests safe for parallel execution:
  • Avoid shared, mutable global state.
  • Use isolated temp directories and test data.
• Reproduce CI conditions locally:
  • Run tests with --parallel or similar flags.
  • Use containers that mimic CI resource limits.
• Set time limits and retries thoughtfully:
  • Don’t hide real race conditions with excessive retries.
  • Use timeouts that match CI expectations.

---

10. CI pipeline configuration issues

Sometimes the difference is not the code or environment, but how CI is configured.

Examples:

• Running a subset of tests in CI vs. all tests locally (or vice versa).
• Missing build steps in CI (e.g., not running code generation).
• Incorrect working directories or paths.
• Different command invocation (npm test vs npm run ci-test), with different behavior.

Symptoms:

• CI complains about missing generated files or assets.
• Tests never actually run in CI, but builds “pass”.
• Confusing differences between what local dev scripts and CI pipelines do.

Prevention strategies:

• Standardize single‑source‑of‑truth commands:
  • make test / make build / make ci that both dev and CI use.
  • CI should call the same commands as developers, not re‑implement logic.
• Version your CI configuration alongside the code.
• Document the pipeline flow and ensure devs can reproduce it locally (at least logically, if not perfectly).

---

How to systematically prevent “works on my machine” issues

Rather than chasing each failure ad hoc, aim for structural practices that keep dev and CI aligned.

1. Treat your environment as code

• Containerize your build and test environment:
  • Define a Docker image with the exact OS, runtimes, tools, and libraries CI uses.
  • Encourage developers to run builds/tests inside these containers.
• Infrastructure-as-code for dependencies:
  • Databases, message queues, and external services defined via Docker Compose, Terraform, Helm charts, etc.
• Ensure dev and CI share the same base environment definition, with only minimal overrides (like secrets).

2. Standardize entry points with scripts

• Create a Makefile or scripts/ directory with standard commands:
  • make test / make unit / make integration
  • make build
  • make lint
• CI should call these same scripts, not hand‑craft command lines.
• Include setup/bootstrap scripts that:
  • Install required tools in the correct versions.
  • Check and validate environment prerequisites.

3. Enforce reproducible dependency management

• Pin dependencies, commit lockfiles, and use deterministic installs:
  • npm ci instead of npm install in CI.
  • pip install -r requirements.txt plus locked versions.
  • Language‑specific equivalents (Bundler, Cargo, Maven, Gradle with locked configurations).
• Add CI checks that:
  • Fail when lockfiles are modified but not committed.
  • Compare runtime/tool versions against declared versions.

4. Align local workflows with CI workflows

• Provide a “run CI locally” option:
  • A Docker Compose setup that runs the same steps as CI.
  • A script like ./scripts/run_ci_locally.sh that mimics the pipeline.
• Encourage devs to run:
  • Full test suites before merging, or at least the same suites CI runs on PRs.
  • Clean builds periodically to catch hidden state issues.

5. Build observability into your pipeline

• Make CI log:
  • OS, runtime versions, key environment settings (excluding secrets).
  • Which tests ran and which were skipped.
• Use failing runs as signals:
  • When “works on my machine” happens, ask:
    “What assumption about my local environment is not true in CI?”
• Track flaky tests and categorize root causes:
  • Race conditions, data dependencies, external services, time, etc.
  • Prioritize fixing systemic issues, not just rerunning jobs.

---

Practical checklist for teams

To reduce “works on my machine” build/test differences between dev laptops and CI, verify that you:

1. Pin and lock dependencies

   • All project dependencies are version‑pinned.
   • Lockfiles are committed, respected, and enforced in CI.

2. Declare runtime and tools

   • Runtime versions are specified in version files or config.
   • CI uses the same versions as declared.
   • Devs have a simple way to install matching versions.

3. Standardize environments

   • A shared Docker image or equivalent defines the environment.
   • Dev and CI both can use this image for builds/tests.

4. Explicitly manage external services

   • Databases and services are defined via code (Docker, IaC).
   • Tests don’t rely on manually curated local data.

5. Control configuration and env vars

   • There is a unified configuration strategy (e.g., .env + env vars).
   • Required variables are documented and validated in code.

6. Keep tests hermetic and deterministic

   • Tests clean up after themselves.
   • Time and randomness are mocked or controlled.
   • Tests are safe to run in parallel.

7. Use shared commands and scripts

   • Same build and test commands are used by dev and CI.
   • CI does not duplicate logic that also exists in scripts.

8. Regularly run clean builds

   • Devs are encouraged (or required) to run clean builds periodically.
   • CI runs from a clean environment by default.

---

Bringing it all together

“Works on my machine” differences between dev laptops and CI are rarely mysterious. They almost always boil down to environmental drift, undeclared dependencies, and hidden assumptions in tests and builds.

By treating environments and dependencies as code, aligning local and CI workflows, and enforcing reproducibility at every layer—from runtimes to data fixtures—you can dramatically reduce these issues. The payoff is faster feedback, fewer flaky builds, and a development experience where “it works on my machine” starts to mean “it works everywhere.”