How do we set up CircleCI self-hosted runners in our VPC and route only specific jobs/workflows to them?

Most teams reach for self-hosted runners when “just run it in the cloud” stops working—usually the moment you need private VPC access, strict data controls, or heavyweight dependencies. The good news: you can keep CircleCI’s managed control plane and selectively route only the jobs that require your VPC into your own infrastructure.

In this guide, I’ll walk through how I’d design and deploy self-hosted runners in a VPC, then wire CircleCI so only specific jobs/workflows use them—while everything else stays on CircleCI’s cloud executors.

Quick Answer: CircleCI self-hosted runners let you run jobs inside your own VPC while keeping CircleCI’s cloud control plane. You attach one or more runner resource classes to a machine or container in your VPC, then reference those classes only in the jobs that need private access or special control.

The Quick Overview

What It Is: CircleCI self-hosted runners are agents that execute CircleCI jobs on infrastructure you control—inside your VPC, on-prem, or in specific regions.
Who It Is For: Platform and DevOps teams that need VPC-only access, strict data residency, or custom OS/hardware, but don’t want to build their own CI/CD control plane.
Core Problem Solved: You can keep shipping at AI speed on CircleCI cloud while routing sensitive or specialized workloads to your own VPC with tight network and governance controls.

How It Works

At a high level, you register a runner “resource class” in CircleCI, install the runner agent on hosts inside your VPC, and bind that host to the resource class. In your .circleci/config.yml, you opt jobs into that runner by specifying resource_class: <namespace>/<name> on each job. Jobs without that resource class continue to run on CircleCI’s cloud executors.

Here’s the flow:

Define runner topology in your VPC:
Decide which subnets, security groups, and instance types you’ll use, then create one or more runner resource classes to represent them.
Install and register self-hosted runners:
Install the CircleCI runner agent on your VPC instances (machine or container), register them with your CircleCI org, and bind them to your resource classes.
Route specific jobs/workflows via config:
In .circleci/config.yml, target those resource classes only from the jobs that need VPC access or custom behavior, and keep all other jobs on CircleCI’s cloud.

Step 1: Design your VPC runner architecture

Before you touch CircleCI, get clear on what actually needs to live in your VPC.

Common VPC-bound workloads:

Jobs that need to reach private databases, internal APIs, or message queues.
Jobs that must stay inside a specific region or network for compliance.
Jobs that rely on large, internal artifacts (e.g., monorepo caches, internal Docker registries).
Security-sensitive build steps (e.g., signing, key handling) that cannot leave your network.

Typical design decisions:

Subnets:
- Private subnets for runner hosts (no public IP)
- NAT gateway or egress proxy if runners need outbound internet (e.g., to reach circleci.com, Git providers, package registries)
Security groups:
- Outbound HTTPS to CircleCI and your Git provider
- Inbound only from your admin or management tools (SSH, SSM, etc.)
- Inbound from internal services the jobs must reach (DB, API, cache)
Instance/host strategy:
- Long-lived runners for consistent, low-latency access to internal services.
- Auto-scaling group or Kubernetes cluster if you expect bursty workloads.
Network egress controls:
For tighter governance, you can restrict egress so runners can only reach:
- CircleCI domains and APIs
- Your Git provider (GitHub/GitLab/Bitbucket)
- Restricted list of external registries/package feeds

You don’t need to move everything into the VPC—only what truly demands it. That’s where selective routing comes in.

Step 2: Create a self-hosted runner resource class

Resource classes are the “handles” you’ll use in your pipeline to point a job at a specific runner pool.

Log into CircleCI as an org admin.
Go to Organization Settings → Self-hosted Runners.
Create a new runner:
- Give it a name (for example: vpc-build, vpc-secure, vpc-db-access).
- CircleCI will create a resource class like:
  your-namespace/vpc-build
- Generate a token used by the runner agent to authenticate.

You can repeat this to create multiple pools:

your-namespace/vpc-build for standard build/test in the VPC
your-namespace/vpc-secure for signing, secrets handling, or PII workloads
your-namespace/vpc-heavy for GPU/large-memory workloads (if you pair with appropriate hardware)

Step 3: Provision and register runner hosts in your VPC

Next, install the CircleCI runner agent on VPC hosts and bind them to the resource class.

CircleCI supports different runner types (machine runner, container runner). The pattern is similar:

Provision hosts in your VPC:
- EC2, GCE, on-prem VMs, or Kubernetes nodes
- Attach them to the right subnets and security groups
- Ensure they have outbound HTTPS connectivity to CircleCI and Git provider
Install the CircleCI runner agent:
- Download the runner package for your OS.
- Install and configure the service (systemd, etc.) or container.
Configure runner with your resource class:

Example (pseudo-config):
```
circleci-runner configure \
  --token <RUNNER_TOKEN> \
  --resource-class your-namespace/vpc-build \
  --name vpc-runner-01
```
Key things to verify:
- The agent is running as a service and starts on boot.
- The host can reach CircleCI endpoints over HTTPS.
- Logs show the runner successfully connecting and waiting for work.
Scale out:
- Add more hosts with the same resource class as you grow.
- Use an auto-scaling group or cluster autoscaler to adjust capacity based on load.

At this point, CircleCI knows “there is capacity for your-namespace/vpc-build” but no jobs are using it yet.

Step 4: Route only specific jobs to the VPC runners

This is where the selective routing happens. Jobs run on the VPC runner only if you explicitly target its resource class.

Basic job targeting

In .circleci/config.yml:

version: 2.1

jobs:
  build_cloud:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run: echo "Running on CircleCI cloud executors"

  build_in_vpc:
    machine:
      resource_class: your-namespace/vpc-build
    steps:
      - checkout
      - run: echo "Running on self-hosted runner in VPC"
      - run: ./scripts/build_internal.sh

workflows:
  version: 2
  build_and_test:
    jobs:
      - build_cloud
      - build_in_vpc

Key details:

Cloud jobs: Use docker:/machine: executors with CircleCI’s shared resource classes like medium, large etc. (no custom resource class).
VPC jobs: Use resource_class: your-namespace/vpc-build to explicitly target your self-hosted pool.

If you don’t set resource_class to your namespace, the job will not run on your VPC runner.

Routing entire workflows vs single jobs

You can choose the blast radius:

Only specific jobs in a workflow go through VPC runners (e.g., db_migrations, internal_e2e).
An entire workflow can be VPC-bound by giving every job in that workflow a VPC resource class.

Example: hybrid workflow

workflows:
  version: 2
  build_test_deploy:
    jobs:
      - build_cloud
      - unit_tests_cloud:
          requires:
            - build_cloud
      - internal_e2e_in_vpc:
          requires:
            - build_cloud
      - deploy_in_vpc:
          requires:
            - internal_e2e_in_vpc

Where:

jobs:
  unit_tests_cloud:
    docker:
      - image: cimg/node:current
    steps:
      - checkout
      - run: npm test

  internal_e2e_in_vpc:
    machine:
      resource_class: your-namespace/vpc-build
    steps:
      - checkout
      - run: ./scripts/e2e_against_internal_env.sh

  deploy_in_vpc:
    machine:
      resource_class: your-namespace/vpc-secure
    steps:
      - checkout
      - run: ./scripts/deploy_to_prod.sh

Unit tests stay in the cloud; only E2E + deploy touch the VPC.

Step 5: Use contexts and policies for safe customization

The power of self-hosted runners is also the risk: you’re closer to production systems and sensitive data. I strongly recommend pairing VPC runners with CircleCI governance controls.

Contexts for secrets and environment separation

Use contexts to manage secrets only available to VPC jobs:

jobs:
  internal_e2e_in_vpc:
    machine:
      resource_class: your-namespace/vpc-build
    context: [vpc-internal]
    steps:
      - checkout
      - run: ./scripts/e2e_against_internal_env.sh

You can:

Restrict vpc-internal context to specific projects/branches.
Put database credentials, internal API tokens, and VPC-specific config in that context.
Ensure cloud jobs never get these secrets.

Policy checks to gate access

With CircleCI’s policy and approval mechanisms, you can enforce:

Only signed commits or protected branches can trigger VPC-deploying workflows.
Manual approvals before a job that touches production services on your VPC.
Restrictions on who can modify workflows that reference VPC resource classes.

Example approval gate:

workflows:
  version: 2
  secure_release:
    jobs:
      - build_cloud
      - internal_e2e_in_vpc:
          requires:
            - build_cloud
      - hold_for_approval:
          type: approval
          requires:
            - internal_e2e_in_vpc
      - deploy_in_vpc:
          requires:
            - hold_for_approval

You keep “expert-in-the-loop” control while letting the rest of the pipeline run with minimal oversight.

Features & Benefits Breakdown

Core Feature	What It Does	Primary Benefit
Self-hosted runner pools	Run jobs on infrastructure inside your VPC using resource classes.	Keep sensitive workloads and data on your network with full control.
Selective job routing	Target specific jobs/workflows to runner resource classes in config.	Only VPC-critical steps leave CircleCI cloud, minimizing complexity.
Hybrid execution model	Mix cloud executors and VPC runners in a single workflow.	Optimize for speed and cost while maintaining enterprise-grade control.

Ideal Use Cases

Best for VPC-only systems integration: Because it lets you run integration/e2e tests against private databases and internal APIs without exposing them to the internet.
Best for controlled, secure deploys: Because deploy jobs can run on tightly locked-down hosts with approvals and policy checks, while build/test stays on fast cloud executors.

Limitations & Considerations

Runner capacity management:
You own scaling and availability of your VPC runners. If there’s no capacity for a resource class, jobs will queue. Use auto-scaling and monitoring on your side to keep pipelines flowing.
Network egress and connectivity:
Runners need outbound connectivity to CircleCI and your Git provider. If you over-restrict egress or use aggressive proxies, runs can fail. Validate connectivity and DNS up front and document the allowed endpoints.

Additional considerations:

OS/patching: You’re responsible for patching and hardening the hosts.
Caching/artifacts: Network egress to CircleCI for caches/workspaces can become a factor; plan bandwidth and policies accordingly.

Pricing & Plans

Self-hosted runners use the same usage-based model as cloud executors. You’re billed based on usage time, not where the job runs.

Key points:

Usage for self-hosted runners contributes to your overall compute usage.
Concurrency can be higher than your nominal plan if you provide more self-hosted capacity; the practical limit is runner capacity and any plan-specific caps.
You don’t pay CircleCI for the underlying machines themselves—you provision and pay your cloud/on-prem provider.

Plan guidance:

Team / mid-size plans: Best for teams needing a small number of VPC-bound workloads (e.g., internal E2E, staging/prod deploys) while the bulk of jobs run on CircleCI cloud.
Scale / enterprise plans: Best for organizations standardizing on hybrid execution across many repos, with multiple runner pools, strict policies, and golden paths for build/test/deploy.

For current pricing and concurrency details, confirm on CircleCI’s pricing page or with sales; specifics can change.

Frequently Asked Questions

How do I ensure only certain jobs use our VPC self-hosted runners?

Short Answer: Only jobs that explicitly specify your self-hosted runner resource_class will use your VPC runners; everything else stays on CircleCI cloud.

Details:
In .circleci/config.yml, you opt into VPC execution by setting resource_class: your-namespace/your-runner on a job that uses the machine/container runner. Jobs that use standard CircleCI resource classes (like medium, large) automatically run on CircleCI’s cloud executors. This opt-in model is what lets you keep 90% of your pipeline in the cloud and route just the 10% that needs private access into your VPC.

Can I run some workflows entirely on self-hosted runners and others entirely on cloud?

Short Answer: Yes. You can mix and match at the workflow and job level within the same project or org.

Details:
You might have:

A “fast feedback” workflow that builds and tests on CircleCI cloud only (no self-hosted resource classes).
A “secure deploy” workflow where every job specifies a VPC runner resource class.
Hybrid workflows that combine the two, cloud for unit tests and VPC for internal E2E + deploy.

CircleCI’s control plane orchestrates all of it; the executor is chosen per job. This gives you a clean path to progressively adopt self-hosted runners: start with a single deploy job, then move more critical steps into the VPC as needed, without disrupting your existing pipelines.

Summary

You don’t need to move your entire CI/CD system into your VPC to get enterprise-grade control. With CircleCI self-hosted runners, you keep the cloud control plane and scale, then drop execution into your VPC only where it matters—jobs that hit private services, handle sensitive data, or require specialized hardware.

By:

Designing runner pools inside your VPC,
Registering them as resource classes,
And explicitly routing only certain jobs and workflows to those resource classes,

you get the best of both worlds: AI-speed delivery on CircleCI cloud plus VPC-level control for your most sensitive workloads. Add contexts and policy checks, and you’ve got a governed, hybrid delivery system that stays fast without sacrificing confidence.

Next Step

Get Started