Why Playwright Tests Pass Locally but Fail in CI

A Playwright test clicks Save and expects a confirmation message. It passes repeatedly on a developer's machine. In CI, the report occasionally says:

Expected string: "Saved"
Received string: "Saving..."

The immediate temptation is to add a two-second delay, increase every timeout, or retry the whole suite. None tells you why the assertion ran before the application reached the expected state.

A CI-only failure is evidence of a difference between the conditions under which the test ran, not proof that CI is unreliable. The useful question is: Which assumption did the test make about timing, data, server startup, or browser environment?

This guide works through a small hypothetical Save form, then gives a repeatable way to isolate those causes. The Playwright snippets target the current Playwright Test API. They are illustrative code reviewed against official documentation, not results from a newly executed CI experiment.

The Save test: clicking is not waiting for completion

Imagine a page that changes its status from Idle to Saving..., sends a request, and finally displays Saved. The first version of the test looks reasonable:

import { test, expect } from '@playwright/test'

test('saves the project — race-prone assertion', async ({ page }) => {
  await page.goto('/projects/123')
  await page.getByRole('button', { name: 'Save' }).click()

  // Reads the current text once; it does not wait for "Saved".
  const status = await page.getByRole('status').textContent()
  expect(status).toBe('Saved')
})

Playwright's click() waits for actionability conditions such as a usable target; it does not guarantee that the application's subsequent network request and state transition have finished. textContent() returns the text at the moment you ask for it. The plain expect(status).toBe(...) comparison does not retry.

On a fast local run, the response may arrive before the read. With a slower service, a busier worker, or a different scheduling order, the test can observe Saving.... This explanation is a possible mechanism, not a diagnosis of a specific CI run without its trace.

Change the assertion to observe the state the user actually needs:

test('saves the project — waits for confirmation', async ({ page }) => {
  await page.goto('/projects/123')
  await page.getByRole('button', { name: 'Save' }).click()

  await expect(page.getByRole('status')).toHaveText('Saved')
})

toHaveText() is a web-first assertion: Playwright rechecks the locator until the condition passes or the assertion timeout expires. That is different from sleeping for a fixed duration. The assertion still fails if the app never reaches Saved, which is exactly the signal you want. See Playwright's assertion behavior.

A minimal way to observe the distinction

This self-contained example uses page.setContent() rather than an external app or database. It models a delayed completion after a click:

// tests/save-status.spec.ts — illustrative; needs @playwright/test installed
import { test, expect } from '@playwright/test'

test('waits for a delayed save confirmation', async ({ page }) => {
  await page.setContent(`
    <button type="button">Save</button>
    <p role="status">Idle</p>
    <script>
      document.querySelector('button').addEventListener('click', () => {
        const status = document.querySelector('[role="status"]')
        status.textContent = 'Saving...'
        setTimeout(() => { status.textContent = 'Saved' }, 500)
      })
    </script>
  `)

  await page.getByRole('button', { name: 'Save' }).click()
  await expect(page.getByRole('status')).toHaveText('Saved')
})

A deliberately race-prone variation would replace the last line with expect(await page.getByRole('status').textContent()).toBe('Saved'). In an isolated editorial browser check, an equivalent Python Playwright script (Playwright 1.57.0, headless Chromium 144.0.7559.96) read Saving... immediately in three of three iterations, then observed Saved using a retrying assertion in all three. That is an observed result for this synthetic interaction; the TypeScript sample itself was not executed, and the result does not reproduce real network latency, backend persistence, or CI resource contention. It illustrates one failure mechanism, not every flaky test.

Start with the failed attempt, not the retry that passed

If the CI run already failed, inspect the first failure before changing code:

1. Find the failed assertion or action and its exact timeout. A locator.click timeout means something different from expect(...).toHaveText timing out.
2. Open the trace and inspect the DOM snapshot immediately before the failure. Was the button present and enabled? Did the status become Saving...? Did it ever become Saved?
3. Inspect the network events around that step. Was there a failing request, a slow response, or no request at all?
4. Compare the environment: target URL, browser/project, worker count, server readiness, test data, and the commit actually checked out.
5. Change one variable and rerun. A passing retry alone does not identify the cause.

Playwright's Trace Viewer lets you inspect actions, DOM snapshots, and network events. If you need the original failing attempt, configure trace: 'retain-on-first-failure'. The frequently shown on-first-retry mode records the retry, not the initial failure. This difference matters when a retry passes and the original evidence is otherwise lost. See trace modes.

// playwright.config.ts — example diagnostics settings
import { defineConfig } from '@playwright/test'

export default defineConfig({
  testDir: './tests',
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 1 : 0,
  workers: process.env.CI ? 1 : undefined,
  globalTimeout: 20 * 60_000,
  use: {
    trace: 'retain-on-first-failure',
  },
})

One retry can help you distinguish a consistent failure from an intermittent one. Playwright labels a test flaky when it initially fails but passes on retry. This classification does not turn the first failure into a successful test design. See retries and the CI guidance.

Four different failures can look like a timeout

Browser isolation does not isolate your database

Playwright gives tests separate browser contexts by default, including separate cookies and local storage. It cannot stop two contexts from editing the same row in the same backend. If Alice's test and Bob's test both rename project-123, they can still race even with perfect browser isolation.

Give each test its own backend data. Where your test environment supports it, derive a record key from testInfo.testId (or provision the record in a fixture), create the record before the interaction, and clean up only what that test owns.

// Pattern only: app-specific seedProject() is not supplied here.
test('renames its own project', async ({ page }, testInfo) => {
  const projectId = `project-${testInfo.testId}`
  // await seedProject({ id: projectId, owner: testInfo.project.name })
  // Exercise the app with this project's unique, seeded ID.
})

That ID is an isolation key, not necessarily a valid production database ID; if your schema requires UUIDs, generate one and associate it with testInfo.testId in the test setup instead. Don't fix data collisions merely by reducing workers to one: sequential execution can hide the race without removing shared-state dependence. The parallelism guide explains backend isolation separately from browser contexts.

CI might be talking to a different server

A local test may reuse an existing development server. CI typically starts from a clean checkout and has to install dependencies, start the app, and wait for it to respond. A process listening on a port does not necessarily mean its database migrations, login endpoint, or seeded data are ready.

Playwright supports a webServer configuration. For an app that really serves a /login page on port 3217:

// Adapt the command and readiness URL to your actual application.
webServer: {
  command: 'npm run dev -- --port 3217',
  url: 'http://127.0.0.1:3217/login',
  reuseExistingServer: !process.env.CI,
  timeout: 120_000,
},
use: {
  baseURL: 'http://127.0.0.1:3217',
},

The snippet belongs inside defineConfig({ ... }) and assumes your package has a compatible dev script. If your tests target a production build, replace that command with your built application's start script and build beforehand. The url check measures HTTP readiness, not database or test-data readiness; if those matter, expose a test-only readiness condition or perform deterministic setup. See the web server option.

Reproduce the failure without changing five settings at once

Use a small investigation sequence. Each command below assumes @playwright/test, the required browser, and a matching project config are already installed in the application under test, not in this portfolio:

First, capture evidence from one test; next, repeat it to investigate intermittent timing; finally, run multiple related test files together to investigate shared backend state. Replace tests/projects with the actual directory containing competing tests.

npx playwright test tests/save-status.spec.ts --workers=1 --trace on
npx playwright test tests/save-status.spec.ts --workers=1 --repeat-each=20
npx playwright test tests/projects --workers=4

Do not interpret 20 passing repeats as proof the bug is gone. A flaky failure may need a particular server load, data collision, time zone, browser engine, or parallel combination. Playwright normally runs different test files in parallel but runs tests within one file sequentially. Increasing workers for a single test does not create a second test that can collide with it. To test a suspected shared-record race, run the relevant tests together (or explicitly enable parallel mode within the file). Conversely, a failure with one worker makes a cross-worker collision less likely, but not an application-level race. See Playwright's execution model.

When local and CI behavior still differ, run the same Git commit, Playwright package and browser versions, operating-system image where practical, environment variables (without exposing secrets), base URL, and seeded data. In CI, use npm ci and install the intended browsers and OS dependencies with the Playwright CLI. The official CI guide recommends starting with one worker on constrained CI runners and considering sharding later.

A fixed page.waitForTimeout(2000) is a poor substitute for an application state condition; Playwright explicitly discourages it in production tests. Waiting for networkidle is not a universal readiness solution either—Playwright discourages it for testing in favor of assertions about actual readiness. See the Page API.

What to change after you've found the cause

Keep the repair as close as possible to the failed assumption:

• The test read too soon: use an auto-retrying assertion for a user-visible state or wait for a specific response when that response is the contract.
• The application never reached the state: debug the request, state update, and backend error; don't change the test to accept Saving....
• Data collided: seed unique rows/accounts and make cleanup independent of test order.
• The wrong app or unready service answered: correct webServer, base URL, build/start sequence, and test database setup.
• CI resources are insufficient: measure where time is consumed, reduce parallel workers if appropriate, and change a particular timeout only with evidence that the operation legitimately needs longer.
• Retry rescued the test: keep the original failure trace; don't count the retry as proof that the bug was repaired.

The lesson is not “always increase the timeout” or even “always use toHaveText.” It is to make the readiness condition, test data, and environment explicit, then verify the next failure against that contract.