Test email OTP login in Playwright without flaky sleeps

Email one-time codes are one of the most common reasons a login test turns flaky. The code arrives some seconds after the click, it lives in a mailbox you don't control, and the test has to find it without a human watching.

This post shows a pattern that holds up in CI: a dedicated inbox for your test account, a wait that only accepts mail that arrived after your click, and a timeout you set per test. It uses @programmableinbox/testing, the Playwright and Cypress kit, and mirrors the sample repo.

Why these tests get flaky#

Three things cause most of the pain:

• Fixed sleeps. waitForTimeout(5000) is too long when delivery is fast and too short when it isn't.
• Stale codes. The inbox still holds the code from the last run, and the test reads that one instead of the new one.
• Hand-rolled parsing. A regex per email template breaks the day someone edits the copy.

One inbox per test account#

Create one inbox for your test account and reuse it across runs. Don't create one per test: an inbox address is permanent, and it can't be reclaimed even after it is deleted.

• Register the test account in your app with that address, and put the inbox ID in PIBX_INBOX_ID.
• Keep the API key in PIBX_API_KEY. It needs the email_messages:read scope.
• Use a neutral local part such as qa-login-7f3k. Brand names are rejected at creation.
• Run login-by-email specs one at a time, or give each worker its own inbox. Two tests waiting on one inbox can take each other's code.

Wait for mail that arrives after your click#

otpAfter (Playwright) and cy.pibxOtp (Cypress) read the server's clock, run the action you pass, and then wait only for mail that arrived after that moment. A code left over from a previous run can never satisfy the current login. It polls every 5 seconds and waits up to 60 seconds by default. Set timeoutMs per call, and use from so a welcome email isn't mistaken for the login code.

import { test as base, expect } from '@playwright/test'
import { pibxFixtures, type PibxFixtures } from '@programmableinbox/testing/playwright'

const test = base.extend<PibxFixtures>(pibxFixtures)

test('login accepts the emailed code', async ({ page, pibx }) => {
await page.goto('https://staging.yourapp.com/login')
await page.fill('#email', process.env.TEST_USER_EMAIL!)

// otpAfter reads the server clock, then runs the click, then waits for new mail.
const { otp } = await pibx.otpAfter(
  () => page.click('button[type=submit]'),
  { from: 'yourapp.com', timeoutMs: 90_000 },
)

await page.fill('#code', otp)
await page.click('button[type=submit]')
await expect(page.locator('[data-testid=account-menu]')).toBeVisible()
})

// cypress.config.ts
import { defineConfig } from 'cypress'
import { pibxTasks } from '@programmableinbox/testing/cypress'

export default defineConfig({
e2e: {
  setupNodeEvents(on, config) {
    pibxTasks(on, config) // keeps PIBX_API_KEY out of the browser
    return config
  },
},
})

// cypress/support/e2e.ts
import '@programmableinbox/testing/cypress/support'

// cypress/e2e/login.cy.ts
it('login accepts the emailed code', () => {
cy.visit('https://staging.yourapp.com/login')
cy.get('#email').type(Cypress.env('TEST_USER_EMAIL'))

cy.pibxOtp({
  after: () => cy.get('button[type=submit]').click(),
  from: 'yourapp.com',
  timeoutMs: 90000,
}).then((code) => {
  cy.get('#code').type(code, { log: false })
  cy.get('button[type=submit]').click()
})

cy.get('[data-testid=account-menu]').should('be.visible')
})

If the deadline passes, the call throws a PibxTimeoutError instead of hanging, so a missing email fails the test with a clear error. A wrong or under-scoped key throws PibxAuthError right away and isn't retried.

Test the negative case too#

A passing login proves the happy path. The cases that actually protect you are the ones where the code should be rejected: an expired code, and a code from an earlier run.

1. Request the code as usual, then wait past your app's expiry window before submitting it.
2. Submit a code captured in a previous run and assert your app refuses it.

Expiry and reuse are your application's behavior, not the inbox's, which is exactly why they belong in your test suite.

Where to go next#

Clone the sample repo to run a full Playwright and Cypress login, or follow the step-by-step guide. The same pattern covers other email-driven flows. If your app uses sign-in links rather than codes, see magic-link testing. For signup confirmation, see email verification testing. The full walkthrough for this flow lives on the email OTP testing page.