Test Runners

FlakeMonster auto-detects your test runner and parses output to identify individual test results. You can also specify the runner explicitly.

Auto-Detection

FlakeMonster examines the test command string you provide (via --cmd or the testCommand config field) and matches it against known runner patterns. The first match wins:

Auto-detection works for the vast majority of projects. If your command string does not contain one of the expected keywords (for example, you wrap your test runner in a shell script), use the --runner flag to tell FlakeMonster which parser to use:

$ flake-monster test --runner jest --cmd "./scripts/run-tests.sh"

Jest

Jest is the most widely used JavaScript test runner, commonly found in React and Node.js projects.

How it works

• Detection: The command string contains jest or react-scripts test
• Output format: JSON (FlakeMonster requires Jest's structured JSON output)
• Reporter flag: FlakeMonster automatically injects --json into the command if not already present
• Parsing: FlakeMonster reads Jest's JSON output to extract the status of every individual test case, including the test name, suite file, pass/fail status, and failure messages

Example

# Auto-detected (command contains "jest")
$ flake-monster test --cmd "npx jest"

# Explicit runner selection
$ flake-monster test --runner jest --cmd "npx jest"

# With Create React App
$ flake-monster test --cmd "npx react-scripts test --watchAll=false"

Jest configuration tips

Tip: If your Jest config uses --forceExit, keep it. FlakeMonster does not interfere with Jest lifecycle flags.

If you use a custom Jest config path, include it in the command:

$ flake-monster test --cmd "npx jest --config jest.integration.config.js"

Node.js Native Test Runner

Node.js 18+ ships with a built-in test runner (node:test). FlakeMonster parses its TAP output to track individual test results.

How it works

• Detection: The command string contains node --test
• Output format: TAP (Test Anything Protocol)
• Reporter flag: FlakeMonster automatically injects --test-reporter tap if no reporter is specified

Example

# Auto-detected (command contains "node --test")
$ flake-monster test --cmd "node --test test/**/*.test.js"

# Explicit runner selection
$ flake-monster test --runner node-test --cmd "node --test test/**/*.test.js"

# With a specific file pattern
$ flake-monster test --cmd "node --test 'src/**/*.spec.js'"

Important — Node 20 JSON reporter crash: Node 20 does not have a built-in json test reporter. Using --test-reporter json causes an exit code 7 crash. FlakeMonster handles this by always using the tap reporter, which is stable across Node 18, 20, and 22.

Reporter behavior

The Node.js test runner has a few output quirks that FlakeMonster accounts for:

FlakeMonster captures both stdout and stderr, but parses TAP output from stdout. If you have explicitly set --test-reporter spec, FlakeMonster will override it to tap so that test results can be parsed correctly.

Playwright

Playwright is a popular browser automation framework used for end-to-end testing. FlakeMonster parses Playwright's JSON reporter output.

How it works

• Detection: The command string contains playwright
• Output format: JSON
• Reporter flag: FlakeMonster automatically appends --reporter=json if no reporter is specified in the command
• Parsing: FlakeMonster reads Playwright's JSON report to extract test titles, file paths, pass/fail status, and error messages per test case

Example

# Auto-detected (command contains "playwright")
$ flake-monster test --cmd "npx playwright test"

# Explicit runner selection
$ flake-monster test --runner playwright --cmd "npx playwright test"

# Run a specific test file
$ flake-monster test --cmd "npx playwright test tests/checkout.spec.ts"

Playwright configuration tips

FlakeMonster injects delays into your source files, not the Playwright test files. The injected await delays execute in the browser context when Playwright drives tests against the modified code.

Tip: For Playwright projects, consider using mode: "hardcore" and higher maxDelayMs values (100-200ms). Browser-based tests are more sensitive to timing differences than unit tests.

If your project uses a custom Playwright config, include it in the command:

$ flake-monster test --cmd "npx playwright test --config=playwright.ci.config.ts"

TAP

TAP (Test Anything Protocol) is FlakeMonster's default fallback parser. It works with any runner that produces TAP-formatted output.

How it works

• Detection: Used when no other runner is detected (default fallback)
• Output format: TAP protocol (versions 12, 13, and 14)
• Parsing: FlakeMonster reads ok / not ok lines to determine per-test pass/fail status

Compatible runners

The TAP parser works with any test runner that outputs standard TAP. Common runners with TAP support:

Example

# Falls back to TAP parser automatically
$ flake-monster test --cmd "npx mocha --reporter tap"

# Explicit runner selection
$ flake-monster test --runner tap --cmd "npm test"

# With tape (natively outputs TAP)
$ flake-monster test --cmd "npx tape 'test/**/*.js'"

# With AVA
$ flake-monster test --cmd "npx ava --tap"

Tip: If your test runner does not produce TAP output by default, check if it has a TAP reporter option. Most popular runners do. The TAP parser is the most universal fallback.

Browser-Based Testing

FlakeMonster's source-to-source injection approach works naturally with browser test runners. The injected await statements are plain JavaScript — they survive bundling, transpilation, and execute correctly in the browser.

Why it works

FlakeMonster modifies your source files, not test files. When a browser test runner builds and serves your code, the injected delays are included in the bundle. This means:

• Delays execute in the real browser context, affecting actual render timing
• Race conditions between components, API calls, and DOM updates are exposed
• No special browser plugins or test runner modifications are needed

Playwright

Playwright drives a real browser against your application. FlakeMonster injects delays into source files before Playwright compiles and serves them:

# Inject delays, then run Playwright tests
$ flake-monster test --cmd "npx playwright test"

# With higher delays for browser-sensitive timing
$ flake-monster test --cmd "npx playwright test" --maxDelayMs 200 --mode "hardcore"

Cypress

Cypress runs tests in a browser and serves your source code through its dev server. FlakeMonster injects delays before Cypress picks up the files:

# Run Cypress in headless mode with TAP output
$ flake-monster test --runner tap --cmd "npx cypress run"

# With a specific spec file
$ flake-monster test --runner tap --cmd "npx cypress run --spec 'cypress/e2e/checkout.cy.js'"

Vitest (browser mode)

Vitest's browser mode runs tests in a real browser. FlakeMonster injects delays into source files before Vitest bundles them:

# Vitest with browser mode
$ flake-monster test --runner tap --cmd "npx vitest run --reporter=tap"

# Vitest with JSON reporter
$ flake-monster test --runner tap --cmd "npx vitest run --reporter=tap-flat"

Browser testing configuration

Browser tests are typically more sensitive to timing than unit tests. Recommended config for browser-based test suites:

// .flakemonsterrc.json
{
  "include": ["src/**/*.js", "src/**/*.ts"],
  "exclude": [
    "**/node_modules/**",
    "**/dist/**",
    "**/build/**",
    "**/playwright-report/**",
    "**/cypress/fixtures/**",
    "**/test-results/**"
  ],
  "mode": "hardcore",
  "minDelayMs": 5,
  "maxDelayMs": 200,
  "skipTryCatch": true,
  "runs": 20
}

The higher maxDelayMs of 200ms exaggerates timing differences that browser rendering is sensitive to. Setting skipTryCatch to true avoids interfering with framework error boundaries and retry logic. Running 20 iterations increases the chance of catching intermittent failures.

Explicit Runner Selection

Use the --runner flag to override auto-detection. This is useful when:

• Your test command is wrapped in a shell script that hides the runner name
• You use an npm script alias that does not contain the runner keyword
• Auto-detection picks the wrong runner (e.g., a command containing "jest" that actually calls a different runner)

Available runner values

Examples

# Explicit Jest
$ flake-monster test --runner jest --cmd "npx jest"

# Explicit Playwright
$ flake-monster test --runner playwright --cmd "npx playwright test"

# Explicit node:test
$ flake-monster test --runner node-test --cmd "node --test"

# Explicit TAP (for any TAP-compatible runner)
$ flake-monster test --runner tap --cmd "npm test"

# Wrapped script — auto-detection cannot see "jest" in the command
$ flake-monster test --runner jest --cmd "./scripts/run-tests.sh"

# npm script alias — "npm test" does not reveal the underlying runner
$ flake-monster test --runner jest --cmd "npm test"