CI failure triage

current.json · 2026-09-26T22:57:27.107Z

6 of 11 tests failed. 3 were decided by deterministic rules, 1 by Jev, and 2 are left for a person.

1
Regression

Likely a real product defect. Hold the merge and send it to the owner of the changed code.

2
Unknown

Not enough evidence to decide. A person should look at these first.

1
Test data

The product is probably fine; the test data or fixtures need fixing.

1
Environment

Infrastructure or a dependency failed. Check the service or runner, not the code.

1
Flaky

Nondeterministic test. Quarantine it and open a stability ticket. It was not re-run to hide it.

Regression 1

Completed filter shows only completed todos

rule

filters.spec.ts

Error: expect(locator).toHaveText(expected) failed · unexpected buy some cheese, book a doctors appointment
Why
  • stable before (8/8 passed), fails on every attempt now
  • diff touches owned application code: demo/app/filters.js
  • history: 8 run(s), first-try pass rate 100%, 0 flip(s)

Unknown 2

renaming a todo keeps its position

Jev, low confidence

edit.spec.ts

Error: locator.fill: Test timeout of 8000ms exceeded. · waiting for getByRole('textbox', { name: 'Edit' })
Why
  • no deterministic rule fired
  • only 0 prior run(s), below the 5 needed to trust history
  • diff touches test code or data: demo/tests/edit.spec.ts
  • history: 0 run(s), first-try pass rate n/a, 0 flip(s)
  • Jev leaned flaky but confidence 0.69 is below 0.75; routed to a human

Jev (jev-latest, recorded) confidence 0.69 · P(pass if re-run) 0.36 · 685 ms

flaky0.77
regression0.13
environment0.09
test-data0.01

marks everything complete through its accessible label

Jev, low confidence

toggle-all.spec.ts

Error: locator.check: Test timeout of 8000ms exceeded. · waiting for getByLabel('Mark all as complete')
Why
  • no deterministic rule fired
  • only 3 prior run(s), below the 5 needed to trust history
  • diff touches owned application code: demo/app/index.html
  • history: 3 run(s), first-try pass rate 100%, 0 flip(s)
  • Jev leaned regression but confidence 0.61 is below 0.75; routed to a human

Jev (jev-latest, recorded) confidence 0.61 · P(pass if re-run) 0.46 · 579 ms

regression0.71
flaky0.27
environment0.02
test-data0.00

Test data 1

seeded shopping list renders in order

Jev

seed-data.spec.ts

Error: expect(locator).toHaveText(expected) failed · missing Milk, Bread; unexpected Oat milk
Why
  • no deterministic rule fired
  • diff touches test code or data: demo/tests/fixtures/shopping-list.json
  • test-data marker(s): fixture/seed reference
  • history: 8 run(s), first-try pass rate 100%, 0 flip(s)
  • Jev: test-data with confidence 1.00 (threshold 0.75)

Jev (jev-latest, recorded) confidence 1.00 · P(pass if re-run) 0.39 · 570 ms

test-data1.00
regression0.00
flaky0.00
environment0.00

Environment 1

backup service answers its health check

rule

sync.spec.ts

Error: apiRequestContext.get: connect ECONNREFUSED 127.0.0.1:9
Why
  • infrastructure marker(s): ECONNREFUSED
  • no owned application code changed
  • history: 8 run(s), first-try pass rate 100%, 0 flip(s)

Flaky 1

counter reflects a completed item right away

rule

counter.spec.ts

Error: expect(received).toBe(expected) // Object.is equality · Expected: "2 items left", Received: "3 items left"
Why
  • failed then passed on retry within this run (2 attempts, same code)
  • history: 8 run(s), first-try pass rate 75%, 4 flip(s)