Annotations Automation Practice
Practice Playwright test annotations — test.skip(), test.slow(), test.fixme(), test.fail(), test.step(), and test.each() — plus TestNG equivalents across realistic interactive scenarios.
Interactive Scenarios
Increments by 2 instead of 1. Use test.fixme('BUG-101').
Bug: +1 click adds 2. Decrement is correct.
Succeeds ~50% of runs. Use test.fixme() or retries: 2.
Always returns an error. test.fail() confirms the bug — test "passes" if it fails.
| Product | Qty | Unit Price |
|---|---|---|
| Wireless Headphones | $149.99 | |
| USB-C Hub | $39.99 |
Total: $229.97
Test Data — Valid Credentials:
| Role | Password | Expects | |
|---|---|---|---|
| Admin | [email protected] | admin123 | Full dashboard |
| Editor | [email protected] | editor123 | Edit panel only |
| Viewer | [email protected] | viewer123 | Read-only view |
| Invalid | [email protected] | wrong | Error message |
| ID | Scenario | Type | Priority | |
|---|---|---|---|---|
| ANN_001 | Beta panel hidden when feature flag is OFF Expected: beta-panel is not visible when toggle aria-checked is false ✅ positivehigh | positive | high | |
| ANN_002 | Beta panel visible after toggling feature flag ON Expected: beta-panel appears with text 'Beta Feature Visible' and enabled action button ✅ positivehigh | positive | high | |
| ANN_003 | Environment badge updates when switching environments Expected: env-badge changes from 'Staging' to 'Production' after button click ✅ positivemedium | positive | medium | |
| ANN_004 | Loading indicator aria-busy attribute transitions during slow op Expected: aria-busy changes from false → true → false as report generates ✅ positivehigh | positive | high | |
| ANN_005 | Report table has exactly 4 rows after generation Expected: report-tbody contains 4 <tr> elements with correct data ✅ positivehigh | positive | high | |
| ANN_006 | Buggy counter increments by 2 instead of 1 (BUG-101) Expected: Counter shows 6 after 3 clicks — annotation test.fixme() marks it broken ❌ negativehigh | negative | high | |
| ANN_007 | Counter decrement works correctly (no bug) Expected: Counter decreases by 1 per click with correct floor at 0 ✅ positivemedium | positive | medium | |
| ANN_008 | test.fail() confirms BUG-101 still exists (regression check) Expected: test.fail() annotation inverts result — test 'passes' when assertion fails ⚠️ edgemedium | edge | medium | |
| ANN_009 | Checkout step 1 — cart review and quantity update Expected: Cart table shows both products; total updates on quantity change ✅ positivehigh | positive | high | |
| ANN_010 | Checkout step 2 — fill shipping form and proceed Expected: Step 1 tab shows done class; step 2 panel is visible after form fill ✅ positivehigh | positive | high | |
| ANN_011 | Checkout step 3 — place order and confirm Expected: order-confirmation is visible with an ORD- prefixed order ID ✅ positivehigh | positive | high | |
| ANN_012 | Data-driven login — Admin user gets full dashboard Expected: dash-admin is visible; dash-editor and dash-viewer are hidden ✅ positivehigh | positive | high | |
| ANN_013 | Data-driven login — Invalid credentials show error, no dashboard Expected: login-result shows 'Invalid credentials'; role-dashboard is hidden ❌ negativehigh | negative | high | |
| ANN_014 | Logout restores initial login state for next test iteration Expected: Form fields cleared, dashboard hidden, login button visible again ✅ positivemedium | positive | medium |
Playwright annotations let you control test execution, timeout behaviour, and reporting without restructuring your test suite. Understanding when to use skip vs fixme vs fail is a common interview topic — and the difference has real impact on how bugs show up in CI reports.
Use test.skip() when a test is intentionally not applicable in the current context — e.g., a feature flag is OFF or you are running against an environment that doesn't support the feature. Always provide a reason so the CI report is readable. For known bugs, prefer test.fixme() over test.skip() so the broken test stays visible in reports.
// Playwright — test.skip() based on feature flag
test('Beta panel is visible when flag is ON', async ({ page }) => {
await page.goto('/practice/annotations');
const toggle = page.locator('[data-testid="toggle-beta-ui"]');
const isBetaOn = await toggle.getAttribute('aria-checked') === 'true';
// Skip if Beta UI is disabled in this environment
test.skip(!isBetaOn, 'Beta UI flag is OFF — skipping beta-only test');
// Only reaches here when flag is ON
await expect(page.locator('[data-testid="beta-panel"]')).toBeVisible();
await expect(page.locator('[data-testid="beta-panel"]')).toContainText('Beta Feature Visible');
await expect(page.locator('[data-testid="btn-beta-action"]')).toBeEnabled();
});test.slow() triples the default Playwright timeout for a single test without hardcoding a number. Call it at the very top of the test before any action. It is designed for tests involving report generation, file exports, or heavy API calls that legitimately take longer than the default 30 seconds.
test.fixme() marks a test as a known broken item — the test is skipped but appears clearly in the HTML report with a 'fixme' badge, so the team can track unresolved bugs. test.fail() is different: the test runs, and it 'passes' only if the assertion fails. Use test.fail() to confirm a bug still exists — if someone fixes it, test.fail() will start failing, signalling you to remove the annotation.
// Playwright — test.slow() + test.fixme() + test.fail()
// ── test.slow(): triple timeout for slow async ops ──────────────
test('Report generation completes', async ({ page }) => {
test.slow(); // 30s → 90s — call FIRST before any action
await page.goto('/practice/annotations');
await page.click('[data-testid="btn-gen-report"]');
// Assert loading state
await expect(page.locator('[data-testid="loading-indicator"]'))
.toHaveAttribute('aria-busy', 'true');
// Web-first: waits until visible automatically (up to 90s)
await expect(page.locator('[data-testid="report-result"]')).toBeVisible();
await expect(page.locator('#report-tbody tr')).toHaveCount(4);
await expect(page.locator('[data-testid="slow-status"]')).toContainText('done');
});
// ── test.fixme(): skip + flag in report ─────────────────────────
test('Counter increments by 1 @fixme', async ({ page }) => {
test.fixme(true, 'BUG-101: counter adds 2 per click instead of 1');
// Body won't run — but BUG-101 is visible in the HTML report
const counter = page.locator('[data-testid="buggy-counter"]');
for (let i = 0; i < 3; i++) await page.click('[data-testid="btn-buggy-inc"]');
await expect(counter).toHaveText('3'); // would fail (actual: "6")
});
// ── test.fail(): confirms bug still exists ───────────────────────
test('BUG-101 regression check', async ({ page }) => {
test.fail(); // EXPECT this to fail — inverts the result
const counter = page.locator('[data-testid="buggy-counter"]');
for (let i = 0; i < 3; i++) await page.click('[data-testid="btn-buggy-inc"]');
await expect(counter).toHaveText('3'); // fails (actual: "6") → test.fail() makes this PASS
// If bug is fixed, this assertion passes → test.fail() makes the WHOLE test fail → remove the annotation
});test.step() wraps a group of actions under a named label. The label appears in the Playwright HTML report and trace viewer, making failures instantly locatable: 'Failed in step: Fill shipping form' beats 'Error at line 47'. Each step can also be independently retried with the step retry plugin.
test.each() (or a for-loop over an array) runs the same test body for every row in a data set. In Playwright, the idiomatic pattern is a for-of loop: for (const user of users) { test(user.role, async ({ page }) => { … }) }. TestNG uses @DataProvider returning Object[][] with @Test(dataProvider='name').
Quick reference across all three frameworks.
| Action | Selenium | Playwright JS | Playwright PY | Cypress |
|---|---|---|---|---|
| Skip unconditionally | @Test(enabled=false) | test.skip() | test.skip() | it.skip() |
| Skip conditionally | throw new SkipException() | test.skip(condition, 'reason') | test.skip(condition, 'reason') | this.skip() / cy.skip() |
| Known bug (tracked in report) | @Test(enabled=false) + comment | test.fixme('BUG-ID') | test.fixme('BUG-ID') | it.skip('BUG-ID') |
| Expected to fail (confirms bug) | @Test(expectedExceptions=...) | test.fail() | test.fail() | No direct equivalent |
| Triple timeout (slow op) | @Test(timeOut=90000) | test.slow() | test.slow() | { timeout: 90000 } |
| Named step in report | @Step (Allure) | await test.step('name', fn) | async with test.step('name'): | cy.log() / step plugin |
| Data-driven (parameterized) | @DataProvider + @Test(dataProvider=) | for (const d of data) test(d.name, fn) | @pytest.mark.parametrize | data.forEach(d => it(d.name, fn)) |
| Run by tag | @Test(groups={'smoke'}) | --grep @smoke | -k smoke | --env grepTags=@smoke |