Skip to main content

Portfolio App: Testing

Purpose​

Define what “testing” means for the Portfolio App at each maturity phase, and specify the CI quality gates required to merge and release.

The emphasis is on enterprise credibility:

  • deterministic validation
  • enforceable gates in CI
  • clear pass/fail criteria

Scope​

In scope​

  • required local validation commands
  • required CI checks (quality + build)
  • phased test strategy (unit then e2e)
  • acceptance criteria for “release-ready”

Out of scope​

  • detailed test case definitions for specific pages (add as the app matures)

Prereqs / Inputs​

  • Portfolio App repo exists and is runnable locally
  • ESLint/Prettier/TypeScript tooling is installed
  • CI workflow exists and is enforced via branch protections

Procedure / Content​

Local validation workflow (required)​

Three-Stage Validation: Local → CI → Staging​

The Portfolio App enforces quality at three distinct stages:

  1. Local validation (developer machine, before PR)

    • Run pnpm verify to catch issues early
    • Prevents CI failures and speeds up review

  2. CI validation (GitHub Actions, on PR and main)

    • Automatic: lint, format, typecheck, unit tests, E2E tests, build
    • Blocks merge if checks fail
    • Required before staging validation

  3. Staging validation (production-like environment, after merge to main)

    • Manual smoke tests on https://staging-bns-portfolio.vercel.app
    • Optional automated Playwright tests against staging domain
    • Required before production is considered "live"

For a streamlined workflow with detailed reporting and full test coverage:

pnpm install
pnpm verify

The verify command runs a comprehensive 12-step validation workflow:

  1. Environment check: Validates Node version, pnpm availability, .env.local existence, and required environment variables
  2. Auto-format: Runs format:write to fix formatting issues automatically
  3. Format validation: Confirms formatting correctness with format:check
  4. Linting: Executes ESLint with zero-warning enforcement
  5. Type checking: Validates TypeScript types across the codebase
  6. Dependency audit: Runs pnpm audit --audit-level=high for high/critical CVEs
  7. Secret scan (lightweight): Pattern-based scan to catch obvious secrets (local-only; CI uses TruffleHog)
  8. Registry validation: Ensures project registry schema compliance and data integrity
  9. Build: Produces production bundle to catch build-time errors
  10. Performance verification: Validates bundle size and cache headers against docs/performance-baseline.yml
  11. Unit tests: Runs the Vitest suite across lib helpers, API handlers, components, pages, data wrappers, proxy middleware, structured data, observability, and security helpers
  12. E2E tests: Runs the Playwright suite across Chromium + Firefox for smoke, route coverage, metadata endpoints, evidence links, and security API coverage

Benefits:

  • Single command runs all pre-commit quality checks and tests
  • Auto-formats code before validation (reduces false failures)
  • Provides color-coded output for quick status assessment
  • Includes detailed troubleshooting guidance for each failure type
  • Protects performance budgets by enforcing bundle size and cache headers locally
  • Mirrors CI workflow for local/remote consistency
  • Generates summary report with next steps

When to use:

  • Before every commit (catches issues early)
  • Before opening a PR (ensures CI will pass)
  • After pulling changes from main (validates clean state)
  • Before final push to production branch

Option 2: Quick verification (fast iteration)​

For rapid feedback during active development without tests:

pnpm verify:quick

Runs steps 1-9 above, skips performance checks and all tests (steps 10-12).

When to use:

  • During active development with frequent small changes
  • When debugging specific issues in a feature branch
  • For rapid iteration cycles
  • Always run full pnpm verify before final commit/push

Option 3: Individual commands (granular control)​

For targeted validation or when you need to run specific checks:

pnpm install
pnpm lint          # ESLint validation
pnpm format:check  # Prettier validation (or format:write to fix)
pnpm typecheck     # TypeScript type checking
pnpm audit         # Dependency audit (high severity)
pnpm build         # Production build
pnpm test:unit     # Unit tests (Vitest)
pnpm test:coverage # Unit tests with full src coverage
pnpm test:e2e       # E2E tests (Playwright)

When to use individual commands:

  • Debugging a specific type of failure
  • Running checks during active development (e.g., typecheck while coding)
  • Understanding what each check does
  • Integrating with editor/IDE workflows
  • Running only unit tests (without E2E): pnpm test:unit
  • Auditing coverage for the full source tree: pnpm test:coverage
  • Debugging E2E tests: pnpm test:e2e:ui or pnpm test:e2e:debug

Local preview server​

To preview the site during development:

pnpm dev

This starts the Next.js development server at http://localhost:3000.

CI quality gates (required)​

Status: Implemented in Phase 1.

Gate 1: Quality​

  • pnpm lint
  • pnpm format:check
  • pnpm typecheck
  • pnpm audit --audit-level=high

Gate 2: Build​

  • pnpm build

These checks must run on:

  • PRs targeting main
  • pushes to main

Linting​

Configuration approach:

  • ESLint 9+ with flat config (eslint.config.mjs)
  • Presets:
    • eslint-config-next/core-web-vitals (Next.js recommended rules)
    • eslint-config-next/typescript (TypeScript integration)
  • Custom global ignores: .next/, out/, dist/, coverage/, .vercel/, next-env.d.ts

Command:

pnpm lint  # fails on warnings (--max-warnings=0)

Rationale:

  • Flat config is the modern ESLint standard (ESLint 9+)
  • Next.js presets provide sensible defaults for App Router + TypeScript
  • Zero warnings enforced to maintain code quality

Formatting​

Configuration (prettier.config.mjs):

{
  semi: true,
  singleQuote: false,
  trailingComma: "all",
  printWidth: 100,
  tabWidth: 2,
  plugins: ["prettier-plugin-tailwindcss"]
}
  • Prettier uses an ESM config (prettier.config.mjs) to satisfy plugin ESM/TLA requirements
  • Tailwind plugin: prettier-plugin-tailwindcss automatically sorts Tailwind utility classes for consistency
  • pnpm format:check is a required gate in CI and must stay stable to avoid check-name drift

Commands:

pnpm format:check  # CI gate
pnpm format:write  # local fix

Merge gates (GitHub ruleset)​

  • Required checks: ci / quality, ci / build (must exist and run to be selectable as required).
  • Pipeline prerequisites: ci / test, ci / link-validation (must pass for build promotion path to succeed).
  • PR-only security gate: secrets-scan runs on pull requests.
  • Checks run on PRs and on pushes to main to gate production promotion and keep ruleset enforcement valid.
  • Check naming stability is mandatory; changing names would break required-check enforcement and Vercel promotion alignment.

Dependabot CI hardening​

This control is active in the current CI baseline.

Problem: Dependabot PRs fail quality checks due to pnpm-lock.yaml formatting violations. Dependabot regenerates lockfiles but does not run Prettier, causing pnpm format:check to fail.

Solution implemented:

  1. Lockfile exclusions in .prettierignore:

    • Added pnpm-lock.yaml, package-lock.json, yarn.lock
    • Rationale: Machine-generated files don't benefit from formatting; prevents CI failures

  2. Auto-format step for Dependabot PRs in CI:

    • Runs in ci / quality job before lint/format/typecheck steps
    • Conditional execution: if: ${{ github.actor == 'dependabot[bot]' }}
    • Actions:
      • Runs pnpm format:write || true
      • Detects changes with git status --porcelain
      • Auto-commits formatting fixes as chore: auto-format (CI)
      • Pushes to PR branch
    • Permissions: Workflow has contents: write to enable auto-commit

  3. Workflow permissions:

    • Changed from contents: read to contents: write
    • Required for CI to push auto-format commits to Dependabot PR branches

Impact:

  • Future Dependabot PRs will auto-fix formatting issues without manual intervention
  • Quality gate failures due to lockfile formatting eliminated
  • Maintains zero-tolerance formatting enforcement for non-generated code

Evidence:

Governance note: Auto-format only runs for Dependabot; human PRs still require manual formatting to maintain developer discipline.

Testing strategy evolution​

Note: This section distinguishes baseline controls from expanded controls for ongoing maintenance.

Baseline: Gates + manual smoke checks​

  • quality + build gates
  • manual review on preview deployments:
    • navigation
    • page rendering
    • key links to /docs

Expanded coverage: Automated E2E tests with Playwright​

Playwright coverage includes evidence-link DOM/href assertions and route-level verification as part of the current baseline.

Framework: Playwright (multi-browser E2E testing)

Coverage:

  • Multi-browser checks across 2 browsers (Chromium, Firefox)
  • Core routes: /, /cv, /projects, /contact
  • Dynamic routes: /projects/[slug] (discovered from /projects)
  • 404 handling for unknown routes and invalid slugs
  • Health + metadata endpoints: /api/health, /robots.txt, /sitemap.xml
  • Evidence link rendering and accessibility on /projects/portfolio-app
  • Security endpoints: /api/csrf, /api/echo (CSRF + rate limit coverage)
  • Security headers: CSP nonce present on HTML responses
  • Responsive checks for evidence content (mobile/tablet/desktop)

Configuration:

  • Test directory: tests/e2e/
  • Config file: playwright.config.ts
  • Browsers: Chromium, Firefox (WebKit excluded for stability)
  • Retries: 2 in CI, 0 locally
  • Workers: 1 in CI (sequential for stability), unlimited locally
  • Base URL: http://localhost:3000 (local/CI dev server)

Running E2E Tests:

pnpm test:e2e          # Run all E2E tests headlessly
pnpm test:e2e:ui       # Open Playwright UI mode (local dev)
pnpm test:e2e:debug    # Run tests in debug mode with inspector
pnpm exec playwright show-report    # View HTML test report

CI Integration:

  • Tests run in ci / test job before build
  • Playwright browsers installed via npx playwright install --with-deps
  • Workflow: external-link-monitor
  • Command: pnpm links:check:external
  • Trigger: daily schedule + manual dispatch
  • Purpose: best-effort live HTTP reachability checks for external evidence URLs
  • Governance: non-blocking for PR merge gates to avoid third-party uptime flakiness
  • Dev server started with pnpm dev & and readiness check via wait-on http://localhost:3000
  • Tests execute with pnpm test:e2e
  • HTML test reports generated (.gitignored)
  • Build fails if any E2E tests fail

Test Scripts:

  • pnpm test:e2e — Run all E2E tests
  • pnpm test:e2e:ui — Interactive UI mode for debugging
  • pnpm test:e2e:debug — Debug mode with step-through inspector

Evidence:

Next.js 15 Compatibility Fix:

  • Fixed dynamic route params (now async in Next.js 15)
  • Changed params: { slug: string } to params: Promise<{ slug: string }>
  • Added await for params destructuring in [slug]/page.tsx

Expanded coverage: Unit tests​

Unit testing is part of the current quality baseline and enforced in CI.

Framework: Vitest (fast, ESM-native unit testing)

Purpose: Validate registry schema, slug rules, API handlers, UI components/pages, and link construction helpers to ensure data integrity and reviewer-facing behavior

Coverage:

  • Broad unit coverage across src/lib/, src/app/ (pages + API handlers), src/components/, src/data/, and src/proxy.ts
  • All tests passing locally and in CI
  • Code coverage: ≥95% for all source modules tracked by Vitest coverage (current scope: src/**/*.{ts,tsx})

Local execution:

pnpm test:unit      # Run unit tests once (CI-like execution)
pnpm test           # Run tests in watch mode (for development)
pnpm test:coverage  # Run tests and generate coverage report
pnpm test:ui        # Visual UI mode for debugging failing tests

Registry Validation Tests​

File: src/lib/__tests__/registry.test.ts (17 tests)

Purpose: Ensure project registry entries are valid according to Zod schema

What's tested:

  • Valid project entries pass schema validation
  • Invalid entries (missing fields, malformed slugs) are rejected
  • Required fields (title, summary, tags, tech stack) are validated
  • Date format enforcement (YYYY-MM for startDate/endDate)
  • Slug uniqueness is enforced (duplicate slugs rejected)
  • Tech stack categories are validated (language, framework, library, tool, platform)
  • Evidence links structure is validated

Key assertions:

it('should accept valid project entries', () => {
  const validProject = {
    slug: 'portfolio-app',
    title: 'Portfolio App',
    summary: 'A comprehensive portfolio application.',
    tags: ['nextjs', 'typescript'],
    // ... other required fields
  };
  expect(ProjectSchema.safeParse(validProject).success).toBe(true);
});

it('should reject projects with invalid slug format', () => {
  const invalid = { slug: 'Invalid Slug!' };
  expect(ProjectSchema.safeParse(invalid).success).toBe(false);
});

Files:

  • src/lib/__tests__/config.test.ts (18 tests)
  • src/lib/__tests__/linkConstruction.test.ts (16 tests)

Purpose: Ensure URL helpers (docsUrl(), githubUrl(), docsGithubUrl(), mailtoUrl()) work correctly

What's tested:

  1. docsUrl(): Builds documentation URLs with NEXT_PUBLIC_DOCS_BASE_URL

    • With environment variable configured
    • Fallback to /docs when env var missing
    • Leading slash normalization
    • Nested path handling

  2. githubUrl(): Builds GitHub URLs with NEXT_PUBLIC_GITHUB_URL

    • With environment variable configured
    • Placeholder return when env var missing
    • Path normalization

  3. docsGithubUrl(): Builds documentation GitHub URLs with NEXT_PUBLIC_DOCS_GITHUB_URL

    • URL construction from environment variable
    • Fallback behavior

  4. mailtoUrl(): Builds mailto links with optional subject parameters

    • Email address handling
    • Subject parameter encoding
    • Special character escaping

Key assertions:

it('should build URL with default base path', () => {
  const result = docsUrl('/portfolio/roadmap');
  expect(result).toBe('/docs/portfolio/roadmap');
});

it('should handle email with subject', () => {
  const result = mailtoUrl('[email protected]', 'Hello World');
  expect(result).toBe('mailto:[email protected]?subject=Hello%20World');
});

Slug Validation Tests​

File: src/lib/__tests__/slugHelpers.test.ts (19 tests)

Purpose: Enforce slug format rules and validate edge cases

What's tested:

  • Valid slug format: lowercase, hyphens, alphanumeric only
  • Regex pattern: ^[a-z0-9]+(?:-[a-z0-9]+)*$
  • Rejection of: uppercase, spaces, special characters, unicode, emoji
  • Edge cases: empty strings, single characters, very long slugs
  • Multiple consecutive hyphens rejected
  • Hyphens at start/end rejected

Key assertions:

it('should accept valid lowercase slugs', () => {
  expect(isValidSlug('portfolio-app')).toBe(true);
  expect(isValidSlug('my-project-2024')).toBe(true);
});

it('should reject uppercase slugs', () => {
  expect(isValidSlug('Portfolio-App')).toBe(false);
});

Running Unit Tests​

# All unit tests in watch mode (development)
pnpm test

# Unit tests once (for CI verification)
pnpm test:unit

# With coverage report (outputs to coverage/index.html)
pnpm test:coverage

# Visual UI dashboard (for debugging)
pnpm test:ui

# Debug mode with inspector
pnpm test:debug

Available test suites:

  • src/lib/__tests__/registry.test.ts — Registry validation (17 tests)
  • src/lib/__tests__/slugHelpers.test.ts — Slug format and deduplication (19 tests)
  • src/lib/__tests__/config.test.ts — Link construction helpers (34 tests)
  • src/app/api/__tests__/ — API route handlers (csrf, echo, health)
  • src/app/__tests__/ — App Router pages (layout, home, contact, projects, not found, error)
  • src/components/__tests__/ — UI components (navigation, theme, badges, evidence blocks)
  • src/data/__tests__/ — Data wrappers and CV content
  • src/__tests__/proxy.test.ts — Middleware CSP and nonce behavior

CI Integration:

  • Runs in ci / test job as prerequisite to build
  • Command: pnpm test:unit
  • Coverage reports uploaded as artifacts
  • Build fails if any tests fail
  • Must pass ≥95% coverage thresholds (lines, functions, branches, statements)

Coverage Report:

After running pnpm test:coverage:

  • Open coverage/index.html in a browser
  • Review per-file coverage metrics
  • Identify uncovered branches and functions

Coverage thresholds (enforced in CI):

  • Lines: ≥95%
  • Functions: ≥95%
  • Branches: ≥95%
  • Statements: ≥95%

Phase 4: Extended E2E coverage (Stage 3.3 enhanced)​

  • Expand Playwright coverage:
    • form submissions (contact page)
    • navigation flows (multi-page journeys)
    • accessibility checks
    • visual regression tests (if needed)

Definition of Done for changes​

A PR is acceptable when:

  • CI gates pass:
    • ci / quality (lint, format, typecheck)
    • ci / build (build + smoke tests)
  • preview deployment renders as expected
  • E2E tests pass across supported browsers
  • no broken evidence links are introduced
  • if behavior changes materially:
    • dossier updated
    • runbooks updated (if ops changes)
    • ADR added/updated (if architectural)
    • smoke tests updated if routes/navigation changes

Validation / Expected outcomes​

  • failures are caught before merge
  • build remains deterministic
  • tests expand over time without slowing delivery unreasonably

Failure modes / Troubleshooting​

  • format drift causes repeated failures:
    • run format:write locally and recommit
  • lint rules too strict early on:
    • tune deliberately; document policy changes if significant
  • typecheck fails due to config mismatch:
    • align tsconfig; keep checks scoped to repo sources

References​