Testing

vidtrace uses layered tests because the product is both a Go codebase and a CLI that orchestrates external media tools.

Test Layers

Layer	Command	Purpose
Unit tests	task test	Fast Go behavior checks
Lint	task lint	Static checks through golangci-lint
Build check	task build	Compile the CLI
Synthetic smoke	task smoke	Run extraction against generated media outside the repo
Docs build	task site	Build the VitePress site for Vercel
Glyphrun E2E	task e2e	Verify specs, real PTY CLI behavior, and artifacts

Standard Checks

Run this before handing off code:

task check

Run full local verification, including E2E:

task all

Glyphrun

End-to-end specs live under e2e/: one flow per file in e2e/flows/, shared bundle builders in e2e/fixtures/, and reusable step snippets in e2e/actions/. See e2e/README.md.

task e2e

Current flows cover:

• cli_doctor.yml: version and doctor output.
• cli_docs.yml: built-in docs for humans and agents.
• cli_compare.yml: ticket comparison and bundle validation JSON.
• cli_evidence_search.yml: evidence indexing and search JSON.
• cli_investigate.yml: investigation handoff JSON and Markdown output.
• cli_investigate_connect.yml and cli_investigate_codemap.yml: optional tool flags.
• cli_clip.yml and cli_stash.yml: clip and stash command surfaces.
• docs_site.yml: VitePress documentation build.
• cli_studio.yml: interactive Studio navigation, metadata toggle, and action status text in a real PTY.
• extract_json.yml: JSON extraction output and generated artifacts.

Artifacts are written to .glyphrun/, which is ignored by Git.

Evidence search is covered by Go tests in internal/evidence and CLI JSON tests in internal/cli. These tests use temporary bundles and temporary .veclite databases outside the repo.

CI

GitHub Actions runs formatting, module drift, unit tests, build, lint, and goreleaser check on every pull request. A mediasmoke job on main and workflow_dispatch installs ffmpeg, tesseract, and whisper and runs a synthetic extract plus validate. Pull requests skip that job to stay fast.

Glyphrun E2E does not run in GitHub Actions. Run this locally before release work:

task all

Real Video Testing

A local sample video may exist at:

~/Downloads/bug.mp4

Do not commit this video or generated bundles. Run real-video checks outside the repo:

bin/vidtrace extract ~/Downloads/bug.mp4 --out /tmp/vidtrace-bug-smoke --name bug --json

Extractor Parity

The legacy scripts/extract.sh was removed after Go pipeline parity was verified on a synthetic clip and on a real video (same frames, OCR, and transcript outputs, plus Go-only metadata.json and timeline.json). The parity decision is recorded in CHANGELOG.md. The ongoing synthetic parity check is task smoke.

What To Assert

For agent-facing behavior, prefer JSON and generated files over human text:

• exit code
• valid JSON on stdout for --json
• artifact bundle exists
• metadata.json exists and has schema_version
• timeline.json exists and has entries
• compare --json emits a stable result shape
• index --json and search --json emit stable evidence-search JSON
• transcript files exist
• OCR files match the frame count