Testing

The Live Product Insights test suite: 104 tests across 10 files, how the integration harness boots a real server, load testing, and documented coverage gaps.

​

Running the suite

npm test      # full suite
npm run qa    # production build + full suite (what CI should run)

The suite uses Vitest with a 30-second timeout per test. As of the current tree: 104 tests across 10 files, all passing.

​

What each file covers

Test file	Tests	Coverage
collect-validate.test.js	12	Ingest contract: traffic-source classification, strict field validation, quantity clamps, hex-only cart tokens, handle vs numeric product identity
time-window.test.js	7	Window resolution (presets, custom ranges, timezone boundaries), equal-length comparison windows, integer-percent convention
orders-math.test.js	7	CSV escaping, null handling, money/percent/delta formatting conventions
csv-injection.test.js	4	OWASP formula-injection guard for CSV exports
ingest.integration.test.js	23	View + presence recording, 30-minute view dedupe (including the concurrent-duplicate regression), cart events with token mapping, batch cap, cross-shop isolation
metrics.integration.test.js	18	Known-answer metric fixtures: event rates, trending bands and minimum baselines, honest “not attributable”/“not supported” flags
orders.integration.test.js	12	Idempotent order upserts, test/cancelled order exclusion, per-unit refund proration, orders/updated-after-refund regression, attribution rules
http.security.test.js	14	Method guards, invalid/tampered signatures, not-installed shops, body-size cap, per-shop rate-limit bursts, webhook HMAC
restock-feed.test.js	5	Restock urgency banding, lead-time evidence, exclusions, cross-shop isolation
clear-data.integration.test.js	2	Merchant “clear analytics” wipes only the shop’s rows; idempotent

​

How the integration harness works

• Global setup builds and boots the real app. The production server build is started on a dedicated test port. API keys and the signing secret are pulled from the Shopify CLI (shopify app env show) and stay inside the test process — never logged, never written to disk. If a build or credentials are unavailable, the affected suites skip with an explicit marker rather than silently passing.
• Files run sequentially. Integration tests share one SQLite database file, so parallel file execution is disabled to avoid write contention.
• Fixtures are isolated by construction. Each file uses dedicated QA shop domains and reserved marker session UUIDs, and cleans up every row it created in reverse-dependency order afterward. This is what makes sharing a database file safe.
• Regression policy: every failing test is root-caused, and the regression test must be shown to fail before the fix lands. Concurrency bugs are re-run repeatedly to confirm the fix holds.

​

Load testing

npm run qa:load

Simulates 10 → 50 → 100 concurrent shoppers at realistic cadence (view on arrival, periodic heartbeats, counter polls every 30 seconds) against a production build, with a dedicated QA shop per stage and cleanup after. The most recent run: 0 errors (HTTP 429/5xx or network) at every stage, p95 latency ≈ 219 ms at 100 concurrent sessions, and active-viewer counts exactly matching concurrent sessions — the honesty-critical invariant.

​

Known coverage gaps

No browser E2E

Interactive OAuth prevents unattended browser automation against the embedded admin. Admin UI behavior is exercised through loader/action-level integration tests instead.

No real webhook or theme-embed delivery tests

Webhook handlers are tested via signed requests to the running server, not through Shopify’s actual delivery pipeline. The tracker script and Liquid blocks have no browser-level tests.

Scheduler is not wall-clock tested

Cron schedules and rollup windows are tested at the function level, not by advancing time through real cron fires.

Verified-in-code behavior that these gaps cover (consent gating, the counter’s hide-at-zero rule, dedupe windows) is additionally verified by signed end-to-end checks against a development store — see Troubleshooting for the practical symptoms when storefront-side setup is wrong.