Write integration tests and post-launch validation checklist
epic-scenario-based-follow-up-prompts-infrastructure-task-008 — Write integration tests covering: successful run with mock scheduler service, kill-switch disabled path, per-activity error isolation (one bad record does not abort run), and idempotency (second invocation in same window produces zero new prompts). Additionally, produce a post-launch validation checklist covering cron trigger firing confirmation, log output verification, kill-switch toggle test, and monitoring metric definitions (error rate, skip rate, run duration).
Acceptance Criteria
Technical Requirements
Execution Context
Tier 6 - 158 tasks
Can start after Tier 5 completes
Implementation Notes
Use Supabase's local dev stack for integration tests — run `supabase start` as a prerequisite and point the test client at `http://localhost:54321`. For mocking the Scheduler Service, start a simple Deno HTTP server on a random port at test setup and pass its URL as `SCENARIO_SCHEDULER_URL` environment variable. In each test, configure the mock server to return success, error, or timeout responses as needed. Use `supabase.from('activities').insert([...])` to seed data and `supabase.from('activities').delete().neq('id', '')` in `afterEach` to clean up.
For the idempotency test, avoid manipulating the system clock — instead insert a `prompt_history` row manually to simulate a prior run within the window, then assert the function skips it. The post-launch checklist should be written as a GitHub-flavoured markdown checklist (`- [ ] item`) so it can be used directly as a PR checklist during production deployment.
Testing Requirements
This task IS the testing task. The integration tests themselves are the primary deliverable. Additionally: (1) ensure tests can be run via a single command `deno test --allow-net --allow-env tests/integration/scenario-evaluation/` from the repository root; (2) add a CI step in the existing pipeline (GitHub Actions or equivalent) that runs these tests on every PR touching the Edge Function source; (3) confirm tests are included in the project's test coverage report.
Supabase Edge Functions on Deno can have cold-start latency of 500ms–2s. If the evaluation window contains many activities (e.g., post-holiday catch-up), the function may approach the 60-second invocation timeout before completing all evaluations.
Mitigation & Contingency
Mitigation: Implement pagination in the activity fetch query with a configurable page size; process pages sequentially and commit history records per page so partial runs are recoverable on the next invocation.
Contingency: If timeout remains an issue at scale, split the evaluation into per-chapter invocations triggered by a fan-out pattern using Supabase Realtime or a lightweight queue.
Supabase cron triggers (pg_cron or Edge Function schedules) may miss invocations during platform maintenance windows, causing evaluation gaps that delay time-sensitive prompts beyond their intended delivery window.
Mitigation & Contingency
Mitigation: Configure the look-back window to be 2× the cron interval (e.g., 2-hour look-back for hourly cron) so a single missed invocation does not result in missed prompts; log each run's look-back range for auditability.
Contingency: If missed invocations are detected via monitoring alerts, implement a manual re-trigger endpoint accessible to admins that runs the evaluation for a specified time range.