Create suspected duplicates table
epic-organizational-hierarchy-management-duplicate-detection-task-003 — Create the Supabase database table for storing flagged duplicate activity pairs, including fields for both activity IDs, similarity score, detection timestamp, review status (pending/confirmed_duplicate/false_positive), and reviewer ID. Add indexes for efficient querying by organization and review status.
Acceptance Criteria
Technical Requirements
Execution Context
Tier 1 - 540 tasks
Can start after Tier 0 completes
Implementation Notes
Enforce the (activity_id_a < activity_id_b) canonical ordering via a CHECK constraint: `CHECK (activity_id_a < activity_id_b)`. Callers (trigger, batch function, manual insert) are responsible for sorting the pair before inserting. For the reviewed_at trigger, use a BEFORE UPDATE trigger that sets `NEW.reviewed_at = now()` when `OLD.review_status = 'pending' AND NEW.review_status != 'pending'`. Expose this table in Flutter through a `SuspectedDuplicatesRepository` with methods: `getPendingPairs(orgId)`, `reviewPair(id, status)`, and a Supabase Realtime subscription stream for the admin review queue.
Use the `detection_method` field to distinguish between trigger-detected and batch-detected pairs — this helps admins understand urgency (trigger = just happened vs batch = historical).
Testing Requirements
pgTAP tests: (1) insert a valid pair and verify defaults; (2) insert (A,B) then (B,A) — second insert must fail UNIQUE constraint; (3) insert pair with similarity_score=1.5 must fail CHECK; (4) set review_status='confirmed_duplicate' and verify reviewed_at is populated by trigger; (5) delete activity_id_a and verify the pair row is cascade-deleted; (6) verify RLS blocks org B admin from reading org A's pairs. Flutter integration test: subscribe to Supabase Realtime on suspected_duplicates for the test org and verify a new insert triggers the stream.
Fingerprint-based similarity matching may produce high false-positive rates for common activity types (e.g., weekly group sessions with the same participants), causing alert fatigue among coordinators and undermining trust in the detection system.
Mitigation & Contingency
Mitigation: Start with conservative, high-confidence thresholds (exact peer mentor match + same date + same activity type) before adding looser fuzzy matching. Allow NHF administrators to tune thresholds based on observed false-positive rates. Log all detection decisions for retrospective threshold calibration.
Contingency: Introduce a snooze mechanism allowing coordinators to dismiss false positives for a configurable period. Track dismissal rates per activity type and automatically raise the similarity threshold for activity types with high dismissal rates.
A database trigger on the activities insert path adds synchronous overhead to every activity registration. For HLF peer mentors with 380 annual registrations or coordinators doing bulk proxy registration, this could create perceptible latency or lock contention.
Mitigation & Contingency
Mitigation: Implement the trigger as a DEFERRED constraint trigger (fires after the transaction commits) or replace it with a LISTEN/NOTIFY pattern that queues detection work asynchronously via an Edge Function, completely decoupling detection from the registration write path.
Contingency: Disable the synchronous trigger entirely and rely solely on the scheduled Edge Function for batch detection. Accept a detection delay of up to the scheduling interval (e.g., 15 minutes) in exchange for zero impact on registration latency.
The duplicate detection logic must be validated and approved by NHF before go-live, including agreement on threshold values and the review workflow. NHF stakeholder availability for sign-off may delay this epic's release independently of technical readiness.
Mitigation & Contingency
Mitigation: Gate the feature behind the NHF-specific feature flag so technical deployment can proceed independently of business approval. Involve an NHF administrator in threshold calibration sessions during QA, reducing the formal sign-off surface to policy and workflow rather than technical details.
Contingency: Release the detection system in 'silent mode' — flagging duplicates internally without surfacing notifications to coordinators — until NHF approves the workflow. Use the silent period to collect real data on false-positive rates and refine thresholds before activating notifications.