Audit Log of All Duplicate Detection Events
Every time the duplicate detection system triggers a warning, the event should be logged with: the two record IDs involved, the coordinator who saw the warning, the resolution action taken (proceed, cancel, merge, mark distinct), a timestamp, and the chapter context. This log should be accessible from the coordinator's settings or admin area. During Bufdir export preparation, coordinators should be able to generate a summary of duplicate-reviewed records to include as supporting documentation.
User Story
Audience Summaries
Bufdir grant compliance requires organizations to demonstrate not merely clean data but a documented, repeatable data integrity process. A persistent audit log of every duplicate detection event — capturing who saw the warning, what they decided, and when — transforms duplicate resolution from an invisible UI interaction into a verifiable governance record. For NHF specifically, this directly supports annual grant reporting cycles by providing exportable documentation that duplicate records were deliberately reviewed, not silently ignored or accidentally missed. Failure to produce this audit trail during a Bufdir review could trigger funding disputes or require manual reconstruction of resolution history from memory.
Delivering this story positions all partner organizations as compliant, professionally managed grantees, reducing regulatory risk and strengthening credibility with funders over time.
This is a medium-priority story with upstream dependencies on both duplicate detection stories (story-duplicate-activity-detection-coordinator-1 and -2), so it cannot begin implementation until those are complete and the duplicate detection event model is stable. Scope includes: audit log write path (triggered on each duplicate resolution action), a read interface in coordinator settings or admin area, and an optional CSV export integration into the Bufdir export configuration flow. Cross-functional needs are moderate: backend for log schema and query API, frontend for the audit log list view and export toggle, and QA for testing all five resolution action types (PROCEED, CANCEL, MERGE, MARK_DISTINCT, and deferred queue resolutions). Acceptance criteria include record preservation after source deletion — requiring soft-delete awareness in the log query layer.
Plan for one sprint of delivery with an additional QA cycle covering the export path. Coordinate with the Bufdir export feature team to align on the supplementary CSV format.
A new `duplicate_audit_log` table is required with columns: id, coordinator_id (FK to users), record_id_a, record_id_b, resolution_action (enum: PROCEED_WITH_DUPLICATE, CANCEL, MERGE, MARK_DISTINCT), resolved_at (timestamp), chapter_id, and a nullable note for deleted-record annotations. Write path: hook into the duplicate warning dismissal handler and the deduplication queue resolution handler to insert log entries synchronously within the same database transaction as the resolution action. Read path: paginated API endpoint scoped to coordinator's chapter, returning human-readable summaries with masked peer mentor names when sensitivity flags apply. The Bufdir export integration needs a boolean toggle in the export config payload that appends a joined query of relevant audit log entries as a supplementary CSV stream.
Deleted record references must be handled via a soft-delete check — if the referenced activity no longer exists, annotate the log entry with a `deleted_record_note` field rather than orphaning the FK. Index on `resolved_at` and `coordinator_id` for query performance.
Acceptance Criteria
- Given a coordinator dismisses a duplicate warning by selecting 'Submit Anyway', When the activity is saved, Then a log entry is written with the coordinator ID, both activity IDs, resolution action 'PROCEED_WITH_DUPLICATE', and timestamp
- Given a coordinator resolves a pair in the deduplication queue, When the resolution is saved, Then a log entry is written with resolution action, resolver ID, and timestamp
- Given the coordinator opens the duplicate audit log, When they view the list, Then each entry shows a human-readable summary: date, peer mentor name (masked if sensitive), resolution action, and resolver name
- Given the coordinator is preparing a Bufdir export, When they access the export configuration, Then they can optionally append the duplicate audit log as a supplementary CSV
- Given a log entry exists for a record that was later deleted, When the coordinator views the audit log, Then the entry is preserved with a note that the referenced record was subsequently removed
Business Value
Bufdir audits require organizations to demonstrate data integrity practices, not just clean data. A persistent, exportable audit trail proves that duplicate resolution is a deliberate, documented process rather than an absence of duplicates by chance. This directly supports NHF's compliance posture during annual grant reporting cycles and reduces the risk of funding disputes arising from data quality questions.
Components
- Duplicate Warning Event Logger infrastructure
- Duplicate Reviewed Flag Middleware infrastructure
- Duplicate Detection Service service
- Duplicate Check Repository data
- Export Data Query Builder data
- CSV Generation Service service