Integration test: full export artifact generation pipeline
epic-bufdir-reporting-export-processing-services-task-016 — Write integration tests that exercise the complete artifact generation pipeline end-to-end using a test Supabase instance with seed data. Test that a given set of seeded activities produces the correct Excel file, CSV file, PDF report, and attachment ZIP bundle. Verify all generated artifacts are stored to the export file storage and accessible via signed URLs.
Acceptance Criteria
Technical Requirements
Execution Context
Tier 4 - 323 tasks
Can start after Tier 3 completes
Implementation Notes
Integration tests require a dedicated CI Supabase project — provision one via Supabase Management API or use the CLI (supabase start for local Docker). Store project URL and service_role_key in GitHub Actions secrets mapped to environment variables read in test setup. The TestSupabaseHelper should use the service_role_key (bypassing RLS) for seeding and teardown, while the pipeline under test should use a test user's JWT to exercise real RLS policies. Use unique organization IDs per test run (UUID.v4()) to ensure parallel CI runs don't interfere.
Teardown should use ON DELETE CASCADE on seed records or explicit deletion in reverse dependency order. For the signed URL verification step, generate signed URLs with 2-hour expiry and use Dart's http package to assert HTTP 200 — this confirms the storage write actually succeeded. Document the test Supabase project setup in test/integration/README.md including required table schemas, RLS policies, and storage bucket configuration.
Testing Requirements
Integration tests using flutter_test with a real test Supabase instance. Use a TestSupabaseHelper class to manage seed data lifecycle: insertSeedActivities(), insertSeedAttachments(), tearDownTestData(). Each test should call setUp/tearDown via flutter_test hooks. Seed data constants defined in test/fixtures/bufdir_seed_data.dart.
Test scenarios: (1) canonical pipeline — 15 activities, 3 attachments, verify all 4 artifacts; (2) zero-activity organization — verify pipeline completes and produces valid empty-state artifacts; (3) large dataset smoke test — 200 activities, verify completion without timeout or error. Use http package to verify signed URLs return HTTP 200. Parse artifact contents using archive (ZIP), csv parser, and PDF byte inspection utilities from prior unit test tasks.
NHF contacts can belong to up to five local chapters simultaneously. If the deduplication logic in the activity query service incorrectly attributes cross-chapter activities, organisations will either under-report or over-report to Bufdir, which could trigger grant clawback or compliance investigations.
Mitigation & Contingency
Mitigation: Implement deduplication using the existing multi-chapter membership service as the source of truth for chapter affiliation. Write test fixtures covering all known multi-chapter edge cases and validate outputs against manually prepared reference exports from NHF.
Contingency: If deduplication cannot be made deterministic for complex hierarchies before release, gate the export behind an org-level feature flag and require NHF to validate a preview export against their manual Excel before enabling in production.
Server-side Dart libraries for Excel generation are less mature than equivalents in Node.js or Python. The chosen library may lack support for Bufdir-required formatting features (merged cells, data validation, specific date formats), requiring significant workaround effort or a library switch mid-implementation.
Mitigation & Contingency
Mitigation: Evaluate the top two Dart xlsx libraries (excel, spreadsheet_decoder) against a Bufdir template sample file before committing. Identify all required formatting features and verify library support in a spike.
Contingency: If no Dart library meets requirements, implement the Excel generation as a Supabase Edge Function in TypeScript using the well-supported ExcelJS library, exposing it to the Dart backend via an internal RPC call.
The attachment bundler must retrieve documents from Supabase Storage that were uploaded by the document attachments feature. If storage paths, RLS policies, or signed URL expiry have not been standardised across features, the bundler may fail to retrieve attachments at export time.
Mitigation & Contingency
Mitigation: Audit the document attachments feature's storage schema and RLS policies before implementing the bundler. Agree on a stable internal service-account access pattern for cross-feature storage reads.
Contingency: If cross-feature storage access cannot be made reliable, implement the bundler to include only attachments that can be retrieved successfully and produce a manifest listing any attachments that could not be bundled, rather than failing the entire export.