Implement ZIP archive packaging for attachment bundles
epic-bufdir-reporting-export-processing-services-task-009 — Build the ZIP archive creation phase of the BufdirAttachmentBundler that packages all collected activity-linked documents into a structured ZIP file. Implement directory structure within the ZIP (organized by activity date and type), handle naming conflicts, and stream the archive to Supabase Storage without loading all files into memory simultaneously.
Acceptance Criteria
Technical Requirements
Execution Context
Tier 2 - 518 tasks
Can start after Tier 1 completes
Implementation Notes
Use the `archive` pub.dev package (`Archive`, `ArchiveFile`, `ZipEncoder`) for ZIP creation. Do NOT load all files into memory first — stream each file by downloading it via its signed URL using `http.get` (or Dart's `HttpClient`) and add the resulting bytes to the archive one file at a time, then release the reference. For the directory structure, build the path string as `'${activity.date.year}-${activity.date.month.toString().padLeft(2,'0')}/${activityTypeSlug}/${resolvedFilename}'`. Conflict resolution: maintain a `Map
For streaming upload to Supabase Storage, use `supabase.storage.from(bucket).uploadBinary(path, bytes, fileOptions: FileOptions(upsert: true))` — for very large archives consider chunking using `upload` with a `Stream
Testing Requirements
Unit tests (flutter_test) must cover: (1) directory structure — given a manifest with activities on two different dates and two types, assert the ZIP contains correctly nested paths; (2) conflict resolution — two attachments with identical filenames in the same directory, assert second is renamed with `_2` suffix; (3) empty manifest — assert method returns null and no upload is attempted; (4) skip on download failure — mock one file download throwing an exception, assert ZIP is created with remaining files and warning is logged; (5) idempotency — calling the method twice with the same report period ID uploads with `upsert: true` and does not throw. Integration test should create a real ZIP from 3 small test files, upload to Supabase test bucket, and verify download via signed URL produces a valid archive. Test coverage for the ZIP packaging method must be at least 85%.
NHF contacts can belong to up to five local chapters simultaneously. If the deduplication logic in the activity query service incorrectly attributes cross-chapter activities, organisations will either under-report or over-report to Bufdir, which could trigger grant clawback or compliance investigations.
Mitigation & Contingency
Mitigation: Implement deduplication using the existing multi-chapter membership service as the source of truth for chapter affiliation. Write test fixtures covering all known multi-chapter edge cases and validate outputs against manually prepared reference exports from NHF.
Contingency: If deduplication cannot be made deterministic for complex hierarchies before release, gate the export behind an org-level feature flag and require NHF to validate a preview export against their manual Excel before enabling in production.
Server-side Dart libraries for Excel generation are less mature than equivalents in Node.js or Python. The chosen library may lack support for Bufdir-required formatting features (merged cells, data validation, specific date formats), requiring significant workaround effort or a library switch mid-implementation.
Mitigation & Contingency
Mitigation: Evaluate the top two Dart xlsx libraries (excel, spreadsheet_decoder) against a Bufdir template sample file before committing. Identify all required formatting features and verify library support in a spike.
Contingency: If no Dart library meets requirements, implement the Excel generation as a Supabase Edge Function in TypeScript using the well-supported ExcelJS library, exposing it to the Dart backend via an internal RPC call.
The attachment bundler must retrieve documents from Supabase Storage that were uploaded by the document attachments feature. If storage paths, RLS policies, or signed URL expiry have not been standardised across features, the bundler may fail to retrieve attachments at export time.
Mitigation & Contingency
Mitigation: Audit the document attachments feature's storage schema and RLS policies before implementing the bundler. Agree on a stable internal service-account access pattern for cross-feature storage reads.
Contingency: If cross-feature storage access cannot be made reliable, implement the bundler to include only attachments that can be retrieved successfully and produce a manifest listing any attachments that could not be bundled, rather than failing the entire export.