All changes deployed to MaxShipping during March 2026
March 31, 2026Incident: An invoice for client Maria Santos (container wGbF1MteMcCaRxSeGzjW) was not auto-sent to Square. Root cause: two packages (889577200647 and 889577200658) were scanned 4 seconds apart during unloading. The Firestore transaction in updateInvoiceOnPackageStatusChange collided — one update succeeded, the other failed after 3 retries, leaving stale cached packageStatus in the invoice and isAllUnloaded: false.
| Fix | What Changed |
|---|---|
| Part 1 Jitter on retries | updateInvoiceOnPackageStatusChange: Increased MAX_RETRIES from 3 to 5. Added random jitter (0–2s) to retry delay to prevent thundering herd when multiple packages update the same invoice simultaneously. |
| Part 2 Daily reconciliation cron | New reconcileStuckInvoices function runs daily at 5:00 AM Manila time. Checks last 3 containers for invoices where isAllUnloaded == false and invoiceSent == false, reads actual package statuses, and fixes mismatches. 256MB, 300s timeout. |
| Part 3 Split invoice bug fix | splitInvoice: Now sets isAllUnloaded: true on the original invoice after splitting out non-unloaded packages, so the original can proceed to auto-send. |
A new in-app notification system for admins to see cloud function alerts, errors, and reconciliation results without relying on email.
Backend:
| Component | Details |
|---|---|
| writeNotification() helper | Shared function in index.js that any cloud function can call. Writes to Notifications Firestore collection with type, group, subject, HTML message, and optional IDs. |
| Notifications collection | New Firestore collection. Fields: type (error/alert/reconciliation/info), group (invoices/packages/clients/containers/system), subject, message (HTML), isRead, timestamp, optional invoiceID/clientID/containerID. |
Frontend (MaxTracks Web):
Badge types: FAILED (type: error) ALERT (type: alert) INFO (all others)
Files: Notifications.dart (new), main.dart (bell icon + panel integration), index.js (writeNotification helper)
Reconciliation notifications show a concise message:
| Field | Content |
|---|---|
| Subject | Auto-Send Invoice Failed — Retried (Client Name) |
| Message | Which invoice failed, which packages had mismatched statuses, and a note to manually send from Invoices page if needed. |
| Type | reconciliation → shows as INFO badge |
| Group | invoices |
Incident: During container unloading on March 27, 80 "Failed Auto-Send Invoice" emails were sent. 22 of these were false alarms caused by a race condition in the Pub/Sub invoice processing pipeline.
Root cause: Firebase Cloud Functions have at-least-once execution. The enqueueInvoiceForProcessing trigger fired twice for the same isAllUnloaded change, publishing two Pub/Sub messages with different operation IDs. The first succeeded at Square, the second was rejected with error 400 ("invoice number already used"). The catch block then overwrote invoiceSent: true with false and sent false failure emails.
| Fix | Where | What It Prevents |
|---|---|---|
| Layer 1 PubSubQueue dedup | enqueueInvoiceForProcessing | Atomic Firestore transaction using invoiceID as key. Only one instance can enqueue. |
| Layer 2 invoiceSent check | processInvoiceInternal | Re-reads invoice before calling Square. If already sent, skips. |
| Layer 3 Catch block check | processInvoiceFromQueue | Before sending failure email or overwriting status, checks if invoice already succeeded. |
Issue: The Make.com paid plan expired around March 8–9. The addClientViaWebhook function stored the raw webhook response text ("Accepted") as squareCustomerId instead of a real Square customer ID. This affected 36 client records.
Resolution: Created 36 new Square customer accounts, updated all 36 Firebase Client records with real Square IDs. The updateSquareCustomerIdInInvoices trigger auto-propagated IDs to invoices. 6 stuck invoices were manually re-sent.
The sendInvoiceFailureEmail function now includes:
| Before | After |
|---|---|
| Client name, date, invoice ID | Client name, container name, date, invoice ID |
| "unknown/API error" | Specific error type, HTTP status code, and error message |
Automation workflows previously running on Make.com (SaaS) have been migrated to n8n, a self-hosted workflow automation platform.
| Before | After |
|---|---|
| Make.com (third-party SaaS) | n8n (self-hosted) |
| Monthly subscription cost | No recurring cost |
| Limited by Make.com plan | Full control over execution |
The "Do not ask for a password for the next 10 minutes" checkbox has been enabled on Container Unloading and Client Pickup screens, completing coverage across all package management screens.
| Screen | File | Status |
|---|---|---|
| Receiving | Receiving.dart | Already had it |
| Check In | CheckIn.dart | Already had it |
| Container Unloading | ContainerUnloading.dart | Enabled |
| Client Pickup | ClientPickUp.dart | Added |
| Unloaded Not Checked In | UnloadedNotCheckedIn.dart | Already had it |
| Unprocessed | Unprocessed.dart | Already had it |
ContainerUnloading.dart — Uncommented the existing checkbox UI in deleteOrMarkMissingPackage() and showEditForm(). Timer check and cloud function call were already wired up.
ClientPickUp.dart — Added from scratch: timer check against Temp/timer, checkbox UI, startNoPasswordPeriod cloud function call, loading spinner, and cloud_functions import in both deletePackage() and showEditForm().