Skip to main content

Image Upload and Processing Flow

This document explains the complete end-to-end flow of how test strip images are captured, uploaded, and processed in the Eli Health platform.

Overview​

The image upload process involves multiple services working together to capture, upload, analyze, and provide results for hormone test strips. The flow utilizes Firebase for temporary storage, Google Cloud Storage for permanent storage, PubSub for asynchronous processing, and the Health Analysis Engine (HAE) for image analysis.

Architecture Components​

Mobile App (React Native)​

  • Captures test strip images using the device camera
  • Detects and validates QR codes on test strips
  • Uploads images to Firebase Storage
  • Monitors processing status and displays results

Backend API (NestJS)​

  • Receives upload notifications from mobile app
  • Copies images from Firebase to GCS
  • Publishes messages to PubSub for processing
  • Manages reading records in PostgreSQL

PubSub Messaging​

  • Asynchronous message queue for background processing
  • Topics: reading-analysis-ca and reading-error-ca
  • Ensures reliable processing with retry mechanisms

Health Analysis Engine (HAE)​

  • External service for image analysis
  • Processes test strip images to extract hormone values
  • Returns quantitative results and recommendations

Detailed Flow​

Step 1: Image Capture (Mobile App)​

Key aspects of image capture:

  • QR Code Detection: Validates hormone type (Cortisol/Progesterone)
  • Quality Checks: Ensures proper lighting, distance, and angle
  • Auto-rotation: Ensures images are in portrait orientation
  • Firebase Upload: Temporary storage in tempImages/{userId}/{date}/ path

Step 2: Backend Processing Initiation​

Request payload structure:

{
  "epochDate": 1758759730,
  "storagePath": "tempImages/userId/2025-09-25/cortisol_1758759728419_sud0asz.jpg",
  "qrCodeString": "C8",
  "qrCodeCoordinates": {
    "x1": 1414, "y1": 2505,
    "x2": 1628, "y2": 2511,
    "x3": 1411, "y3": 2295,
    "x4": 1631, "y4": 2299
  },
  "wakeupTime": "2025-09-24T08:00:00-04:00",
  "timeZone": "2025-09-24T20:22:10-04:00",
  "hormone": "cortisol",
  "tags": "tag1,tag2"
}

Step 3: Asynchronous Analysis​

HAE Request includes:

  • Image URL from GCS
  • Historical readings for context
  • QR code coordinates
  • Wake-up time for cortisol circadian rhythm
  • Period history for progesterone

Step 4: Client Polling and Result Display​

Error Handling​

Error Recovery Mechanisms​

  • Dead Letter Queue: Failed messages after 5 retries
  • Retry Policy: Exponential backoff (1s to 600s)
  • Fallback Values: Hardcoded results for demo accounts
  • Cleanup: Automatic deletion of temporary Firebase files

Complete System Flow Diagram​

Performance Optimizations​

Mobile App​

  • Smart Capture Mode: QR detection with quality validation
  • Auto-rotation: Ensures correct image orientation
  • Retry Mechanism: Handles camera connection errors on newer iPhones
  • Device-specific Settings: Optimal camera configuration per device model

Backend​

  • Server-side Copy: Efficient GCS-to-GCS transfer
  • Parallel Processing: Async PubSub handling
  • Connection Pooling: Optimized database connections
  • Trace Context: End-to-end request tracking

Infrastructure​

  • CDN: CloudFlare for static assets
  • Regional Buckets: northamerica-northeast1 for low latency
  • Auto-scaling: Cloud Run scales based on load
  • Caching: 15-minute cache for frequently accessed data

Monitoring and Observability​

Trace IDs​

Every request is tracked with a unique trace ID that flows through all services:

  • x-cloud-trace-context: Google Cloud trace format
  • x-eli-trace-id: Custom trace ID for internal tracking
  • x-eli-span-id: Span ID for distributed tracing

Key Metrics​

  • Upload Success Rate: Target >95%
  • QR Detection Rate: Target >95%
  • Processing Time: P95 Less than 15 seconds
  • Error Rate: Target Less than 1%

Log Correlation​

All logs include:

  • traceId: For request correlation
  • readingId: For reading-specific tracking
  • userId: For user-specific debugging
  • type: For categorizing log events

Security Considerations​

  1. Authentication: Firebase Auth + JWT validation
  2. File Access: Signed URLs with 15-minute expiration
  3. Data Isolation: User-specific paths in storage
  4. Cleanup: Automatic deletion of temporary files
  5. Encryption: TLS for all API calls
  6. RBAC: Role-based access control for resources

Troubleshooting Guide​

Common Issues​

  1. "No active video connection" error

    • Occurs on iPhone 16/17 with iOS 18+
    • Solution: Retry with connection stabilization delay

  2. QR not detected

    • Check lighting conditions
    • Ensure QR code is flat and unobstructed
    • Verify QR format matches expected pattern

  3. Upload failures

    • Check network connectivity
    • Verify Firebase Storage permissions
    • Ensure storage quota not exceeded

  4. Processing timeouts

    • Check HAE service health
    • Verify PubSub subscription active
    • Review dead letter queue for failed messages

Future Improvements​

  1. WebSocket Updates: Real-time status updates instead of polling
  2. Edge Processing: On-device ML for faster QR detection
  3. Batch Processing: Multiple test strips in one session
  4. Offline Mode: Queue uploads when offline
  5. Progressive Upload: Stream image while capturing