Sentry Error Monitoring

This document explains how Eli Health's Sentry error monitoring is configured and managed. All configuration is done via code in the eli-devops repository, enabling version control and reproducible setups.

Overview

Item	Value
Organization	eli-health
Dashboard	sentry.io/eli-health
Configuration	`eli-devops/sentry/sentry_config.yaml`
Management Script	`eli-devops/sentry/sentry_manager.py`

Environments and Projects

Environment	Sentry Project	Slack Channel
Development	`eli-react-native-dev`	#alerts-dev
Staging	`eli-react-native-staging`	#alerts-staging
Production	`eli-react-native`	#alerts-production

Architecture

All Sentry configuration is managed at the server level, not in the mobile app code:

Benefits of Server-Side Configuration

Instant updates - No app deployment required
Version controlled - All changes tracked in Git
Consistent - Same filters across all environments
Auditable - Clear history of configuration changes

Member Management

Current Members

Members are defined in sentry_config.yaml:

members:
  - email: chip@eli.health
    role: owner
    name: Ciprian (Chip) Rarau
  - email: thomas@eli.health
    role: owner
    name: Thomas Cortina
  - email: daniel@eli.health
    role: member
    name: Daniel Shaban
  # ... more members

Available Roles

Role	Access Level
`owner`	Full organization access including billing
`admin`	Admin access (cannot change billing)
`manager`	Can manage teams and projects
`member`	Standard member access to view and resolve issues

Adding a New Member

Edit the configuration file:

cd eli-devops/sentry
nano sentry_config.yaml

Add the new member to the members section:

members:
  # ... existing members ...
  - email: newuser@eli.health
    role: member
    name: New User Name

Apply the changes:

# Dry run first to see what will happen
python3 sentry_manager.py members --dry-run

# Apply the changes
python3 sentry_manager.py members

The new member will receive an email invitation to join Sentry

How Member Sync Works

The sync is idempotent:

New members in config → Invited to Sentry
Role changed in config → Updated in Sentry
Member removed from config → NOT removed from Sentry (manual removal required)

Alert Configuration

Alert Types

Alert	Condition	Default Threshold
New Error Detection	First time an error is seen	N/A
High User Impact	Multiple users experience same error	10 users/hour
Error Rate Spike	Error count exceeds threshold	50 errors/hour
Repeated Error	Same error occurs multiple times	3 occurrences/hour

Environment-Specific Thresholds

Updating Alert Thresholds

Edit the alert_types section in sentry_config.yaml:

alert_types:
  error_spike:
    environments:
      production:
        threshold: 20  # Change from 15 to 20
        window: 60

Apply the changes:

# Preview changes
python3 sentry_manager.py setup --env production --dry-run

# Apply (this deletes ALL existing alerts and recreates them)
python3 sentry_manager.py setup --env production

Important: The setup command always performs a complete cleanup before creating alerts. This ensures no duplicate or orphaned alerts.

Error Filtering

How Filters Work

Inbound filters drop errors before they appear in Sentry. This reduces noise and keeps your dashboard focused on real issues.

Currently Active Filters

The following error patterns are automatically filtered out:

Category	Patterns
Network Errors	`no-internet`, `network-request-failed`, `Network request failed`, `fetch error`, `The network connection was lost`, `The request timed out`
Auth Errors	`auth/invalid-credential`, `The supplied auth credential is malformed or has expired`
Development Noise	`Metro is running`, `Connection to http://localhost:8081`, `Eli test`
Browser Issues	`ResizeObserver loop limit exceeded`, `Non-Error promise rejection captured`
Camera/Hardware	`The camera is too close`, `QR detected in preview but finder patterns not found`

Adding a New Filter

Edit sentry_config.yaml:

inbound_filters:
  ignored_errors:
    # Add your new filter
    - message: "your error text here"
      partial_match: true  # ALWAYS use true
      enabled: true
      description: "Brief description of what this filters"

Apply the filter:

# Preview
python3 sentry_manager.py filters --dry-run

# Apply to all environments
python3 sentry_manager.py filters

# Or apply to specific environment
python3 sentry_manager.py filters --env production

Filter Rules

Rule	Explanation
Always use `partial_match: true`	Adds wildcards (`*`) automatically so `"network error"` matches `"A network error occurred"`
Be specific	`"network-request-failed"` is better than `"failed"`
Case sensitive	Add multiple patterns if needed for different cases

Data Scrubbing

Sentry automatically scrubs sensitive data before storing errors:

What Gets Scrubbed

Data Type	Pattern	Replacement
Hormone Values	`12.5 ng/mL`	`[HORMONE_VALUE]`
Test Results	`45.2 nmol/L`	`[TEST_RESULT]`
Classifications	`above range`, `optimal`	`[CLASSIFICATION]`
IP Addresses	Automatic	Removed

Configuring Data Scrubbing

# Apply data scrubbing rules
python3 sentry_manager.py scrub --env production

Note: Advanced pattern-based scrubbing rules must also be configured in Sentry UI: Project Settings > Security & Privacy > Data Scrubbing

Commands Reference

Quick Reference

cd eli-devops/sentry

# Member management
python3 sentry_manager.py members              # Sync members
python3 sentry_manager.py members --dry-run    # Preview changes

# Alert management
python3 sentry_manager.py setup                # Set up all alerts
python3 sentry_manager.py setup --env staging  # Set up staging only
python3 sentry_manager.py setup --dry-run      # Preview changes

# Filter management
python3 sentry_manager.py filters              # Apply all filters
python3 sentry_manager.py filters --env prod   # Apply to production

# Data scrubbing
python3 sentry_manager.py scrub                # Configure scrubbing

# View configuration
python3 sentry_manager.py config               # Show all config
python3 sentry_manager.py config --env prod    # Show production config

Command Details

Command	Description
`members`	Sync organization members from config. Invites new, updates roles, never removes.
`setup`	Delete all existing alerts and recreate from config. Use `--no-clean` to skip deletion.
`filters`	Apply inbound error filters to drop unwanted errors.
`scrub`	Configure data scrubbing rules for PII/PHI protection.
`config`	Display current configuration without making changes.

Slack Integration

Alerts are sent to environment-specific Slack channels:

GCP Cloud Logging Integration

All Sentry errors are also synchronized to Google Cloud Logging for:

Long-term retention (beyond Sentry's 90-day limit)
Correlation with other system logs using trace IDs
BigQuery analysis for complex error pattern queries
Compliance - centralized audit trail

Troubleshooting

Filter Not Working?

Check the exact error message in Sentry - Copy the EXACT text

Verify wildcards are applied:

curl -H "Authorization: Bearer YOUR_TOKEN" \
  https://sentry.io/api/0/projects/eli-health/eli-react-native/ | \
  python3 -m json.tool | grep -A2 "filters:error_messages"

Wait 1-2 minutes - Filters take time to propagate
Note: Filters only affect NEW errors, existing ones remain

Permission Errors (403)?

The auth token needs these scopes:

alerts:read, alerts:write
project:read, project:write
org:read, org:write
member:read, member:write, member:admin

To create a new token:

Go to sentry.io/settings/account/api/auth-tokens/
Create token with required scopes
Update auth_token in sentry_config.yaml

Alert Not Firing?

Check alert is enabled for the environment in config
Verify threshold - May be set too high
Check Slack integration - Workspace ID must be correct
Review check frequency - Some alerts only check every 30-60 minutes

Developer Onboarding - Includes Sentry access setup for new developers
Internal Security Overview - Sentry data retention and backup policies
DevOps Overview - Overall infrastructure management

File Locations

File	Purpose
`eli-devops/sentry/sentry_config.yaml`	All configuration (members, alerts, filters)
`eli-devops/sentry/sentry_manager.py`	Python script to apply configuration
`eli-devops/sentry/CLAUDE.md`	Detailed operational guide
`eli-devops/sentry/README.md`	Quick start guide

Overview​

Environments and Projects​

Architecture​

Benefits of Server-Side Configuration​

Member Management​

Current Members​

Available Roles​

Adding a New Member​

How Member Sync Works​

Alert Configuration​

Alert Types​

Environment-Specific Thresholds​

Updating Alert Thresholds​

Error Filtering​

How Filters Work​

Currently Active Filters​

Adding a New Filter​

Filter Rules​

Data Scrubbing​

What Gets Scrubbed​

Configuring Data Scrubbing​

Commands Reference​

Quick Reference​

Command Details​

Slack Integration​

GCP Cloud Logging Integration​

Troubleshooting​

Filter Not Working?​

Permission Errors (403)?​

Alert Not Firing?​

Related Documentation​

File Locations​