# OAuth Token Refresh Implementation

## Overview

This document describes the automatic OAuth token refresh functionality implemented to prevent sync jobs from silently failing due to expired authentication tokens.

## Problem Statement

Sync jobs were failing because OAuth tokens expired and the automatic token refresh was not working properly. The issues were:

1. **Silent Failures**: Sync jobs completed with status "completed" but files weren't synced
2. **No Token Validation**: No token validation or refresh before sync operations
3. **Errors Not Surfaced**: Errors were caught but not surfaced to users
4. **No User Notification**: No mechanism to notify users of authentication failures

## Solution Architecture

### 1. Unified Token Refresh Utility

**File**: `src/lib/auth/unified-token-refresh.ts`

This module provides automatic OAuth token refresh for all cloud storage services (Google Drive, OneDrive, Dropbox) and works in both session-based (user-initiated) and non-session (cron job) contexts.

**Key Functions**:

- `areTokensExpired(tokens)`: Check if tokens are expired or expiring soon (within 5 minutes).
- `getTokensWithAutoRefresh(options)`: Get tokens with automatic refresh if expired.
- `getTokensWithAutoRefresh({ forceRefresh: true, ... })`: Refresh even when the access token has not reached the expiry buffer, used by background job launch preflight.
- `getTokensWithAutoRefreshOrThrow(options)`: Convenience wrapper that throws on failure.
- `batchRefreshTokens(requests)`: Batch refresh tokens for multiple services/accounts.

**Token Refresh Flow**:

```
1. Get current tokens from database or session
2. Check if tokens are expired (within 5-minute buffer), unless forceRefresh is set
3. If expired or forceRefresh is set:
   a. Check if refresh token is available
   b. Call service-specific refresh function
   c. If refresh succeeds, return new tokens
   d. If refresh fails, return error with needsReauth flag
4. If not expired, return current tokens
```

### 2. Background Job Token Preflight

**File**: `src/lib/jobs/background-token-preflight.ts`

Scheduled backups and syncs now proactively refresh every unique source and
destination account before provider validation, folder creation, or Fly rclone
launch. This keeps background jobs running with fresh access tokens and persists
rotated refresh tokens in Neon before rclone receives its temporary config.

The preflight:

- Deduplicates repeated accounts across multi-source jobs.
- Calls `getTokensWithAutoRefresh({ forceRefresh: true })` in cron context.
- Continues with the existing unexpired access token if a proactive refresh has a transient provider failure.
- Treats `invalid_grant`, `interaction_required`, and consent-required provider errors as reconnect-required.
- Uses non-reconnect wording for transient provider/configuration failures so the UI does not send users through OAuth unnecessarily.

### 3. Integration with Rclone Config Generation

**File**: `src/lib/rclone/core/flyio-client.ts`

Updated `createRcloneConfigForFlyIo()` to use the new unified token refresh:

**Before**:

```typescript
// Get tokens
const sourceTokens = await getTokensOrThrow({...});

// Check if expired
if (!isTokenValid(sourceTokens)) {
  // Attempt refresh using session-server
  const refreshedTokens = await refreshTokens(...);
  // ...
}
```

**After**:

```typescript
// Get tokens with automatic refresh
const sourceTokens = await getTokensWithAutoRefreshOrThrow({
  service: sourceService,
  accountId: finalSourceAccountId,
  userId,
});
```

This ensures that:

- Tokens are validated before every sync operation.
- Expired tokens are automatically refreshed.
- Refresh failures throw descriptive errors.
- Works for both source and destination services.

### 4. Enhanced Error Handling in Sync Jobs

**File**: `src/lib/sync/execute-sync-job.ts`

Updated error handling to detect and surface authentication errors:

```typescript
catch (error) {
  const errorMsg = error instanceof Error ? error.message : String(error);

  // Check if error is due to token expiration/authentication
  const isAuthError = errorMsg.toLowerCase().includes('token') ||
                     errorMsg.toLowerCase().includes('auth') ||
                     errorMsg.toLowerCase().includes('expired') ||
                     errorMsg.toLowerCase().includes('disconnect and reconnect');

  // Update item status with error
  await db.update(syncJobItems).set({
    status: "failed",
    error: errorMsg,
    updatedAt: now,
  });

  // If it's an auth error, also update the job's last_error to surface it
  if (isAuthError) {
    await db.update(syncJobs).set({
      lastError: errorMsg,
      updatedAt: now,
    });
  }
}
```

This ensures that:

- Authentication errors are detected.
- Errors are stored in both `syncJobItems.error` and `syncJobs.lastError`.
- Errors can be queried and surfaced to users.

### 5. User Notification System

#### API Endpoint

**File**: `src/app/api/sync/check-auth-errors/route.ts`

Created an API endpoint to check for sync jobs with authentication errors:

```typescript
GET /api/sync/check-auth-errors

Response:
{
  hasAuthErrors: boolean,
  errors: SyncAuthError[],
  affectedServices: ServiceType[]
}
```

The endpoint:

- Queries all sync jobs for the current user.
- Filters for jobs with authentication-related errors.
- Groups errors by service.
- Returns affected services and error details.

#### React Hook

**File**: `src/hooks/useSyncAuthErrorNotifications.ts`

Created a React hook to monitor and notify users of authentication errors:

```typescript
useSyncAuthErrorNotifications((checkIntervalMs = 60000));
```

The hook:

- Periodically checks for sync jobs with authentication errors (default: every 60 seconds).
- Shows toast notifications for each affected service.
- Only shows each notification once per session.
- Provides guidance to disconnect and reconnect accounts.

**Usage**:

```typescript
import { useSyncAuthErrorNotifications } from '~/hooks/useSyncAuthErrorNotifications';

function MyComponent() {
  // Check for auth errors every minute
  useSyncAuthErrorNotifications();

  return <div>...</div>;
}
```

#### Jobs Page Feedback

**Files**:

- `src/lib/jobs/job-error-feedback.ts`.
- `src/lib/mappers/job-mappers.ts`.
- `src/components/jobs/JobErrorNotice.tsx`.
- `src/app/user/jobs/page.tsx`.

The Jobs API now maps stored raw launch errors into human-safe feedback before
returning list/detail responses. Authentication failures such as raw
`[SYNC-SERVICE]` launch errors are displayed as provider-specific reconnect
prompts with a link to `/user/accounts/connect?service=...`. The raw database
error remains available server-side for operators, but the Jobs page does not
show internal service tags or provider item IDs to users.

If a raw rclone auth error does not name the provider, the Jobs API falls back to
the job payload and offers reconnect actions for the candidate providers instead
of showing a generic "Storage account" prompt.

## Service-Specific Token Refresh

The implementation uses existing service-specific refresh functions from `src/lib/service-token-refresh.ts`:

- **Google Drive**: Uses Google OAuth2 client to refresh tokens.
- **OneDrive**: Uses Microsoft OAuth endpoint to refresh tokens.
- **Dropbox**: Uses Dropbox OAuth endpoint to refresh tokens.

Each refresh function:

1. Takes the current refresh token
2. Calls the service's OAuth endpoint
3. Returns new access token and expiry date
4. Updates tokens in the database
5. Returns `{ success, tokens, error, needsReauth }` result

## Error Messages

User-friendly error messages are generated for token refresh failures:

```typescript
// When refresh token is invalid/expired
"Google Drive authentication has expired. Please disconnect and reconnect your Google Drive account to continue.";

// When refresh fails for other reasons
"Failed to refresh authentication for google:default. Please try again.";
```

## Testing

To test the implementation:

1. **Simulate Token Expiration**:
   - Manually set token expiry date to past in database.
   - Trigger a sync operation.
   - Verify automatic refresh occurs.

2. **Simulate Refresh Failure**:
   - Manually invalidate refresh token in database.
   - Trigger a sync operation.
   - Verify error is caught and surfaced.

3. **Verify User Notification**:
   - Create sync job with expired tokens.
   - Wait for cron job to execute.
   - Verify toast notification appears in UI.

## Benefits

1. **Automatic Recovery**: Tokens are automatically refreshed before operations and proactively force-refreshed before scheduled backup/sync launches
2. **No Silent Failures**: Authentication errors are detected and surfaced
3. **Human Intervention Only When Required**: Users are asked to reconnect only when the provider rejects the refresh token or requires interactive consent
4. **Multi-Service Support**: Works for all cloud storage services
5. **Context-Aware**: Works in both user-initiated and cron job contexts
6. **Consistent Error Handling**: Standardized error detection and reporting

## Future Enhancements

1. **Email Notifications**: Send email when manual reconnection is required
2. **Auto-Pause Jobs**: Automatically pause sync jobs with auth errors
3. **Retry Logic**: Implement exponential backoff for transient failures
4. **Token Refresh Metrics**: Track refresh success/failure rates
5. **Persist rclone-side Refreshes**: Capture provider token rotations that occur inside long-running Fly rclone processes and persist them back to Neon

## Related Files

- `src/lib/auth/unified-token-refresh.ts` - Core token refresh logic.
- `src/lib/auth/token-retrieval.ts` - Token retrieval utilities.
- `src/lib/service-token-refresh.ts` - Service-specific refresh implementations.
- `src/lib/jobs/background-token-preflight.ts` - Proactive refresh for scheduled backup/sync job accounts.
- `src/lib/rclone/core/flyio-client.ts` - Rclone config generation with token refresh.
- `src/lib/sync/execute-sync-job.ts` - Sync job execution with error handling.
- `src/app/api/sync/check-auth-errors/route.ts` - API endpoint for checking auth errors.
- `src/hooks/useSyncAuthErrorNotifications.ts` - React hook for user notifications.