# System Logs Documentation

**Last Updated:** September 11, 2026
**Status:** ✅ Production-Ready

---

## Table of Contents

1. [Overview](#overview)
2. [Operation Logging with Resolved Paths](#operation-logging-with-resolved-paths)
3. [System Logs Enhancements](#system-logs-enhancements)
4. [Implementation Details](#implementation-details)
5. [Usage Examples](#usage-examples)
6. [Related Documentation](#related-documentation)

---

## Overview

Open **Settings → Admin → System Logs** at `/admin/system-logs`.
Access requires an admin role and recent MFA verification. The viewer uses
the existing admin-only `/api/logs/audit` endpoint. Legacy `/dev/system-logs`
bookmarks redirect to the admin page; other `/dev` surfaces remain blocked
in production. These are application audit logs, not Caddy or Docker logs.

The system logs infrastructure provides comprehensive logging for all file operations across cloud storage services. Key features include:

- **Path Resolution**: Human-readable file paths alongside cloud storage IDs.
- **Consolidated Search**: Unified search across operation IDs, request IDs, and user identifiers.
- **Enhanced User Display**: Shows both email addresses and user IDs for better readability.
- **Multi-Service Support**: Works across Google Drive, OneDrive, Dropbox, Box, pCloud, and Jupiter.
- **Terminal Status Finalization**: Activity logs are finalized from both SSE and polling status paths, so fallback polling still records complete/error phases.
- **Batch Delete Consolidation**: Multi-item deletes are sent as one request, logged as one `delete-batch` timeline, and shown as a single visible row in activity/system logs after completion.
- **Singular Operation Labels**: `copy-batch` and `move-batch` entries render as `Copy` or `Move` when the request only contains one item, while true multi-item requests continue to show `Copy Batch` and `Move Batch`.
- **Failure Diagnostics**: Partial transfer entries now preserve failed-item previews and expose the same diagnostics in the activity details view when Fly/rclone reports them.
- **Google Drive to OneDrive Validation**: Copy, move, backup, and sync launches fail fast when Google Drive contains sibling file/folder name collisions that OneDrive cannot represent.
- **Folder Conflict Policies**: Folder copy and folder move currently support only `skip` and `overwrite` conflict handling across services; `rename`/`ask` are rejected until folder-level rename mapping exists.

---

## Operation Logging with Resolved Paths

### Purpose

The operation logging system includes resolved file and folder paths alongside their IDs, making logs more human-readable and easier to debug by showing actual file paths (e.g., `/Documents/Projects/Report.docx`) instead of opaque IDs (e.g., `abc123xyz`).

### Features

#### 1. Enhanced Log Entries

All operation logs now include:

- **File/Folder IDs**: The original cloud storage identifiers.
- **Resolved Paths**: Human-readable full paths when available.
- **Fallback Handling**: Graceful degradation to ID-only display if resolution fails.

#### 2. Supported Operations

Path resolution is applied to all logged file and folder operations:

- Copy operations (single file, single folder, batch).
- Move operations (single file, single folder, batch).
- Delete operations.
- Rename operations.
- Download operations.
- Upload operations.
- Any other file/folder operations.

#### 3. Multi-Service Support

Works across all supported cloud storage services:

- Google Drive.
- OneDrive.
- Dropbox.
- Box.
- pCloud.
- Jupiter.

### Data Structures

#### OperationItem Interface

```typescript
export interface OperationItem {
  id?: string; // Cloud storage ID
  name: string; // File/folder name
  type?: "file" | "folder";
  sizeBytes?: number;
  mimeType?: string;
  relativePath?: string;
  checksum?: string;
  resolvedPath?: string; // NEW: Full path resolved from ID
}
```

#### OperationEndpoint Interface

```typescript
export interface OperationEndpoint {
  service?: ServiceType;
  accountId?: string;
  folderId?: string;
  path?: string;
  remoteName?: string;
  label?: string;
  rootPath?: string;
  resolvedPath?: string; // Full path resolved from folderId
}
```

### Path Resolution Utilities

#### resolvePathsForLogging()

Main utility function for resolving paths in logging context:

```typescript
import { resolvePathsForLogging } from "~/lib/logging/path-resolver";

const resolved = await resolvePathsForLogging({
  service: "google",
  accountId: "default",
  items: [
    { id: "abc123", name: "Report.docx", type: "file" },
    { id: "xyz789", name: "Projects", type: "folder" },
  ],
  source: {
    service: "google",
    accountId: "default",
    folderId: "parent123",
  },
  destination: {
    service: "onedrive",
    accountId: "default",
    folderId: "dest456",
  },
});

// resolved.items[0].resolvedPath = "/Documents/Projects/Report.docx"
// resolved.items[1].resolvedPath = "/Documents/Projects"
// resolved.source.resolvedPath = "/Documents"
// resolved.destination.resolvedPath = "/Backups"
```

#### Key Features

1. **Batch Resolution**: Resolves multiple items in a single call
2. **Service-Agnostic**: Works with any cloud storage service
3. **Error Handling**: Gracefully handles resolution failures
4. **Performance**: Caches parent folder paths to minimize API calls
5. **Fallback**: Returns original data if resolution fails

### Integration Points

#### Activity Logs

Path resolution is automatically applied when logging user operations:

```typescript
import { logUserActivity } from "~/lib/logging/activity-logger";

await logUserActivity({
  userId: user.id,
  operationType: "copy",
  service: "google",
  accountId: "default",
  items: [{ id: "file123", name: "Report.docx", type: "file" }],
  source: { service: "google", folderId: "parent123" },
  destination: { service: "onedrive", folderId: "dest456" },
  status: "success",
});

// Automatically resolves paths before logging
```

#### System Logs

Path resolution is automatically applied when logging system operations:

```typescript
import { logSystemOperation } from "~/lib/logging/system-logger";

await logSystemOperation({
  operationType: "batch_copy",
  service: "google",
  accountId: "default",
  items: items,
  source: source,
  destination: destination,
  status: "success",
  metadata: { totalFiles: 10, totalSize: 1024000 },
});

// Automatically resolves paths before logging
```

---

## System Logs Enhancements

### Overview

The system logs page at `/admin/system-logs` has been enhanced with consolidated search fields and improved user display functionality, following the same patterns implemented for activity logs.

### Features Implemented

#### 0. Terminal Completion Finalization for Polling Fallbacks

Activity log completion/error entries are now finalized from both status paths:

- `/api/ops` (SSE stream).
- `/api/rclone/operations/[id]` (polling endpoint).

This ensures operation logs reach a terminal phase even when clients fall back from SSE to polling in production environments (for example, during cold starts or transient SSE connection failures).

#### 0.5. Consolidated Operation Type Filtering

Operation filtering now uses category-level options instead of separate single/batch options:

- Uploads.
- Downloads.
- Copy/Move.
- Delete.
- Create/Modify.
- Sync.
- Backup.

Each category expands to one or more concrete `operationType` values in the API request, so existing backend filters continue to work unchanged.

#### 1. Consolidated Operation/Request ID Search

**Before**: Two separate input fields for `operationId` and `requestId`
**After**: Single unified search field that searches both fields with OR logic

**Benefits:**

- Simplified UI (reduced from 6 filter columns to 5).
- Easier to use - type once to search both fields.
- Consistent with activity logs UX.

#### 2. Enhanced User Search

**Before**: Exact match on `userId` field only
**After**: Partial matching across multiple user identifier formats:

- Clerk user ID (e.g., "user_2abc123...").
- User email addresses (e.g., "user@example.com").

**Search Location**: Searches both `userId` column and `payload.user.email` JSONB field

**Benefits:**

- Find users by typing partial email domain (e.g., "example.com").
- Find users by partial Clerk ID (e.g., "user_2abc").
- Case-insensitive for better usability.

#### 3. Improved User Column Display

**Before**: Showed only user ID
**After**: Shows both email and user ID:

- Email as primary text (if available).
- User ID as secondary/muted text below email.
- Falls back to user ID only if email not available.

**Benefits:**

- Easier to identify users at a glance.
- More human-readable than UUIDs.
- Maintains responsive design on mobile.

---

## Implementation Details

### Database Query Changes

#### Search Parameter (Operation/Request ID)

```typescript
if (filters.search) {
  const sanitizedSearch = sanitizeLikePattern(filters.search);
  const pattern = `%${sanitizedSearch}%`;
  conditions.push(
    or(
      ilike(systemAuditLogs.operationId, pattern),
      ilike(systemAuditLogs.requestId, pattern),
    ),
  );
}
```

#### UserSearch Parameter (User ID/Email)

```typescript
if (filters.userSearch) {
  const sanitizedUserSearch = sanitizeLikePattern(filters.userSearch);
  const pattern = `%${sanitizedUserSearch}%`;
  conditions.push(
    or(
      ilike(systemAuditLogs.userId, pattern),
      // Search email in JSONB payload using PostgreSQL #>> operator
      ilike(sql`${systemAuditLogs.payload}#>>'{user,email}'`, pattern),
    ),
  );
}
```

### API Route Changes

Added new parameters to `/api/logs/audit`:

- `search` - Unified search for operationId and requestId.
- `userSearch` - Search for userId or email.
- `operationId` - Individual operationId filter (backward compatible).

### UI Changes

#### Filter Grid

```typescript
// Before: 6 columns with 3 separate text inputs
<div className="grid gap-3 sm:grid-cols-2 md:grid-cols-6">
  <Input placeholder="Filter by operation ID" />
  <Input placeholder="Filter by request ID" />
  <Input placeholder="Filter by user ID" />
</div>

// After: 5 columns with 2 consolidated inputs
<div className="grid gap-3 sm:grid-cols-2 md:grid-cols-5">
  <Input placeholder="Search by operation ID or request ID" />
  <Input placeholder="Search by user ID or email" />
</div>
```

---

## Usage Examples

### Example 1: Searching by Operation ID

```
Search: "6b24aa81"
Results: All logs with operationId or requestId containing "6b24aa81"
```

### Example 2: Searching by User Email

```
User Search: "example.com"
Results: All logs for users with email addresses containing "example.com"
```

### Example 3: Viewing Resolved Paths

```
Log Entry:
- Operation: Copy
- Source: /Documents/Projects/Report.docx (Google Drive)
- Destination: /Backups/2025-10-20/Report.docx (OneDrive)
- Status: Success
```

---

## Related Documentation

- [Activity Logs UI](./ACTIVITY_LOGS_UI.md) - User activity tracking and visualization.
- [Path Resolution](./PATH_RESOLUTION.md) - Comprehensive path resolution guide.
- [Architecture](./ARCHITECTURE.md) - Overall system architecture.
- [Testing](./TESTING.md) - Testing strategies and guidelines.

---

**Implementation Complete**: October 2025
**Status**: ✅ Production-Ready
**Coverage**: All system logging functionality