# AI Indexing Phase 7 Follow-Up - 2026-05-07

## Scope

- Local app: `http://localhost:3000`
- Indexing backend: local Next.js process with rclone service available on port `3001`
- Provider scope: Google account-scope indexing against a test account
- Live job: `a4cb3595df7204b328747022eecb45d4`
- Capture time: 2026-05-07 09:40 UTC

No credentials, raw provider paths, extracted text, raw provider errors, or raw account IDs are recorded in this note.

## Verification

The Phase 7 telemetry migration was applied to the local database through the canonical Drizzle stream after the app reported missing telemetry columns on `ai_indexed_files`.

After a Next.js server restart, the same account-scope indexing request reused the active job and re-kicked execution in the restarted process. Processed item count advanced from 239 to 253 during a 30 second poll window, confirming the resumed process was writing job progress again.

The live indexing job reached a mixed state during verification:

| Metric           |   Value |
| ---------------- | ------: |
| Status           | running |
| Total discovered |    1022 |
| Total eligible   |     330 |
| Indexed          |      21 |
| Skipped          |     123 |
| Failed           |      14 |

The health endpoint briefly returned 500 during server rollover/Fast Refresh, then returned 200 after the restarted server stabilized.

The `/user/ai-indexing` page rendered:

- account-level health summary
- active job progress
- diagnostics dialog for the active job
- extraction method breakdown
- issue groups with recommendations
- retryability count
- extraction latency percentiles
- affected samples and slowest-item tables

UI privacy check:

- raw Clerk requester IDs were not visible after the requester-label fix
- raw `system:*` requester values were not visible after the requester-label fix
- diagnostics dialog text did not include stack traces or raw user IDs

## Latency Baseline

This is an early baseline from a still-running job. Treat methods with very small sample counts as provisional.

| Extraction method | Samples | Avg ms | P50 ms | P95 ms | P99 ms | Min ms | Max ms |
| ----------------- | ------: | -----: | -----: | -----: | -----: | -----: | -----: |
| docx              |       4 |     31 |     22 |     61 |     67 |     11 |     68 |
| ocr_image         |       2 |      0 |      0 |      0 |      0 |      0 |      0 |
| plain_text        |      15 |      0 |      0 |      2 |      3 |      0 |      3 |
| pptx              |       1 |      0 |      0 |      0 |      0 |      0 |      0 |
| xlsx              |       5 |     36 |      0 |    139 |    165 |      0 |    171 |

Methods without completed extraction-duration samples at capture time are omitted from the baseline table.

## Outcome Mix

| Method      | Status   | Count |
| ----------- | -------- | ----: |
| docx        | failed   |     5 |
| docx        | indexed  |     4 |
| ocr_image   | skipped  |     2 |
| pdf         | failed   |     7 |
| plain_text  | indexed  |    15 |
| plain_text  | indexing |     1 |
| pptx        | failed   |     1 |
| pptx        | skipped  |     1 |
| unsupported | skipped  |   117 |
| xlsx        | failed   |     1 |
| xlsx        | indexed  |     2 |
| xlsx        | skipped  |     3 |
| unrecorded  | pending  |   863 |

## Error Message Review

Useful messages observed in the diagnostics UI:

- PDF parse failures recommend exporting or saving the PDF again before retrying.
- DOCX parse failures recommend saving the document again as DOCX before retrying.
- PPTX and XLSX encrypted failures recommend using unlocked copies.
- Unsupported file type failures point users to supported formats.

Issues found and fixed:

- Job cards rendered internal requester IDs. They now show `Requested by: You` or `Requested by: System`.
- OCR-disabled image skips collapsed to `unknown_error`. The taxonomy now maps OCR-required/disabled wording to `ocr_invalid_provider_config`.
- Explicit allow-list unsupported wording collapsed to `unknown_error`. The taxonomy now maps allow-list wording to `unsupported_file_type`.

Existing rows produced before the taxonomy fixes retain their original telemetry. New item attempts use the updated mappings.

## Follow-Ups

- Refresh the latency baseline after the live job completes.
- Re-run a smaller OCR-specific job after the taxonomy fix to verify `ocr_invalid_provider_config` appears in the diagnostics UI.
- Consider adding a recurring regression query that alerts when p95 or p99 extraction latency doubles for a method with at least 20 samples.