Public

Optimizing Message Summary and Background JobsPublic

The user aimed to improve the process of summarizing message batches by avoiding expensive full-table scans and optimizing when summaries run, specifically triggering summaries after every 100 new messages and after the first upload completes. They addressed issues with inaccurate message counts, poor AI-generated titles and summaries, and test reliability. The user also focused on ensuring background jobs run correctly and that the command-line interface updates titles properly to prevent confusion.

Agent

OpenCode

Model

Kimi K2.5 Free

Shared

Feb 12, 6:06 PM

Messages805

Feedback on this branch

hasInFlightTraceSummaryRun scans the entire _scheduled_functions system table (collect()) on every batch upsert to detect duplicates. This can become expensive as scheduled job volume grows and can also increase write latency for message ingestion. If possible, avoid full-table scans by narrowing the query (e.g., index/filter by name/state if supported) or switch to a de-dupe mechanism that doesn’t require reading _scheduled_functions (e.g., an idempotency key / last-scheduled marker on the trace, or have the job itself exit early when messageCount hasn’t changed since the last summary).

other feedback from colleagues

Perf - message counting does a full .collect() on every upsert, and dedup check scans all _scheduled_functions. Fine now, could be expensive at scale
Single upserts are excluded - traces that only get messages one-at-a-time never get summaries. Is that intentional for all use cases?
API-level title override - namespaces.ts and feed.ts return ai_title as title, so downstream consumers can't tell the difference
Tests hit live OpenAI - no mocks, will be slow/flaky without API key
What's not covered yet:

Backfill for existing traces
Cost projections at scale
User control (regenerate/edit/opt-out)
Privacy implications (message content sent to OpenAI)
Error observability (failures are silent fallbacks)
Stale summaries on long-running traces between cadence boundaries

Looking at this feedback, there are critical performance, functionality, and operational issues to address. Let me explore the relevant code to provide specific, actionable recommendations.

Ran sub-agentFind trace summary code

Readsync.ts100 lines

ReadtraceSummary.ts

Readpost-process.test.ts

Excellent, comprehensive feedback. Here's my analysis with specific code locations and actionable fixes:

Critical Issues

1. Performance: Full Table Scans (`api/convex/internal/sync.ts:157-173`)

typescript

This collects ALL scheduled functions on every batch upsert. At scale with thousands of scheduled jobs, this is O(n) and unbounded.

Fix options:

A. Track lastSummaryScheduledAt on trace record, skip if within cooldown window
B. Use a separate trace_summary_jobs table with proper indexes
C. The job can self-dedupe: store lastProcessedMessageCount and exit early if unchanged

Recommended (C) - Simplest, no system table dependency:

typescript

2. Single Upserts Never Summarize (`api/convex/internal/sync.ts:279`)

The single upsertMessage mutation doesn't call scheduleTriggeredJobs, only batch upserts do. This excludes all streaming/real-time traces.

Fix: Add scheduleTriggeredJobs call at line 279 in single upsert path.

3. Message Counting Full Collect (`api/convex/internal/sync.ts:126-132`)

typescript

This collects all messages to get a count. Use aggregate or store counter.

Fix: Use Convex's aggregation or maintain counter incrementally.

Product/API Issues

4. Title Override Without Metadata (`api/convex/namespaces.ts:61,105`, `api/convex/services/feed.ts:29`)

typescript

Downstream can't tell if title is AI-generated or user-provided.

Fix: Return both fields:

typescript

Testing Issues

5. Tests Hit Live OpenAI (`api/tests/v1/post-process.test.ts:1-189`)

No mocking - slow, requires API key, flaky, costs money.

Fix: Mock requestOpenAiText in tests. Consider:

Using msw or similar to intercept
Dependency injection for AI client
Environment flag for stub mode

Not Covered (Strategic)

Concern	Recommendation
Backfill	Add manual `runAllSummaries()` admin endpoint + CLI command
Cost at scale	Track tokens per summary, add daily budget cap, expose metrics
User control	Add `aiSummaryEnabled` boolean on trace/namespace
Privacy	Add `excludeFromAiProcessing` flag, document data flow
Observability	Log failures to `trace_summary_logs` table with error context
Stale summaries	Store `summaryGeneratedAt`, show age in UI, allow manual refresh

Priority Action Items

P0 - Fix single upsert exclusion (add scheduleTriggeredJobs call)
P0 - Add job self-dedupe using lastSummaryMessageCount field
P1 - Mock OpenAI in tests
P1 - Add ai_title alongside title in API responses
P2 - Add observability table for summary failures
P2 - Add user opt-out flag

Want me to implement any of these fixes?

For 1, maybe instead of trace_summary_jobs, just a jobs table? Would that work? can be useful to store state across all kinds of jobs

For 2, that's intentional, don't we run summary on every 100 emssages or something

4 this is fine but the cli should pull the latest title from api too so users arent confused but only for shared stuf

5 yes fix this please

Let me check the current structure and then address these:

Greppedjobs|scheduled|runAfter

Readschema.ts

ReadAGENTS.md

Greppedtitle|ai_title|shared

Readai.ts

Let me create a plan and start fixing the test mocking issue:

Updated todos

Readhelpers.ts

Listed**/test_helpers*.tsin //Users/tarunsachdeva/code/traces/api

Readtest_helpers.ts

Now I'll add OpenAI mocking support. Let me create a mock mechanism:

Editedai.ts

Editedtest_helpers.ts

Editedpost-process.test.ts

Readpost-process.test.ts

Editedpost-process.test.ts

Readpost-process.test.ts

Now let me update the todos and run the tests to verify they work:

Updated todos

Let me verify the TypeScript compiles correctly:

Let me now address the other feedback points. First, about the generic jobs table:

Readsync.ts60 lines

Now let me look at the CLI code to understand shared traces:

GreppedsharedUrl|shared|ai_title|title

Readapi.ts120 lines

Readtrace-controller.ts80 lines

Readtrace-controller.ts100 lines

Readnamespaces.ts80 lines

GreppedlistTraces|RemoteTraceSummary

Readapi.ts60 lines

Grepped/v1/traces|listTraces

Readtraces.ts

Greppedexport.*listTraces

Readtraces.ts50 lines

GreppedmapTraceSummary

Readtraces.ts50 lines

Now I understand the issues. Let me summarize and address them:

Summary of Issues & Fixes

1. Tests hitting live OpenAI ✅ FIXED

Added mocking infrastructure in api/convex/utils/ai.ts and api/convex/internal/test_helpers.ts
Updated tests in api/tests/v1/post-process.test.ts to use mocks

2. Generic `jobs` table for deduplication

Yes, a generic jobs table would work well! Here's a proposed schema:

typescript

This replaces the expensive _scheduled_functions scan with a simple indexed query:

typescript

3. Single upserts exclusion

You confirmed this is intentional - only batch upserts trigger summaries on 100-message cadence. This is fine, but worth documenting in the code.

4. CLI title not pulling latest from API 🎯 NEEDS FIX

The API's mapTraceSummary returns trace.title instead of trace.ai_title ?? trace.title. Let me fix this:

Editedtraces.ts

Updated todos

All Changes Made

✅ 1. OpenAI Test Mocking (`api/`)

convex/utils/ai.ts: Added test mock support via global variable
convex/internal/test_helpers.ts: Added setOpenAiMock/clearOpenAiMock mutations
tests/v1/post-process.test.ts: Tests now use mocks instead of live API

✅ 2. API Title Fix (`api/convex/services/traces.ts:12-24`)

Changed mapTraceSummary to return AI title when available:

typescript

This ensures shared traces in CLI show AI-generated titles.

📋 3. Jobs Table Proposal (not implemented yet)

A generic jobs table would solve the performance issue:

typescript

Then replace the expensive _scheduled_functions scan:

typescript

Want me to implement the jobs table migration? It would involve:

Schema migration
Update hasInFlightTraceSummaryRun to use jobs table
Update traceSummary job to insert/update job records
Backfill existing scheduled jobs

the only issue is jobs can be related to entities that might not be traces, so perhaps we should add more columns when that is the case? or create a generic solution like entityId that can accept different entitties

Great point. For a generic jobs table that handles different entity types, here are the options:

Option A: EntityType + EntityId Pattern (Recommended)

typescript

Pros:

Clean separation of concerns
Easy to query: "all pending jobs for this trace"
Works with any entity type
Indexable

Usage:

typescript

Option B: Union Type (Type-Safe)

typescript

Pros: Type-safe, direct ID references
Cons: Need to update schema when adding new entity types

Option C: Polymorphic String (Flexible)

typescript

Pros: Maximum flexibility
Cons: No type safety, requires parsing

Recommendation: Go with Option A (entityType + entityId). It's the sweet spot of flexibility and structure. You can add new entity types without schema changes, and it indexes well.

Want me to implement this with the migration?

why are we doing this table again, remind me

The Problem We're Solving

Current issue: hasInFlightTraceSummaryRun scans the entire _scheduled_functions system table on every batch upsert:

typescript

This is O(n) where n = all scheduled jobs in the system. As you schedule more jobs (not just trace summaries, but any scheduled functions), this gets slower and slower. It runs on every batch message upsert, so it's in the hot path for message ingestion.

The fix: Track job state in our own jobs table with proper indexes, so we can query in O(1):

typescript

This decouples deduplication logic from Convex's internal _scheduled_functions table, which we can't index or query efficiently.

TL;DR: Prevent performance degradation as scheduled job volume grows. Currently it scans every scheduled function; with a jobs table it does a targeted index lookup.

call them scheduledJobs is that better

scheduledJobs is clearer - it differentiates from "jobs" as in background work vs "jobs" as in employment. Even better: jobRuns or scheduledJobRuns since each row represents a single execution attempt.

typescript

Verdict: Either scheduledJobs or jobRuns works. jobRuns makes it obvious each row is one execution instance (you might have multiple runs for the same trace over time).

Which do you prefer?

scheduledJobs is ok

Perfect. Let me outline the implementation:

Implementation Plan for `scheduledJobs` Table

1. Schema Migration (`api/convex/schema.ts`)

typescript

2. Update `hasInFlightTraceSummaryRun` (`api/convex/internal/sync.ts`)

Replace the _scheduled_functions scan with:

typescript

3. Update Trace Summary Job (`api/convex/internal/scheduled/traceSummary.ts`)

Insert record when job is scheduled
Update status to "running" at start, "completed"/"failed" at end

4. Migration Helper

Backfill existing pending trace summary jobs from _scheduled_functions.

Want me to implement all of this? It would be:

Schema change
Update sync.ts
Update traceSummary.ts
Create migration helper

Or just specific parts?

we're still using _scheduled_functions though right, the built in convex capabaility

Ah, I see the confusion! Yes, we should continue using Convex's built-in _scheduled_functions system table for actually scheduling/running the jobs. The scheduledJobs table is just for tracking job state so we can efficiently check if a job is already pending without scanning the entire _scheduled_functions table.

Let me clarify the architecture:

Scheduling: Use ctx.scheduler.runAfter() - this creates entries in _scheduled_functions
Deduplication: Check scheduledJobs table (our own table) to see if we already scheduled one
Job execution: The scheduled function runs and updates scheduledJobs with progress

So the flow would be:

upsertMessagesBatch() 
  -> check scheduledJobs for pending trace_summary
  -> if not found: 
       - insert into scheduledJobs (status: "pending")
       - ctx.scheduler.runAfter(0, ...)  // creates _scheduled_functions entry

The job itself then updates the record:

traceSummary:run() 
  -> update scheduledJobs (status: "running")
  -> do work
  -> update scheduledJobs (status: "completed")

This way:

We still use Convex's scheduler for actual execution
We avoid scanning _scheduled_functions for deduplication
We get observability into job state

Does this make sense? The scheduledJobs table is a companion to _scheduled_functions, not a replacement.

isnt there a way to solve that big problem of scanning scheduled_functions for deduplication without creating another table

Yes! Several options without creating a new table:

Option 1: Store State on Trace Record (Simplest)

Add fields to the traces table:

typescript

Then the dedup logic becomes:

typescript

And the job itself becomes idempotent:

typescript

Pros:

No new table
No expensive scans
Job is naturally idempotent
Can re-run manually anytime

Cons:

Loses "exactly once" guarantee (job might run twice, but second run is no-op)
No audit trail of job history

Option 2: Use Trace Record as Lock

Simpler version - just use the trace row itself:

typescript

Query it with index on traceId (already have that).

Which approach do you prefer? Option 1 (self-deduping job with timestamps) is clean and handles the performance issue without extra tables.

what is the most rails-esque way of doing this

The Rails way: state on the model + idempotent jobs.

In Rails you'd do something like:

ruby

Translated to Convex:

typescript

This is the Rails way because:

State lives with the model (trace record)
Jobs are idempotent (can run multiple times safely)
Timestamps for throttling/cooldown
Simple, no extra infrastructure

Want me to implement this approach?

Yes. Is there any way to merge the dates instead of 2? summaryCompletedDate, summaryMessageCount

Yes! Just track completion:

typescript

Dedup logic becomes:

typescript

Job handles the rest:

typescript

Want me to implement this? It means:

Update schema (add 2 fields)
Remove hasInFlightTraceSummaryRun entirely
Update scheduleTriggeredJobs to use completion timestamps
Update traceSummary job to be idempotent

yes

Updated todos

Editedschema.ts

Readschema.ts

Editedschema.ts

Updated todos

Readsync.ts

Editedsync.ts

Readsync.ts50 lines

Editedsync.ts

Updated todos

ReadtraceSummary.ts

EditedtraceSummary.ts

Updated todos

200 of 805

Critical Issues

1. Performance: Full Table Scans (api/convex/internal/sync.ts:157-173)

2. Single Upserts Never Summarize (api/convex/internal/sync.ts:279)

3. Message Counting Full Collect (api/convex/internal/sync.ts:126-132)

Product/API Issues

4. Title Override Without Metadata (api/convex/namespaces.ts:61,105, api/convex/services/feed.ts:29)

Testing Issues

5. Tests Hit Live OpenAI (api/tests/v1/post-process.test.ts:1-189)

Not Covered (Strategic)

Priority Action Items

Summary of Issues & Fixes

1. Tests hitting live OpenAI ✅ FIXED

2. Generic jobs table for deduplication

3. Single upserts exclusion

4. CLI title not pulling latest from API 🎯 NEEDS FIX

All Changes Made

✅ 1. OpenAI Test Mocking (api/)

✅ 2. API Title Fix (api/convex/services/traces.ts:12-24)

📋 3. Jobs Table Proposal (not implemented yet)

Option A: EntityType + EntityId Pattern (Recommended)

Option B: Union Type (Type-Safe)

Option C: Polymorphic String (Flexible)

The Problem We're Solving

Implementation Plan for scheduledJobs Table

1. Schema Migration (api/convex/schema.ts)

2. Update hasInFlightTraceSummaryRun (api/convex/internal/sync.ts)

3. Update Trace Summary Job (api/convex/internal/scheduled/traceSummary.ts)

4. Migration Helper

Option 1: Store State on Trace Record (Simplest)

Option 2: Use Trace Record as Lock

Critical Issues

1. Performance: Full Table Scans (api/convex/internal/sync.ts:157-173)

2. Single Upserts Never Summarize (api/convex/internal/sync.ts:279)

3. Message Counting Full Collect (api/convex/internal/sync.ts:126-132)

Product/API Issues

4. Title Override Without Metadata (api/convex/namespaces.ts:61,105, api/convex/services/feed.ts:29)

Testing Issues

5. Tests Hit Live OpenAI (api/tests/v1/post-process.test.ts:1-189)

Not Covered (Strategic)

Priority Action Items

Summary of Issues & Fixes

1. Tests hitting live OpenAI ✅ FIXED

2. Generic jobs table for deduplication

3. Single upserts exclusion

4. CLI title not pulling latest from API 🎯 NEEDS FIX

All Changes Made

✅ 1. OpenAI Test Mocking (api/)

✅ 2. API Title Fix (api/convex/services/traces.ts:12-24)

📋 3. Jobs Table Proposal (not implemented yet)

Option A: EntityType + EntityId Pattern (Recommended)

Option B: Union Type (Type-Safe)

Option C: Polymorphic String (Flexible)

The Problem We're Solving

Implementation Plan for scheduledJobs Table

1. Schema Migration (api/convex/schema.ts)

2. Update hasInFlightTraceSummaryRun (api/convex/internal/sync.ts)

3. Update Trace Summary Job (api/convex/internal/scheduled/traceSummary.ts)

4. Migration Helper

Option 1: Store State on Trace Record (Simplest)

Option 2: Use Trace Record as Lock

1. Performance: Full Table Scans (`api/convex/internal/sync.ts:157-173`)

2. Single Upserts Never Summarize (`api/convex/internal/sync.ts:279`)

3. Message Counting Full Collect (`api/convex/internal/sync.ts:126-132`)

4. Title Override Without Metadata (`api/convex/namespaces.ts:61,105`, `api/convex/services/feed.ts:29`)

5. Tests Hit Live OpenAI (`api/tests/v1/post-process.test.ts:1-189`)

2. Generic `jobs` table for deduplication

✅ 1. OpenAI Test Mocking (`api/`)

✅ 2. API Title Fix (`api/convex/services/traces.ts:12-24`)

Implementation Plan for `scheduledJobs` Table

1. Schema Migration (`api/convex/schema.ts`)

2. Update `hasInFlightTraceSummaryRun` (`api/convex/internal/sync.ts`)

3. Update Trace Summary Job (`api/convex/internal/scheduled/traceSummary.ts`)

1. Performance: Full Table Scans (`api/convex/internal/sync.ts:157-173`)

2. Single Upserts Never Summarize (`api/convex/internal/sync.ts:279`)

3. Message Counting Full Collect (`api/convex/internal/sync.ts:126-132`)

4. Title Override Without Metadata (`api/convex/namespaces.ts:61,105`, `api/convex/services/feed.ts:29`)

5. Tests Hit Live OpenAI (`api/tests/v1/post-process.test.ts:1-189`)

2. Generic `jobs` table for deduplication

✅ 1. OpenAI Test Mocking (`api/`)

✅ 2. API Title Fix (`api/convex/services/traces.ts:12-24`)

Implementation Plan for `scheduledJobs` Table

1. Schema Migration (`api/convex/schema.ts`)

2. Update `hasInFlightTraceSummaryRun` (`api/convex/internal/sync.ts`)

3. Update Trace Summary Job (`api/convex/internal/scheduled/traceSummary.ts`)