Optimizing Message Summary and Background JobsPublic

Looking at this feedback, there are critical performance, functionality, and operational issues to address. Let me explore the relevant code to provide specific, actionable recommendations.

Excellent, comprehensive feedback. Here's my analysis with specific code locations and actionable fixes:

Critical Issues

1. Performance: Full Table Scans (api/convex/internal/sync.ts:157-173)

This collects ALL scheduled functions on every batch upsert. At scale with thousands of scheduled jobs, this is O(n) and unbounded.

Fix options:

• A. Track lastSummaryScheduledAt on trace record, skip if within cooldown window
• B. Use a separate trace_summary_jobs table with proper indexes
• C. The job can self-dedupe: store lastProcessedMessageCount and exit early if unchanged

Recommended (C) - Simplest, no system table dependency:

2. Single Upserts Never Summarize (api/convex/internal/sync.ts:279)

The single upsertMessage mutation doesn't call scheduleTriggeredJobs, only batch upserts do. This excludes all streaming/real-time traces.

Fix: Add scheduleTriggeredJobs call at line 279 in single upsert path.

3. Message Counting Full Collect (api/convex/internal/sync.ts:126-132)

This collects all messages to get a count. Use aggregate or store counter.

Fix: Use Convex's aggregation or maintain counter incrementally.

Product/API Issues

4. Title Override Without Metadata (api/convex/namespaces.ts:61,105, api/convex/services/feed.ts:29)

Downstream can't tell if title is AI-generated or user-provided.

Fix: Return both fields:

Testing Issues

5. Tests Hit Live OpenAI (api/tests/v1/post-process.test.ts:1-189)

No mocking - slow, requires API key, flaky, costs money.

Fix: Mock requestOpenAiText in tests. Consider:

• Using msw or similar to intercept
• Dependency injection for AI client
• Environment flag for stub mode

Not Covered (Strategic)

Priority Action Items

1. P0 - Fix single upsert exclusion (add scheduleTriggeredJobs call)
2. P0 - Add job self-dedupe using lastSummaryMessageCount field
3. P1 - Mock OpenAI in tests
4. P1 - Add ai_title alongside title in API responses
5. P2 - Add observability table for summary failures
6. P2 - Add user opt-out flag

Want me to implement any of these fixes?

Let me check the current structure and then address these:

Let me create a plan and start fixing the test mocking issue:

Now I'll add OpenAI mocking support. Let me create a mock mechanism:

Now let me update the todos and run the tests to verify they work:

Let me verify the TypeScript compiles correctly:

Let me now address the other feedback points. First, about the generic jobs table:

Now let me look at the CLI code to understand shared traces:

Now I understand the issues. Let me summarize and address them:

Summary of Issues & Fixes

1. Tests hitting live OpenAI ✅ FIXED

• Added mocking infrastructure in api/convex/utils/ai.ts and api/convex/internal/test_helpers.ts
• Updated tests in api/tests/v1/post-process.test.ts to use mocks

2. Generic jobs table for deduplication

Yes, a generic jobs table would work well! Here's a proposed schema:

This replaces the expensive _scheduled_functions scan with a simple indexed query:

3. Single upserts exclusion

You confirmed this is intentional - only batch upserts trigger summaries on 100-message cadence. This is fine, but worth documenting in the code.

4. CLI title not pulling latest from API 🎯 NEEDS FIX

The API's mapTraceSummary returns trace.title instead of trace.ai_title ?? trace.title. Let me fix this:

All Changes Made

✅ 1. OpenAI Test Mocking (api/)

• convex/utils/ai.ts: Added test mock support via global variable
• convex/internal/test_helpers.ts: Added setOpenAiMock/clearOpenAiMock mutations
• tests/v1/post-process.test.ts: Tests now use mocks instead of live API

✅ 2. API Title Fix (api/convex/services/traces.ts:12-24)

Changed mapTraceSummary to return AI title when available:

This ensures shared traces in CLI show AI-generated titles.

📋 3. Jobs Table Proposal (not implemented yet)

A generic jobs table would solve the performance issue:

Then replace the expensive _scheduled_functions scan:

Want me to implement the jobs table migration? It would involve:

1. Schema migration
2. Update hasInFlightTraceSummaryRun to use jobs table
3. Update traceSummary job to insert/update job records
4. Backfill existing scheduled jobs

Great point. For a generic jobs table that handles different entity types, here are the options:

Option A: EntityType + EntityId Pattern (Recommended)

Pros:

• Clean separation of concerns
• Easy to query: "all pending jobs for this trace"
• Works with any entity type
• Indexable

Usage:

Option B: Union Type (Type-Safe)

Pros: Type-safe, direct ID references
Cons: Need to update schema when adding new entity types

Option C: Polymorphic String (Flexible)

Pros: Maximum flexibility
Cons: No type safety, requires parsing

Recommendation: Go with Option A (entityType + entityId). It's the sweet spot of flexibility and structure. You can add new entity types without schema changes, and it indexes well.

Want me to implement this with the migration?

The Problem We're Solving

Current issue: hasInFlightTraceSummaryRun scans the entire _scheduled_functions system table on every batch upsert:

This is O(n) where n = all scheduled jobs in the system. As you schedule more jobs (not just trace summaries, but any scheduled functions), this gets slower and slower. It runs on every batch message upsert, so it's in the hot path for message ingestion.

The fix: Track job state in our own jobs table with proper indexes, so we can query in O(1):

This decouples deduplication logic from Convex's internal _scheduled_functions table, which we can't index or query efficiently.

TL;DR: Prevent performance degradation as scheduled job volume grows. Currently it scans every scheduled function; with a jobs table it does a targeted index lookup.

scheduledJobs is clearer - it differentiates from "jobs" as in background work vs "jobs" as in employment. Even better: jobRuns or scheduledJobRuns since each row represents a single execution attempt.

Verdict: Either scheduledJobs or jobRuns works. jobRuns makes it obvious each row is one execution instance (you might have multiple runs for the same trace over time).

Which do you prefer?

Perfect. Let me outline the implementation:

Implementation Plan for scheduledJobs Table

1. Schema Migration (api/convex/schema.ts)

2. Update hasInFlightTraceSummaryRun (api/convex/internal/sync.ts)

Replace the _scheduled_functions scan with:

3. Update Trace Summary Job (api/convex/internal/scheduled/traceSummary.ts)

• Insert record when job is scheduled
• Update status to "running" at start, "completed"/"failed" at end

4. Migration Helper

Backfill existing pending trace summary jobs from _scheduled_functions.

Want me to implement all of this? It would be:

1. Schema change
2. Update sync.ts
3. Update traceSummary.ts
4. Create migration helper

Or just specific parts?

Ah, I see the confusion! Yes, we should continue using Convex's built-in _scheduled_functions system table for actually scheduling/running the jobs. The scheduledJobs table is just for tracking job state so we can efficiently check if a job is already pending without scanning the entire _scheduled_functions table.

Let me clarify the architecture:

1. Scheduling: Use ctx.scheduler.runAfter() - this creates entries in _scheduled_functions
2. Deduplication: Check scheduledJobs table (our own table) to see if we already scheduled one
3. Job execution: The scheduled function runs and updates scheduledJobs with progress

So the flow would be:

upsertMessagesBatch() 
  -> check scheduledJobs for pending trace_summary
  -> if not found: 
       - insert into scheduledJobs (status: "pending")
       - ctx.scheduler.runAfter(0, ...)  // creates _scheduled_functions entry

The job itself then updates the record:

traceSummary:run() 
  -> update scheduledJobs (status: "running")
  -> do work
  -> update scheduledJobs (status: "completed")

This way:

• We still use Convex's scheduler for actual execution
• We avoid scanning _scheduled_functions for deduplication
• We get observability into job state

Does this make sense? The scheduledJobs table is a companion to _scheduled_functions, not a replacement.

Yes! Several options without creating a new table:

Option 1: Store State on Trace Record (Simplest)

Add fields to the traces table:

Then the dedup logic becomes:

And the job itself becomes idempotent:

Pros:

• No new table
• No expensive scans
• Job is naturally idempotent
• Can re-run manually anytime

Cons:

• Loses "exactly once" guarantee (job might run twice, but second run is no-op)
• No audit trail of job history

Option 2: Use Trace Record as Lock

Simpler version - just use the trace row itself:

Query it with index on traceId (already have that).

Which approach do you prefer? Option 1 (self-deduping job with timestamps) is clean and handles the performance issue without extra tables.

The Rails way: state on the model + idempotent jobs.

In Rails you'd do something like:

Translated to Convex:

This is the Rails way because:

• State lives with the model (trace record)
• Jobs are idempotent (can run multiple times safely)
• Timestamps for throttling/cooldown
• Simple, no extra infrastructure

Want me to implement this approach?

Yes! Just track completion:

Dedup logic becomes:

Job handles the rest:

Want me to implement this? It means:

1. Update schema (add 2 fields)
2. Remove hasInFlightTraceSummaryRun entirely
3. Update scheduleTriggeredJobs to use completion timestamps
4. Update traceSummary job to be idempotent

All done! Here's a summary of the Rails-style implementation:

Changes Made

1. Schema (api/convex/schema.ts)

Added to traces table:

2. Removed Scanning (api/convex/internal/sync.ts)

• Deleted hasInFlightTraceSummaryRun and helper functions
• Deleted isInFlightScheduledFunction, hasTraceSummaryName, getTraceIdFromScheduledArgs
• Added shouldScheduleSummary function (Rails-style dedup)

3. New Scheduling Logic (api/convex/internal/sync.ts)

4. Idempotent Job (api/convex/internal/scheduled/traceSummary.ts)

• Job now self-dedupes: exits early if messageCount <= summaryMessageCount
• Returns { skipped: true } when deduped
• Updates summaryCompletedAt and summaryMessageCount on completion

5. API Title Fix (api/convex/services/traces.ts)

Changed mapTraceSummary to return AI title:

6. Test Mocks (api/tests/v1/post-process.test.ts)

Added OpenAI mocking to avoid hitting live API

Result: No more scanning _scheduled_functions. Instead we use O(1) indexed lookups on the trace record itself. The job is naturally idempotent - can run multiple times safely, only processes when there's new content.

Done.

Summary of Changes:

1. schema.ts - Added summaryCompletedAt and summaryMessageCount to traces
2. sync.ts - Removed hasInFlightTraceSummaryRun, replaced with shouldScheduleSummary (checks cooldown + message count)
3. traceSummary.ts - Job now self-dedupes, updates completion fields
4. traces.ts - mapTraceSummary returns ai_title ?? title
5. post-process.test.ts - Added OpenAI mocking

Done. All "Rails" comments removed from:

• schema.ts
• sync.ts
• traceSummary.ts

Yes, much cleaner. Just use message counts: