Signals call Jev one item at a time, so a new Signal takes about 20 minutes to fill #275

Closed
opened 2026-10-17 06:58:32 -07:00 by rays · 1 comment
Owner

signals::pass awaits each Jev call before the next: a full pass of 50 items took 7.2 s on 2026-10-16 (trace fcace60dda9a2f5fc2ac05fad19896b), 50 calls of 74-267 ms. A new or edited Signal looks back 30 days, about 8,000 items of production's subscribed feeds, so at ~150 ms a call it fills over roughly 20 minutes. A few calls in flight at once (bounded, for TypeSafe's sake) would cut that to a few minutes. Also: on the server's key, two people's Signals on the same item are two calls with the same text where one would do; and the answers' usage (input and output tokens) is thrown away, so what Signals cost is not visible.

signals::pass awaits each Jev call before the next: a full pass of 50 items took 7.2 s on 2026-10-16 (trace fcace60dda9a2f5fc2ac05fad19896b), 50 calls of 74-267 ms. A new or edited Signal looks back 30 days, about 8,000 items of production's subscribed feeds, so at ~150 ms a call it fills over roughly 20 minutes. A few calls in flight at once (bounded, for TypeSafe's sake) would cut that to a few minutes. Also: on the server's key, two people's Signals on the same item are two calls with the same text where one would do; and the answers' usage (input and output tokens) is thrown away, so what Signals cost is not visible.
rays added the enhancement label 2026-10-17 06:58:32 -07:00
Author
Owner

Done in 2077ea5: six calls to Jev in flight (IN_FLIGHT); on the server's key everyone's Signals on one item go in one call; TypeSafe's usage counted in ipx_jev_tokens_total{kind} and on each jev span, with a Tokens row on the dashboard; TEXT_CHARS 2000 -> 1000 after a test on the development key (55 items of 8 podcast feeds, 6 Signals: 1000 changed 0 of 330 verdicts and saved 12% of input tokens, 1266 -> 1112 a call; 500 changed 1). Deployed in ipx 1.3.1-dev.

Done in 2077ea5: six calls to Jev in flight (IN_FLIGHT); on the server's key everyone's Signals on one item go in one call; TypeSafe's usage counted in ipx_jev_tokens_total{kind} and on each jev span, with a Tokens row on the dashboard; TEXT_CHARS 2000 -> 1000 after a test on the development key (55 items of 8 podcast feeds, 6 Signals: 1000 changed 0 of 330 verdicts and saved 12% of input tokens, 1266 -> 1112 a call; 500 changed 1). Deployed in ipx 1.3.1-dev.
rays closed this issue 2026-10-17 07:21:19 -07:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rays/ipx#275