Signals: calls to TypeSafe paced under its rate limits, a 429 honoured, and the wait traced, counted and drawn (#277)
TypeSafe allows 80 requests and 100,000 input tokens a second. Six calls in flight was a cap on concurrency, not rate: at 74 ms a call it is 81 a second. Every call to Jev now takes a turn, a 60th of a second or its guessed tokens' share of 80,000, so up to ten can be in flight. A 429 holds every call back for Retry-After and ends the pass. Each jev span records paced_ms, /metrics ipx_jev_paced_seconds_total and a rate_limited result, and the dashboard draws calls and tokens a second against the limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -93,8 +93,14 @@ is asked by; without it each wake read all of `entries`, 605 MB, 566 ms to find
|
||||
|
||||
Rows for one item and one person are grouped, so an item is one call whatever the number of
|
||||
Signals. On the server's key, everyone's Signals on the same item go in one call too, its text
|
||||
sent once (#275). Six calls are in flight at once (`IN_FLIGHT`): one at a time, about 150 ms each,
|
||||
a month of items took twenty minutes. A full batch means there may be more, and the next pass
|
||||
sent once (#275). Up to ten calls are in flight at once (`IN_FLIGHT`): one at a time, about 150 ms
|
||||
each, a month of items took twenty minutes. Every call to Jev, Signals', icons' and key checks',
|
||||
takes a turn (`pace`, #277): TypeSafe's default limits are 80 requests and 100,000 input tokens a
|
||||
second, and ipx keeps to 60 and 80,000, each call holding the line for a 60th of a second or its
|
||||
guessed tokens' share, whichever is longer. A 429 holds every call back for its `Retry-After`, or
|
||||
a second, and ends the pass. Each `jev` span says how long it waited (`paced_ms`), and
|
||||
`ipx_jev_paced_seconds_total` adds them up; the dashboard draws calls and tokens a second against
|
||||
the limits. A full batch means there may be more, and the next pass
|
||||
starts at once. Otherwise the judge sleeps until woken or for 5 minutes.
|
||||
|
||||
### Every answer is kept
|
||||
@@ -181,6 +187,7 @@ missing from the answer is left unjudged and asked again next pass.
|
||||
| 401, 402, 403, on the server's key | Everyone's key is refused. The pass ends and the queue waits for the next one, five minutes on. Logged: "TypeSafe refused the server's key for Signals". |
|
||||
| 401, 402, 403, on a person's key | The key is refused. That person is passed over until they save a key or the daemon restarts, so a bad key is not sent the whole queue. Logged once: "TypeSafe refused a Signal key". |
|
||||
| 400, 422 | The request itself is wrong, so every item's would be. The pass ends; the items wait for the next one. Logged: "TypeSafe refused a Signals request as malformed", with TypeSafe's reason. |
|
||||
| 429 | Over TypeSafe's limits. Every call waits for `Retry-After` or a second; the pass ends, what was in flight kept, and the rest wait for the next. Logged once: "TypeSafe asked Signals to slow down". |
|
||||
| anything else, or no answer | That item is left unjudged and asked again next pass. Logged: "Jev could not judge an item". |
|
||||
|
||||
The error keeps TypeSafe's reply, cut to 500 characters, for example:
|
||||
|
||||
Reference in New Issue
Block a user