Signals pick out far too much: each keyword is a subject on its own #259

Closed
opened 2026-10-16 11:01:57 -07:00 by rays · 1 comment
Owner

Reported 2026-10-16: the AI Development Signal (ai, development, programming) holds 54 items, many nothing to do with it: an Apple TV discount, Sega pulling a game remaster, a Surface Laptop review, Waymo in Canada. Jev is asked 'Is this item about any of these: ai, development, programming?', so any one broad word is enough, and the Signal's name, which says what is meant, is never sent. Scored against TypeSafe: Sega 0.90, Surface review 0.95, C++ futures 0.99 with the current question; 0.10, 0.26, 0.03 asked about 'AI Development' described by those words.

Reported 2026-10-16: the AI Development Signal (ai, development, programming) holds 54 items, many nothing to do with it: an Apple TV discount, Sega pulling a game remaster, a Surface Laptop review, Waymo in Canada. Jev is asked 'Is this item about any of these: ai, development, programming?', so any one broad word is enough, and the Signal's name, which says what is meant, is never sent. Scored against TypeSafe: Sega 0.90, Surface review 0.95, C++ futures 0.99 with the current question; 0.10, 0.26, 0.03 asked about 'AI Development' described by those words.
rays added the bug label 2026-10-16 11:01:57 -07:00
Author
Owner

Fixed in ce485a8, deployed: Jev is asked about one subject, the Signal's name described by its words and an optional sentence of its owner's, not about any one word. Every verdict was dropped on the first start, so both Signals are being judged again. Before deploying, the 54 items AI Development held were scored with the new question and the sentence 'AI tools for writing software: coding agents, AI dev tools, vibe coding, AI in code review.': 5 stay in.

Fixed in ce485a8, deployed: Jev is asked about one subject, the Signal's name described by its words and an optional sentence of its owner's, not about any one word. Every verdict was dropped on the first start, so both Signals are being judged again. Before deploying, the 54 items AI Development held were scored with the new question and the sentence 'AI tools for writing software: coding agents, AI dev tools, vibe coding, AI in code review.': 5 stay in.
rays closed this issue 2026-10-16 11:27:08 -07:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rays/ipx#259