The weekly AI skills study has produced a figure since early August. What it could not do was tell anyone how it was getting on. Its only outbound channel was an alarm that fired when something went wrong, and that channel had been broken since the day it was written, in a way that looked exactly like nothing being wrong. It now sends a short report after every run, whether the run went well or badly.
- The alarm mail had never sent once. The call that sent it passed the wrong argument names to the shared mail function, which raised a type error on every attempt. The surrounding handler caught it, wrote a single line to a log, and carried on. Two runs finished in a degraded state and the inbox stayed quiet, because a broken alarm and a healthy job produce exactly the same evidence: nothing. The line was sitting in the log the whole time, which is the part worth noticing. Nothing reads a log that nothing is asking you to read.
- The replacement sends on every run, which is what makes silence mean something. A channel that speaks only when there is a problem cannot report the problem of itself being broken. The report now goes out after a successful run as well as a failed one, so an empty inbox means the job did not run rather than the job ran and was fine. Every exit path leads to it, including a failure that escapes the wrappers meant to catch it, which was the one case the old design could never have reported: the code that would have raised the alarm sat downstream of whatever had gone wrong.
- The first version of the new report called a finished study broken. Collection stops on purpose once it has gathered enough postings to answer the question, and the report inferred that a run with no new collection had failed. So it opened every message with a problem that was not one. Reporting a normal condition as a fault every week is how a status report teaches its reader to skim it, which would have undone the reason for sending one at all. It now states completion as a fact, and still reports a missing collection run when the target has not been met.