The Model Vanished. The Alarm Stayed Silent for Two Weeks

Our Citation Tracker went dark for two weeks after Perplexity's sonar-pro model vanished from the AI gateway we use — the safety check correctly failed red twice, but nobody saw it. The fix: a current model, a test against the real model list, and a new check that verifies the outcome, not the job's colour.

Our Citation Tracker measures weekly whether ChatGPT, Claude, Gemini and Perplexity even mention a brand, and whether they cite that brand's own website as a source — the foundation of our KI-Sichtbarkeit (AI Visibility) product. For two weeks, this measurement tool produced no new data for a single customer. Nobody noticed. Not because the system failed, but because it did exactly what it was built to do — and because that alert never reached anyone. A run that delivered nothing The tracker runs every Monday for several customer tenants, including our NGO pilot, SOS Kinderdorf Schweiz. The last run that actually wrote data to the database was on September 28, 2026. The scheduled runs on October 5 and October 12 both failed completely, before a single question was ever sent to a language model. The cause was simple: the AI gateway we use had removed Perplexity's sonar-pro model from its offering. Our tracker configures every model through a fixed model ID — and on that Monday, that ID simply no longer existed. The failure was loud — nobody was listening That is not an accident; it is a deliberate design choice. Before every run, the tracker checks whether every configured model is actually available at the gateway, and aborts the entire run the moment one is missing — better to fail loudly than to quietly measure something wrong. That rule exists because of the opposite problem earlier in the summer: for weeks, Gemini returned answers that looked normal but were silently cut off mid-sentence at a 1024-token limit, because that provider counts its own reasoning steps against the same limit. The runs reported success while delivering almost nothing usable. That incident is where the rule that correctly fired on October 5 came from. Twice in a row, with a complete error log, the runs failed red. And twice in a row, nobody saw it. A scheduled run that fails does not generate a notification anyone actively reads — it simply leaves a red status in a list nobody routinely opens. Meanwhile, the public dashboard quietly kept showing the September 28 numbers, with no indication that nothing new had been measured since. How it surfaced The stall wasn't caught by monitoring — it surfaced through a customer question. A contact at our NGO pilot asked how to read a specific column on the dashboard, an entirely unrelated question. Checking the underlying …

Author
Nexbid Engineering
Published
2026-10-22
Topics
Research, Verification, Agentic