Groq retired the model and my health check still said ok

Ask AI is the live chat on dhseadev.online: a small model hosted on Groq, behind a Val Town proxy that holds the API key and enforces a daily budget. At some point before September 14 it stopped answering questions, and every check I had said it was fine.

What actually broke?

The model it called was retired. The proxy was written on August 6 against llama-3.1-8b-instant, and Groq’s deprecations page lists that model’s shutdown date as August 16, 2026, with openai/gpt-oss-20b as the recommended replacement. The model name in my code stopped pointing at anything. I cannot prove that from a log line, because nothing logged Groq’s reply, but swapping the model name is what brought the chat back.

I noticed on September 14. The widget said only “Could not reach the answers desk,” which is its fallback for a network failure.

Why did the health check say ok?

The proxy has a /health endpoint. On September 21, before the fix, it returned ok: true, a configured key, a clean budget store, and a model field reading llama-3.1-8b-instant. All of it was accurate, and none of it tested an answer.

/health checked that the parts were present. It never asked Groq anything.

The browser made it harder to see. The proxy is written to return a proper error when Groq refuses a request: a JSON error with the headers the page needs to read it. What actually reached the browser was a bare “error code: 502” with none of those headers, so the browser would not hand the response to the page at all, and the widget fell back to its connection-hiccup message. The real reason never made it to the screen.

How was it fixed?

Four edits to the Val Town proxy behind Ask AI, made on September 21:

• The model changed from llama-3.1-8b-instant to openai/gpt-oss-20b, the replacement Groq names.
• gpt-oss thinks before it answers, so reasoning effort is set to low, the reasoning stays out of the reply, and the output cap went from 400 to 700 tokens, since I could not find whether reasoning counts against that cap.
• The budget came down to 3 requests a minute, 800 a day and 150,000 tokens a day, each under the free plan’s limit for the new model.
• A failed Groq call now logs its status code and returns 503 instead of 502. Whether a 503 survives the trip to the browser intact is untested.

Then I checked it the way /health should have: one real question sent from the site’s own address, and a streamed answer coming back. The likely cause was known on September 14. The fix went in on the 21st.

What should you monitor on an LLM feature?

• Your provider’s deprecations page. Groq publishes the shutdown date and the replacement model. I was not reading it.
• A canary, not a status page. Send one real prompt on a schedule and fail loudly if no token comes back. /health still does not do this, and it is the next change.
• The upstream status code, written to a log. My proxy threw Groq’s answer away, so the diagnosis was inference rather than a log line.
• The model name in configuration. The proxy already read a MODEL environment variable. I had never set it, so the fix was a code edit instead of a setting.
• The error your visitor sees. Mine blamed the connection for a retired model.

Would I still use Groq?

Yes. My budget sits under the free plan’s limits, and the deprecation notice named a replacement, which is why the repair was a model string and a few limits rather than a rewrite. You need a Groq API key, and free-plan limits are listed per model, so check the rate-limits page for the one you pick. For openai/gpt-oss-20b on September 21 it read 30 requests a minute, 1,000 a day, 8,000 tokens a minute and 200,000 tokens a day.

The first post on this setup covers the proxy and the budget in detail: Putting an AI on the site: Val Town, Groq, and the budget that keeps it free.

Can I try the Ask AI page?

It is live at /ask/. Ask how something here was built, what a project does, or bring a decision you are stuck on. Answers are short by design and can be wrong, and the pages it points you to are the source of truth. If you would rather talk to a person, the contact form reaches me directly.

A related lesson from a different week, about a test suite that could not fail either: The suite that passed with the arrows unplugged.