Errors and cold start

What Grid returns when something is wrong, and what to do about it. The one you must handle is 503 with Retry-After.

Most of this page is the usual list. One entry is not, and it is the one to read first.

Cold start on Serverless#

Grid Serverless scales to zero. When a lane has had no traffic it releases its capacity, and the next request has to wait for it to come back.

Grid does not hold your request open while that happens. It answers:

代码

HTTP/1.1 503 Service Unavailable
Retry-After: 5

This is a documented state, not a fault in your code. The correct handling is to wait for the interval in Retry-After and retry, with backoff if it happens again.

We do not promise zero cold start, and you should not design as though there were one. What we do promise is that the wait is visible: a header you can read and act on, rather than a request that hangs until your own timeout fires and leaves you guessing why.

Python

import time
import openai

def complete(client, **kwargs):
    for attempt in range(5):
        try:
            return client.chat.completions.create(**kwargs)
        except openai.APIStatusError as e:
            if e.status_code != 503 or attempt == 4:
                raise
            # Prefer the server's interval; fall back to exponential backoff.
            wait = float(e.response.headers.get("retry-after") or 2 ** attempt)
            time.sleep(wait)

If you already retry on 503, you are done. If your client treats 503 as fatal, change that before you go to production.

Workloads that cannot tolerate a warm-up wait at all are the case for Grid Dedicated, where capacity is committed rather than elastic.

Status codes#

Code	Meaning	What to do
`400`	The request is malformed or a parameter is invalid.	Fix the request. Retrying will not help.
`401`	The API key is missing or not valid.	Check the `Authorization` header. See the note below on the body format.
`403`	The key is not permitted to do this.	Confirm the key has inference access, and that the model is in your catalog.
`404`	No such route, or no such model for this account.	Check the path, then list the catalog.
`429`	Rate limited, or a quota is exhausted.	Back off and retry. If it persists, it is a quota rather than a burst.
`402`	Billing. The account cannot currently be charged for this request.	Check billing status in the Console. Retrying will not help.
`503`	Cold start, or capacity is momentarily unavailable.	Retry per `Retry-After`. See above.
`5xx`	Something failed on our side.	Retry with backoff. If it persists, tell us.

Retry 429, 503 and 5xx. Do not retry 400, 401, 402 or 403: they describe a condition that will still be true on the second attempt.

Error bodies are not always JSON#

Authentication failures currently answer with content-type: text/plain and a short message, rather than the JSON {"error": {...}} envelope the OpenAI SDKs expect.

The status code is correct and the SDKs surface it correctly. What may not work is code that reads a parsed error body, because on a 401 there is no JSON object to parse.

Branch on the status code. If you need the message, read the raw response text rather than a decoded field.

We would rather tell you this than let you find it in production. It is a gap against strict OpenAI compatibility and it is on the list to close.

Response headers#

x-grid-routed-model names the model that produced the response. It is present on every call, routed or pinned.

Log it. When someone asks in three months why an answer looked different, the resolved model next to your own request id is the difference between an investigation and a guess.

Other headers exist on responses. They are not documented here yet because their names are not settled, and a header name in docs is a name we would owe you stability on.

Timeouts and streaming#

Grid supports streaming on chat completions. A streamed response can fail after the first token has arrived, which means a 200 is not a guarantee of a complete answer: handle a stream that ends early as its own case rather than assuming success from the status.

When it is not you#

Platform status is published at status.forlex.ai. Grid's own component is not wired into that page yet, so during the invite the fastest answer to "is it me or is it you" is to ask us directly.