DATAGOATDocsQuick startConnectKeysBilling

Run it

You'll end with a scheduled job that keeps each customer's model honest: outcomes reported as they arrive, the model refitted on new data, and a drift check deciding whether to keep, replace or retire it. The fitting and scoring themselves are in Ship a product.

The weekly loop

Score, report, refit, check drift, and act on what it says; then again next week.

  1. Score each day's new cases from the model you trust: a question with model_ref, no fit (Score from the model).

  2. Report outcomes as you learn them, with an event_id on each so a retry is safe:

    {"model_ref": "mr1_…", "outcomes": [{"entity_id": "deal_0412", "outcome": 1, "observed_at": "2026-09-30", "event_id": "deal_0412:won"}]}

    Outcomes are evidence about the model's track record; they do not change the model or any answer.

  3. Refit on a schedule (weekly or monthly, when the record has new outcomes): ask with the new record and refit_of: "<the model_ref you are using>".

  4. Read dg_drift for the new model and act on it:

recommendation Do
keep dg_extend_model on the model you use, and carry on
refit switch to the new model_ref
abandon stop asking this question; the pattern no longer holds
no_check_yet nothing has been compared; refit first, and never renew on it

A single refusal on a refresh is not drift. More: Watching for drift.

Lifetimes

A model answers until model_expires_at (90 days by default, 1 to 365 with model_ttl_days on the ask that fits it). The lifetime is a privacy limit and a safety net; drift decides whether a model is good. Set it longer than your refit cycle. Details: Model lifetime.

A customer leaves

dg_delete_model removes the model and its outcome record at once. It needs the "Can report outcomes" capability, because it ends the outcome record too.

What to watch

Signal Where Worry when
refusals on refits the refit's answer state a question that answered starts refusing every time
model_expires_at every answer it is closer than your next refit
row_not_scoreable the daily job your data stopped sending a column in model_columns
dg_evidence Prove it acting stops helping

The agent skill for this page: datagoat-product.

Next

Show that the calls were right: Prove it.