DATAGOATDocsQuick startConnectKeysBilling

Prove it

You'll end with a record of calls anyone can check: each made before its outcome, each signed, and each graded against what happened. It is the question a model that reads text cannot answer for you, because it needs outcomes.

1. Keep every call, signed

Each answer carries verdicts: the chances and reasons, computed_at, expires_at, the record's dataset_content_hash, the engine's core_hash, and a signature. Datagoat keeps answers for 24 hours, so store each Verdict and its signature when you receive them.

Anyone can check one later, with or without Datagoat (Verdicts):

from datagoat import fetch_keys, verify_offline
keys = fetch_keys()                                  # the published Ed25519 keys; keep them
verify_offline(v["verdict"], v["signature"], keys)   # "valid" | "invalid_signature" | "expired" | "unknown_key"

Change one number in a stored Verdict and it no longer verifies.

2. Grade calls against what happened

A walk-forward replay is how you show a model would have called the future, without looking at it:

  1. Fit on the past only. Send only rows before a cutoff date; time_column orders them.
  2. Call each later period. Score that period's cases from the saved model_ref (no fit) and store the signed answer before looking at anything after it.
  3. Grade once, after the loop. Only now read the outcomes. For each period: how many of the flagged cases (the top band, or the top k of a rank) had the outcome, against the rate across all cases that period; and how far ahead the first call came.
  4. Say what it does not show. One dataset or many, which outcome, how cases were chosen, and that past calls do not promise future ones.

The grading is your own counting of outcomes against signed calls; say so when you show it. Datagoat's own check is the same idea: every answer's quality is measured on rows the model never saw, the most recent ones when the record has a time column (How answers are checked).

3. Did acting work?

  • Record the action. When you act on a case through one of its levers, dg_attest with the lever's lever_token exactly as given, the feature's new value, when, and an event_id (Recording an action).
  • Report the outcome with dg_report_outcomes (Run it).
  • Compare. dg_evidence compares cases acted on with cases not acted on, once each group has 30 outcomes (Did acting work?). Cases acted on were chosen, not randomised, so the difference is an association, not proof of cause.

Writing outcomes and actions needs the "Can report outcomes" capability.

The agent skill for this page: datagoat-prove.

Next

Worked examples, each with what it does not claim: Examples.