The record

The record is what Datagoat learns from: past cases and what happened to them. Jev evaluates questions against state, the context of the case in front of it. Datagoat answers from a record, the history of cases like it.

A record has:

  • an entity column (entity_column) that names each case, such as customer_id
  • an outcome column that says, for each past case, whether the outcome happened: churned, converted, failed
  • the columns that might explain it: numbers, dates, counts and categories
customer_id  tenure_months  plan   support_tickets  logins_last_30d  churned
cust_0001    21             pro    12               11               yes
cust_0002    48             basic  1                64               no
…

A row whose outcome is known is a labeled row. Datagoat holds some labeled rows back and checks every pattern on them, so a record needs about 500 labeled rows or more. With fewer, the answer is not_yet, and needs.labeled_rows tells you how many more to add.

A record is usually a table with one row per case. When your data is an event log, a time series, a panel of periods, sensor readings or a log of agent runs, add a shape and the engine reads it into rows per case first.

Sending a record

data takes exactly one of:

data When Kept
{"rows": [...]} or {"csv": "..."} up to 10,000 rows, inline deleted when the call ends
{"fetch_url": "https://..."} a CSV Datagoat downloads (public https hosts only, up to 50 MB) deleted when the call ends
{"dataset_id": "ds_..."} a record of any shape, stored with dg_add_dataset 24 hours after last use
{"dataset_id": "sample:..."} a free sample always

For a large table, or one you'll ask about more than once, store it first: dg_add_dataset takes rows, CSV, a URL or a presigned upload, and appends pieces until the first ask. The Python SDK wraps both: dg.upload_rows(rows) and dg.upload_file(path). dg_delete_dataset deletes a stored dataset at once. See Your data for everything Datagoat keeps.

A fit is cached by the record's content, not its name. The same bytes sent again, inline or stored, with the same outcome, outcome_is_desirable and positive_values, reuse the fit.

The outcome column

Each question names its outcome_column. Datagoat works out which value means yes when the column has two values and one of them is a familiar no: 0/1, true/false, yes/no, y/n. Otherwise, pass positive_values with the values that mean yes:

{"type": "yesno", "outcome_column": "status", "positive_values": ["won"]}

Without positive_values, a column with one value, or with more than 12, is refused before any fit, with the reason.

Cases

cases names what you want answered:

cases Answers about
{"ids": ["cust_0001", …]} rows already in the record. With several rows per id, the latest wins: set time_column to say which is latest.
{"rows": [{…}, …]} new cases, not in the record. Each row needs the entity column and every column the model uses.

Up to 10,000 cases per call. A rank question can leave cases out to rank the whole record.

Who the cases are

subject_kind says what a case is: person, org, object, event or other. Questions about people are decision support: they need acknowledge_decision_support: true, and a person should make the decision (see Your data).

Checking a record first

dg_preflight looks at a stored table without fitting anything. It reports the grain (whether one row is one kind of thing) and what each column's values look like. If you name the outcome, it also checks whether there are enough positives for the number of usable columns. blocking entries mean an answer from this table would not be believable. A clean preflight doesn't promise a pattern exists; only an ask can find one.

Samples

Eight synthetic records, free on any key. dg_describe returns a ready-to-run ask for each.

dataset_id Shape Case Question it's built for
sample:saas_churn table customer_id Which accounts will churn?
sample:b2b_leads table lead_id Which leads will convert?
sample:telco_churn table account_id Which contract keeps an account?
sample:customer_events events customer_id Which customers have gone quiet?
sample:store_weekly series store_id Which stores run out of stock next week?
sample:usage_panel panel account_id Whose usage is declining?
sample:sensor_stream signals asset_id Which machines fault in the next three days?
sample:agent_traces traces run_id Which agent runs will fail?