The record
The record is what Datagoat learns from: past cases and what happened to them. Jev evaluates questions against state, the context of the case in front of it. Datagoat answers from a record, the history of cases like it.
A record has:
- an entity column (
entity_column) that names each case, such ascustomer_id - an outcome column that says, for each past case, whether the outcome happened:
churned,converted,failed - the columns that might explain it: numbers, dates, counts and categories
customer_id tenure_months plan support_tickets logins_last_30d churned
cust_0001 21 pro 12 11 yes
cust_0002 48 basic 1 64 no
…
A row whose outcome is known is a labeled row. Datagoat holds some labeled rows back and
checks every pattern on them, so a record needs about 500 labeled rows or more. With fewer, the
answer is not_yet, and needs.labeled_rows tells you how many more to add.
A record is usually a table with one row per case. When your data is an event log, a time series, a panel of periods, sensor readings or a log of agent runs, add a shape and the engine reads it into rows per case first.
Sending a record
data takes exactly one of:
data |
When | Kept |
|---|---|---|
{"rows": [...]} or {"csv": "..."} |
up to 10,000 rows, inline | deleted when the call ends |
{"fetch_url": "https://..."} |
a CSV Datagoat downloads (public https hosts only, up to 50 MB) | deleted when the call ends |
{"dataset_id": "ds_..."} |
a record of any shape, stored with dg_add_dataset |
24 hours after last use |
{"dataset_id": "sample:..."} |
a free sample | always |
For a large table, or one you'll ask about more than once, store it first: dg_add_dataset takes
rows, CSV, a URL or a presigned upload, and appends pieces until the first ask. The Python SDK
wraps both: dg.upload_rows(rows) and dg.upload_file(path). dg_delete_dataset deletes a
stored dataset at once. See Your data for everything Datagoat keeps.
A fit is cached by the record's content, not its name. The same bytes sent again, inline or
stored, with the same outcome, outcome_is_desirable and positive_values, reuse the fit.
The outcome column
Each question names its outcome_column. Datagoat works out which value means yes when the
column has two values and one of them is a familiar no: 0/1, true/false, yes/no,
y/n. Otherwise, pass positive_values with the values that mean yes:
{"type": "yesno", "outcome_column": "status", "positive_values": ["won"]}
Without positive_values, a column with one value, or with more than 12, is refused before any
fit, with the reason.
Cases
cases names what you want answered:
cases |
Answers about |
|---|---|
{"ids": ["cust_0001", …]} |
rows already in the record. With several rows per id, the latest wins: set time_column to say which is latest. |
{"rows": [{…}, …]} |
new cases, not in the record. Each row needs the entity column and every column the model uses. |
Up to 10,000 cases per call. A rank question can leave cases out to rank the whole record.
Who the cases are
subject_kind says what a case is: person, org, object, event or other. Questions about
people are decision support: they need acknowledge_decision_support: true, and a person should
make the decision (see Your data).
Checking a record first
dg_preflight looks at a stored table without fitting anything. It reports the grain (whether
one row is one kind of thing) and what each column's values look like. If you name the outcome,
it also checks whether there are enough positives for the number of usable columns. blocking entries mean an answer
from this table would not be believable. A clean preflight doesn't promise a pattern exists; only
an ask can find one.
Samples
Eight synthetic records, free on any key. dg_describe returns a ready-to-run ask for each.
dataset_id |
Shape | Case | Question it's built for |
|---|---|---|---|
sample:saas_churn |
table | customer_id |
Which accounts will churn? |
sample:b2b_leads |
table | lead_id |
Which leads will convert? |
sample:telco_churn |
table | account_id |
Which contract keeps an account? |
sample:customer_events |
events | customer_id |
Which customers have gone quiet? |
sample:store_weekly |
series | store_id |
Which stores run out of stock next week? |
sample:usage_panel |
panel | account_id |
Whose usage is declining? |
sample:sensor_stream |
signals | asset_id |
Which machines fault in the next three days? |
sample:agent_traces |
traces | run_id |
Which agent runs will fail? |