DataSchema

Schema

Describe how Umnai should interpret source data during ingestion and onboarding.
View as Markdown

A data schema tells Umnai how to read the columns in your source data.

Create a schema before ingestion. Ingestion applies the schema to validate and prepare the source data. Onboarding then uses the same schema context when creating a dataset for training.

Define schema fields

A schema is made up of fields. Each field represents one source column and defines how that column should be used.

At minimum, each field needs:

FieldPurpose
nameThe source column name. It must match the column in your data.
data_typeThe physical type of the value, such as string, number, boolean, array, or any.
designationHow Umnai should use the column: FEATURE, TARGET, or IGNORED.

Most schemas also define:

FieldPurpose
semantic_typeThe modelling meaning of the column, such as CATEGORICAL, CONTINUOUS, or SEQUENTIAL.
encoding_typeOptional field-level encoding behaviour for categorical, sequential, or other non-continuous values.
vectorWhether the field contains vector data.

If semantic_type, encoding_type, or vector is omitted, Umnai may infer it during processing. Define these values explicitly when you need predictable behaviour.

Features, targets, and ignored columns

Use designation to decide how each column participates in modelling.

DesignationUse for
FEATUREInput columns the model can use to make predictions.
TARGETThe value the model should learn to predict.
IGNOREDColumns that should not be used for model training.

A dataset used for training should have at least one feature and one target.

Use IGNORED for row identifiers, record IDs, transaction IDs, UUIDs, or other columns that identify a row but do not help the model generalise.

Data types and formats

Use data_type to describe the value stored in each source column.

Data typeSupported formats
ANYNone
STRINGCATEGORY, DATETIME, JSON
NUMBERINT8, INT16, INT32, INT64, FLOAT32, FLOAT64
BOOLEANNone
ARRAYNone

Although ARRAY and STRING: JSON are exposed in the schema, complex nested values are not currently supported for processing. Avoid arrays and JSON objects as model features unless you have confirmed the workflow supports them.

Free-form natural language text is not currently supported for processing. Transform text before ingestion, or mark text columns as ignored if they must remain in the source data.

Encoding

Encoding controls how non-continuous values are represented during onboarding.

For most datasets, you only need to set the field’s semantic_type and let Umnai determine the encoding during processing. Define encoding_type explicitly when you need to control how a field is encoded.

Common mappings include:

Source fieldTypical schema options
Categorical string featuredata_type: STRING, semantic_type: CATEGORICAL, encoding_type: ONE_HOT or TARGET
Date or time string featuredata_type: STRING, format: DATETIME, semantic_type: SEQUENTIAL, encoding_type: TIMESTAMP
Numeric continuous featuredata_type: NUMBER, semantic_type: CONTINUOUS
Numeric targetdata_type: NUMBER, designation: TARGET

For the full set of encoding options and when to use them, see Encoding.

Infer a schema from storage

After creating a storage connector, you can retrieve an inferred schema from a path inside the connected bucket.

Use this as a starting point, then review and adjust the generated fields before creating the schema.

GET
/storage-connectors/:storageConnectorId/schema
1curl https://api.hi.umnai.com/storage-connectors/{storageConnectorId}/schema \
2     -H "Authorization: Bearer <token>"
Response
1{
2  "data": [
3    {
4      "id": "ea8b745ead39ec442fc3b4fba359bf98",
5      "name": "Sample Organisation",
6      "status": "ACTIVE",
7      "created_at": "2026-03-11T16:04:19.749000Z",
8      "updated_at": null
9    }
10  ],
11  "pagination": {
12    "total_items": 1,
13    "items_per_page": 10,
14    "current_page": 1,
15    "total_pages": 1
16  }
17}

Create a schema

Create a schema once you have reviewed the source columns and decided how each one should be used.

POST
/data-schemas
1curl -X POST https://api.hi.umnai.com/data-schemas \
2     -H "Authorization: Bearer <token>" \
3     -H "Content-Type: application/json" \
4     -d '{
5  "name": "Sample Schema",
6  "fields": [
7    {
8      "name": "age",
9      "data_type": {
10        "data_type": "NUMBER",
11        "format": "INT64"
12      },
13      "designation": "FEATURE",
14      "semantic_type": "CONTINUOUS"
15    },
16    {
17      "name": "occupation",
18      "data_type": {
19        "data_type": "STRING"
20      },
21      "designation": "FEATURE",
22      "semantic_type": "CATEGORICAL"
23    },
24    {
25      "name": "income",
26      "data_type": {
27        "data_type": "NUMBER",
28        "format": "INT64"
29      },
30      "designation": "TARGET",
31      "semantic_type": "CATEGORICAL"
32    }
33  ]
34}'
Response
1{
2  "id": "22f2c353201d472fbfc7c856eaecaccd",
3  "account": {
4    "id": "cf5b093e7f65480e98107feb78649222",
5    "name": "Sample Account"
6  },
7  "name": "Sample Schema",
8  "fields": [
9    {
10      "created_at": "2026-04-02T08:36:14.974937Z",
11      "data_type": {
12        "data_type": "NUMBER",
13        "format": "INT64"
14      },
15      "designation": "FEATURE",
16      "id": "37fd2c48429440ae981cb9831c66464b",
17      "name": "age",
18      "created_by": {
19        "id": "eca422469dc3270625bf646df3472a45",
20        "name": "Sample User"
21      },
22      "dimension": null,
23      "encoding_type": null,
24      "semantic_type": "CONTINUOUS",
25      "unit": null,
26      "updated_at": null,
27      "vector": false
28    },
29    {
30      "created_at": "2026-04-02T08:36:14.982037Z",
31      "data_type": {
32        "data_type": "STRING",
33        "format": null
34      },
35      "designation": "FEATURE",
36      "id": "95dc4f2aab9242e491b1f109778b0d10",
37      "name": "occuptation",
38      "created_by": {
39        "id": "eca422469dc3270625bf646df3472a45",
40        "name": "Sample User"
41      },
42      "dimension": null,
43      "encoding_type": null,
44      "semantic_type": "CATEGORICAL",
45      "unit": null,
46      "updated_at": null,
47      "vector": false
48    },
49    {
50      "created_at": "2026-04-02T08:36:15.063293Z",
51      "data_type": {
52        "data_type": "NUMBER",
53        "format": "INT64"
54      },
55      "designation": "TARGET",
56      "id": "058383ab0f6c4ce9a078b5f842ace676",
57      "name": "income",
58      "created_by": {
59        "id": "eca422469dc3270625bf646df3472a45",
60        "name": "Sample User"
61      },
62      "dimension": null,
63      "encoding_type": null,
64      "semantic_type": "CATEGORICAL",
65      "unit": null,
66      "updated_at": null,
67      "vector": false
68    }
69  ],
70  "friendly_names": [],
71  "category_friendly_names": [],
72  "data_anchors": [],
73  "data_protections": [],
74  "created_at": "2026-04-02T08:36:14.968725Z",
75  "created_by": {
76    "id": "eca422469dc3270625bf646df3472a45",
77    "name": "Sample User"
78  },
79  "updated_at": null
80}

Save the returned schema id. You will use it when creating an ingestion configuration.

Retrieve schemas

Retrieve schemas when you need to reuse an existing schema or inspect its fields.

GET
/data-schemas
1curl https://api.hi.umnai.com/data-schemas \
2     -H "Authorization: Bearer <token>"
Response
1{
2  "data": [
3    {
4      "id": "ea8b745ead39ec442fc3b4fba359bf98",
5      "name": "Sample Organisation",
6      "created_at": "2026-03-11T16:04:19.749000Z",
7      "updated_at": null,
8      "status": "ACTIVE"
9    }
10  ],
11  "pagination": {
12    "total_items": 1,
13    "items_per_page": 10,
14    "current_page": 1,
15    "total_pages": 1
16  }
17}

To inspect a specific schema, retrieve it by id.

GET
/data-schemas/:dataSchemaId
1curl https://api.hi.umnai.com/data-schemas/{dataSchemaId} \
2     -H "Authorization: Bearer <token>"
Response
1{
2  "data": [
3    {
4      "id": "ea8b745ead39ec442fc3b4fba359bf98",
5      "name": "Sample Organisation",
6      "status": "ACTIVE",
7      "created_at": "2026-03-11T16:04:19.749000Z",
8      "updated_at": null
9    }
10  ],
11  "pagination": {
12    "total_items": 1,
13    "items_per_page": 10,
14    "current_page": 1,
15    "total_pages": 1
16  }
17}

Update a schema

You can update an existing schema, including schema fields.

Be careful when changing a schema that has already been used for ingestion. Existing datasources keep the schema that was applied when they were ingested. Schema changes apply to future ingestion runs.

Delete a schema

Delete a schema when it should no longer be used.

DELETE
/data-schemas/:dataSchemaId
1curl -X DELETE https://api.hi.umnai.com/data-schemas/dataSchemaId \
2     -H "Authorization: Bearer <token>"

Advanced schema metadata

The API also exposes optional schema metadata.

MetadataWhat it stores
Dimensions and unitsMeasurement context for fields, such as time, currency, or other units.
Friendly namesHuman-readable labels for fields.
Category friendly namesHuman-readable labels for category values.
StakeholdersAudience or stakeholder context for friendly names.
Data anchorsFields, interactions, or values intended to act as stable reference points.
Data protectionsFields, interactions, or values intended to identify sensitive or protected characteristics.

Advanced schema metadata can be defined through the API, but it is not yet fully used by downstream platform workflows. Do not rely on dimensions, units, friendly names, category friendly names, stakeholders, data anchors, or data protections to affect ingestion, onboarding, training, or explanations until the functionality is complete.