> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.umnai.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.umnai.com/_mcp/server.

# Encoding

Encoding controls how non-continuous values are represented during onboarding. Most datasets do not need manual encoding configuration. Define the field's semantic type in the schema, and let Umnai choose the encoding during processing. Set an encoding type only when you need a specific behaviour. In the web app, these schema fields are labelled **Semantic type** and **Encoding type**; in the REST API, they are `semantic_type` and `encoding_type`.

## Choose an encoding

Use encoding for fields that are not already continuous numerical values.

| Field shape                                 | Typical encoding          |
| ------------------------------------------- | ------------------------- |
| Low-cardinality categorical string feature  | `ONE_HOT`                 |
| High-cardinality categorical string feature | `TARGET`                  |
| Date or time feature                        | `TIMESTAMP`               |
| Continuous numeric feature                  | No encoding type required |
| Ignored field                               | No encoding type required |

The onboarding configuration also includes **One hot encoding max categories** (`one_hot_encoding_max_categories`), which controls when categorical fields switch from one-hot encoding to target encoding if encoding is inferred.

## Categorical encoding

Categorical features need to be represented numerically before they can be used for modelling.

### One-hot encoding

One-hot encoding represents each category as a separate binary feature.

For example, a `colour` field with values `red`, `blue`, and `green` becomes separate indicators for each category. A row with `colour = blue` activates the `blue` indicator and leaves the others inactive.

Use one-hot encoding when a categorical feature has a small, stable set of values.

### Target encoding

Target encoding is useful for categorical features with many distinct values.

Instead of creating one binary feature per category, Umnai represents each category using information derived from the target. This can reduce the number of generated features and make high-cardinality categorical fields easier to use.

Use target encoding when a categorical feature has many values and one-hot encoding would create too many features.

![Target encoding](/_fern-files/_img/6e41ee8c105fe24c99edc21dc3318c9d148817e2937fd67685687758e9fc638c.webp)

### Label encoding

Label encoding assigns each category an integer value.

> **Warning**
>
> `LABEL` encoding is exposed in the API, but it is not currently supported for general use. Use `ONE_HOT` or `TARGET` for categorical features unless you have confirmed that label encoding is supported for your workflow.

## Date and time encoding

Timestamp encoding expands a date or time field into derived temporal features, such as year, month, day, day of week, quarter, and related calendar indicators.

Use `TIMESTAMP` when a field contains time information that may be useful to the model.

Typical timestamp fields include:

| Source field    | Data type (`data_type`) | Data format (`format`) | Semantic type (`semantic_type`) | Encoding type (`encoding_type`) |
| --------------- | ----------------------- | ---------------------- | ------------------------------- | ------------------------------- |
| Datetime string | `STRING`                | `DATETIME`             | `SEQUENTIAL`                    | `TIMESTAMP`                     |
| POSIX timestamp | `NUMBER`                | `INT64`                | `SEQUENTIAL`                    | `TIMESTAMP`                     |

Timestamp-encoded fields are treated as feature groups during modelling and explanations.

## Text encodings

The API exposes `GLOVE` and `GPT` as encoding types.

> **Warning**
>
> Free-form natural language text is not currently supported for processing. Do not use `GLOVE` or `GPT` encoding unless you have confirmed that the workflow supports it. Transform text before ingestion, or mark text columns as ignored if they must remain in the source data.

## Encoding options during onboarding

Some encoding behaviour is controlled in the onboarding configuration rather than the schema.

| Option                          | API field                         | Use                                                                                                 |
| ------------------------------- | --------------------------------- | --------------------------------------------------------------------------------------------------- |
| One hot encoding max categories | `one_hot_encoding_max_categories` | Sets the threshold beyond which categorical fields switch from one-hot encoding to target encoding. |
| Handle unseen categories        | `handle_unseen_categories`        | Enables masking for categories not seen during training.                                            |
| Infrequency threshold           | `infrequency_threshold`           | Groups infrequent categories under an infrequent category representation.                           |
| Strip categories                | `strip_categories`                | Removes leading and trailing whitespace from categorical values.                                    |
| Encoding strategy               | `encoding_strategy`               | Controls how encoding is executed during onboarding.                                                |

For most workflows, keep the defaults unless you are addressing a specific data quality or performance issue.

### Encoding strategy

**Encoding strategy** (`encoding_strategy`) controls how onboarding performs encoding.

| Strategy | Use                                                                                            |
| -------- | ---------------------------------------------------------------------------------------------- |
| `STREAM` | Default strategy. Suitable for most datasets.                                                  |
| `SHARD`  | Loads and encodes data by shard. Use only when you have a reason to tune onboarding behaviour. |