Prepare data

Before you connect storage or create a schema, make the source data predictable enough for Umnai to read, validate, and prepare for modelling.

Good preparation reduces ingestion failures and helps avoid issues later in onboarding and training.

Prepare source files

Store the data you want to ingest in an Amazon S3 bucket that Umnai can access with read-only permissions.

Source data should be available as CSV or Parquet files. Files used in the same ingestion job should be structurally consistent: the same columns, compatible data types, and the same meaning for each field.

Review the columns

Before ingestion, decide how each column should be represented in the schema.

Column names

Use stable, API-safe column names.

Column names should start with a letter or number and use only letters, numbers, underscores, or hyphens. Avoid spaces, punctuation, duplicate names, and names that change between increments.

Features, targets, and ignored columns

Decide which columns are:

• features, used as model inputs;
• targets, used as the value to predict; and
• ignored, kept out of model training.

Each dataset used for training should have at least one feature and one target.

Ignore columns that identify a row but do not help the model generalise, such as transaction IDs, record IDs, UUIDs, or other values that are unique for each row.

Unsupported values

Keep each column focused on one kind of value. For example, do not combine IDs, labels, free-form text, and missing-value placeholders in the same field.

Check data quality

Clean or normalise values before ingestion when possible. The cleaner the source data, the less you need to compensate for it during onboarding.

Missing values

Decide how to handle missing values before onboarding.

For simple cases, clean or impute missing values before uploading the data. During onboarding, you can also enable imputation and configure how unseen categories are handled.

If missing values are meaningful in your domain, represent them consistently so they can be handled intentionally rather than accidentally.

Categorical values

Keep categorical values clean and intentional.

Check for accidental duplicates caused by whitespace, casing, or inconsistent spelling. For example, Approved, approved, and Approved may be interpreted as different values unless you clean or normalise them.

If a categorical column has many unique values, confirm that it is a real feature and not an identifier. High-cardinality identifiers should usually be ignored.

Targets

Choose a target that matches the prediction task.

For classification, each class should have enough examples for the model to learn from. If one class represents less than 10% of the data, consider whether the dataset needs more examples, a different split, or an onboarding configuration that handles imbalance more deliberately.

For regression, check for missing, invalid, or extreme target values before ingestion.

Plan for scale

Large datasets and wide schemas can require more processing resources and may produce more complex models.

As practical guidance:

• if the dataset has more columns than rows, consider collecting more rows or reducing the number of features;
• if the dataset has more than 256 features, review whether all features are useful for modelling; and
• if the dataset has more than 1 million rows, validate the workflow on a representative sample before running the full job.

For very large datasets, choose an appropriate processing compute configuration and expect ingestion and onboarding jobs to take longer.

Before ingestion

Before you create an ingestion configuration, you should have:

• source files in S3;
• read-only storage access for Umnai;
• a clear plan for features, targets, and ignored columns;
• a data schema or enough information to create one; and
• a processing compute configuration.