> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.umnai.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.umnai.com/_mcp/server.

# Ingestion configuration

An ingestion configuration associates a data schema with observation metadata mappings. Creating the configuration saves these settings. An ingestion job uses them to import the data.

> **Note**
>
> Before configuring ingestion, review [Prepare data](/ui-guides/data/prepare-data) to check source file formats, column consistency and data quality. Confirm that the schema matches the prepared files.

## Before you begin

Create and review the [data schema](/ui-guides/data/schemas). Identify any source columns that already carry observation metadata, particularly the column distinguishing training and test rows.

## Create an ingestion configuration

1. Select **Data Lab > Datasources**, then select **Ingestion Configurations**.

2. Select **Create** to open **Create ingestion configuration**.

3. Enter a descriptive **Configuration name** and select the previously created **Data schema**.

4. Review **Observation specification**. This section maps source columns to observation metadata fields. Keep the prefilled names when they match your source data; otherwise enter the corresponding source column names.

5. Select **Create** to save the configuration. Confirm that it appears in **Ingestion Configurations** before selecting it in a job.

![Ingestion configurations and their schema references](/_fern-files/_img/14e4a3913cdefd837b07f7e74c2338c96d02ca39c7e307b52005b06845c319cf.webp)

## Map observation metadata

The form provides the following observation metadata mappings. If your source supplies observation types, check that training and test rows are mapped correctly. When no observation type is supplied, observations default to training data; test evaluation later in the workflow requires test observations.

| **Field**                 | **Prefilled column name**   |
| ------------------------- | --------------------------- |
| Observation type column   | observation_type            |
| Observation source column | observation_source          |
| Shard ID column           | observation_shard_id        |
| File index column         | observation_file_index      |
| Observation index column  | observation_index           |
| Mask index column         | observation_mask_index      |
| Mask file index column    | observation_mask_file_index |
| Increment index column    | observation_increment_index |

The configuration list displays **Configuration name**, **Data schema**, **Created at**, and **Updated at**, so you can check which schema a configuration references.

![Prefilled observation metadata mappings](/_fern-files/_img/74a3f4f7541392cec6bfd2b082ee566145c84bfcb59bfb63b2e8b74f65396fe3.webp)

## Choose observation mappings

The mappings identify source columns that already represent observation metadata. You do not need to add all eight metadata columns to an ordinary source file: the platform can add observation fields during ingestion. Change a mapping when your source uses a different name for that concept.

| Metadata                         | Purpose                                                                                                   |
| -------------------------------- | --------------------------------------------------------------------------------------------------------- |
| Observation type                 | Separates training and test observations. Preserve this distinction when you need independent evaluation. |
| Observation source               | Records source information associated with an observation.                                                |
| Shard ID                         | Identifies the observation's shard.                                                                       |
| File index and Observation index | Track the source file and observation within the data.                                                    |
| Mask index and Mask file index   | Identify masking-related observation and file information.                                                |
| Increment index                  | Tracks the data increment associated with an observation.                                                 |

## Review and reuse a configuration

Open the saved record and confirm the schema reference and observation mappings. Reuse it for source files with the same interpretation. The source bucket, path, machine and runtime belong to the ingestion job and are selected when that job is created.

## Next steps

Continue to [Ingestion jobs](/ui-guides/data/ingestion-jobs).