DataOnboarding

Onboarding

Turn datasources into datasets for training.
View as Markdown

Onboarding is the stage that prepares a datasource for model training.

It applies the choices that make the data model-ready: the prediction type, target, selected features, preprocessing behaviour, sharding, and encoding settings. The result is a dataset that can be used by training workflows.

How onboarding works

1

Datasource

Onboarding starts from a datasource created by ingestion.

2

Configuration

The onboarding configuration defines how the datasource should be prepared for training.

3

Job

The onboarding job runs the configuration against the datasource using a processing compute configuration.

4

Dataset

The completed job creates or updates the dataset used for training.

Dataset onboarding flow

Dataset onboarding flow.

Initial and incremental onboarding

The first onboarding job for a datasource creates a dataset.

Later onboarding jobs can update that dataset from a newer datasource. Updates point to the existing dataset id and can include an increment_label to identify the new increment.