Jobs | Umnai | Documentation

An onboarding job runs an onboarding configuration against a datasource.

Jobs are asynchronous. Create a job, then retrieve it to monitor status, progress, current stage, and the dataset produced by the run.

Create an initial job

An initial onboarding job creates a new dataset.

POST

/data-processing-onboarding-jobs

curl -X POST https://api.hi.umnai.com/data-processing-onboarding-jobs \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{
  "job_type": "INITIAL",
  "compute_configuration_id": "351df95006b1350e597762a1d58e9319",
  "data_processing_configuration_id": "3dd973d3fda8604fba040381bfcaab29",
  "dataset_name": "Sample Dataset",
  "datasource_id": "1ea7e575defdf6bc3f26a3f127e98170",
  "name": "Sample Initial Onboarding Job"
}'

Response

{
  "job_type": "INITIAL",
  "account": {
    "id": "e268443e43d93dab7ebef303bbe9642f",
    "name": "Sample Account"
  },
  "compute_configuration": {
    "id": "351df95006b1350e597762a1d58e9319",
    "name": "Sample Compute Configuration"
  },
  "created_at": "2026-03-15T10:15:30.000000Z",
  "data_processing_configuration": {
    "id": "3dd973d3fda8604fba040381bfcaab29",
    "name": "Sample Onboarding Configuration"
  },
  "datasource": {
    "id": "1ea7e575defdf6bc3f26a3f127e98170",
    "name": "Sample Datasource"
  },
  "id": "7c2e73157a35658c245511bcf2a95195",
  "maximum_runtime_seconds": 3600,
  "name": "Sample Initial Onboarding Job",
  "status": "STARTING",
  "created_by": {
    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
    "name": "Sample User"
  },
  "dataset": {
    "id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
    "name": "Sample Dataset"
  },
  "progress": 0,
  "stage": "initializing",
  "stage_progress": 0,
  "updated_at": "2026-03-15T10:15:30.000000Z"
}

The job references the datasource, onboarding configuration, and processing compute configuration. It also names the dataset that will be created.

Create an incremental job

An incremental onboarding job adds new data to an existing dataset.

POST

/data-processing-onboarding-jobs

curl -X POST https://api.hi.umnai.com/data-processing-onboarding-jobs \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{
  "job_type": "INCREMENT",
  "compute_configuration_id": "351df95006b1350e597762a1d58e9319",
  "data_processing_configuration_id": "3dd973d3fda8604fba040381bfcaab29",
  "datasource_id": "3914cecdf57d2eade4435a94f82faf4e",
  "name": "Sample Incremental Onboarding Job",
  "dataset_id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
  "increment_label": "March update"
}'

Response

{
  "job_type": "INCREMENT",
  "account": {
    "id": "e268443e43d93dab7ebef303bbe9642f",
    "name": "Sample Account"
  },
  "compute_configuration": {
    "id": "351df95006b1350e597762a1d58e9319",
    "name": "Sample Compute Configuration"
  },
  "created_at": "2026-03-16T10:15:30.000000Z",
  "data_processing_configuration": {
    "id": "3dd973d3fda8604fba040381bfcaab29",
    "name": "Sample Onboarding Configuration"
  },
  "datasource": {
    "id": "3914cecdf57d2eade4435a94f82faf4e",
    "name": "Sample Incremental Datasource"
  },
  "id": "8b1130720f054149ac83bca083f8d216",
  "maximum_runtime_seconds": 3600,
  "name": "Sample Incremental Onboarding Job",
  "status": "STARTING",
  "created_by": {
    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
    "name": "Sample User"
  },
  "dataset": {
    "id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
    "name": "Sample Dataset"
  },
  "increment_label": "March update",
  "progress": 0,
  "stage": "initializing",
  "stage_progress": 0,
  "updated_at": "2026-03-16T10:15:30.000000Z"
}

Incremental jobs use a newer datasource and can include an increment_label to identify the update.

Monitor jobs

Retrieve jobs to monitor onboarding progress or inspect previous runs.

GET

/data-processing-onboarding-jobs

curl https://api.hi.umnai.com/data-processing-onboarding-jobs \
     -H "Authorization: Bearer <token>"

Response

{
  "data": [
    {
      "job_type": "INITIAL",
      "account": {
        "id": "e268443e43d93dab7ebef303bbe9642f",
        "name": "Sample Account"
      },
      "compute_configuration": {
        "id": "351df95006b1350e597762a1d58e9319",
        "name": "Sample Compute Configuration"
      },
      "created_at": "2026-03-15T10:15:30.000000Z",
      "data_processing_configuration": {
        "id": "3dd973d3fda8604fba040381bfcaab29",
        "name": "Sample Onboarding Configuration"
      },
      "datasource": {
        "id": "1ea7e575defdf6bc3f26a3f127e98170",
        "name": "Sample Datasource"
      },
      "id": "7c2e73157a35658c245511bcf2a95195",
      "maximum_runtime_seconds": 1,
      "name": "Sample Onboarding Job",
      "status": "STARTING",
      "created_by": {
        "id": "dad46b2058752ffde5eaf02f1eb6abd5",
        "name": "Sample User"
      },
      "dataset": {
        "id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
        "name": "Sample Dataset"
      },
      "progress": 0.5,
      "stage": "sample_stage",
      "stage_progress": 0.5,
      "updated_at": "2026-03-15T10:15:30.000000Z"
    }
  ],
  "pagination": {
    "total_items": 2,
    "items_per_page": 1,
    "current_page": 1,
    "total_pages": 1
  }
}

To inspect a specific job, retrieve it by id.

GET

/data-processing-onboarding-jobs/:dataProcessingOnboardingJobId

curl https://api.hi.umnai.com/data-processing-onboarding-jobs/dataProcessingOnboardingJobId \
     -H "Authorization: Bearer <token>"

Response

{
  "job_type": "INITIAL",
  "account": {
    "id": "e268443e43d93dab7ebef303bbe9642f",
    "name": "Sample Account"
  },
  "compute_configuration": {
    "id": "351df95006b1350e597762a1d58e9319",
    "name": "Sample Compute Configuration"
  },
  "created_at": "2026-03-15T10:15:30.000000Z",
  "data_processing_configuration": {
    "id": "3dd973d3fda8604fba040381bfcaab29",
    "name": "Sample Onboarding Configuration"
  },
  "datasource": {
    "id": "1ea7e575defdf6bc3f26a3f127e98170",
    "name": "Sample Datasource"
  },
  "id": "7c2e73157a35658c245511bcf2a95195",
  "maximum_runtime_seconds": 1,
  "name": "Sample Onboarding Job",
  "status": "STARTING",
  "created_by": {
    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
    "name": "Sample User"
  },
  "dataset": {
    "id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
    "name": "Sample Dataset"
  },
  "progress": 0.5,
  "stage": "sample_stage",
  "stage_progress": 0.5,
  "updated_at": "2026-03-15T10:15:30.000000Z"
}

Job responses include the job status, overall progress, current stage, and stage progress.

Status	Meaning
`STARTING`	The job has been created and is being prepared.
`RUNNING`	The job is processing data.
`FINALIZING`	The job is completing final processing steps.
`COMPLETED`	The job finished successfully.
`FAILED`	The job stopped because of an error.
`STOPPED`	The job was stopped before completion.

Initial job stages

An initial onboarding job can move through these stages:

Data validation
Masking
Analysis
Statistics
Preprocessing
Sharding
Transformation
Initialisation
Persistence
Teardown

Incremental job stages

An incremental onboarding job can move through these stages:

Masking
Analysis
Statistics
Preprocessing
Sharding
Transformation
Metadata
Persistence
Teardown

The masking stage is only used when unseen category handling is enabled.

Update a job

Update an onboarding job when you need to rename it.

PATCH

/data-processing-onboarding-jobs/:dataProcessingOnboardingJobId

curl -X PATCH https://api.hi.umnai.com/data-processing-onboarding-jobs/dataProcessingOnboardingJobId \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{
  "name": "New Sample Onboarding Job"
}'

Response

{
  "job_type": "INITIAL",
  "account": {
    "id": "e268443e43d93dab7ebef303bbe9642f",
    "name": "Sample Account"
  },
  "compute_configuration": {
    "id": "351df95006b1350e597762a1d58e9319",
    "name": "Sample Compute Configuration"
  },
  "created_at": "2026-03-15T10:15:30.000000Z",
  "data_processing_configuration": {
    "id": "3dd973d3fda8604fba040381bfcaab29",
    "name": "Sample Onboarding Configuration"
  },
  "datasource": {
    "id": "1ea7e575defdf6bc3f26a3f127e98170",
    "name": "Sample Datasource"
  },
  "id": "7c2e73157a35658c245511bcf2a95195",
  "maximum_runtime_seconds": 1,
  "name": "New Sample Onboarding Job",
  "status": "STARTING",
  "created_by": {
    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
    "name": "Sample User"
  },
  "dataset": {
    "id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
    "name": "Sample Dataset"
  },
  "progress": 0.5,
  "stage": "sample_stage",
  "stage_progress": 0.5,
  "updated_at": "2026-03-15T10:15:30.000000Z"
}

Delete a job

Delete an onboarding job when it should no longer be kept.

DELETE

/data-processing-onboarding-jobs/:dataProcessingOnboardingJobId

curl -X DELETE https://api.hi.umnai.com/data-processing-onboarding-jobs/dataProcessingOnboardingJobId \
     -H "Authorization: Bearer <token>"