Jobs | Umnai | Documentation

An ingestion job runs an ingestion configuration against source data in storage. Jobs are asynchronous. Create a job, then retrieve it to monitor status, progress, and the datasource produced by the run.

Create an initial job

An initial ingestion job creates a new datasource.

POST

/data-processing-ingestion-jobs

1 curl -X POST https://api.hi.umnai.com/data-processing-ingestion-jobs \
2      -H "Authorization: Bearer <token>" \
3      -H "Content-Type: application/json" \
4      -d '{
5   "job_type": "INITIAL",
6   "data_path": "/sample/initial/path",
7   "data_processing_configuration_id": "3dd973d3fda8604fba040381bfcaab29",
8   "datasource_name": "Sample Datasource",
9   "storage_connector_id": "02910f90b30a5b94ad505efe83280dca",
10   "name": "Sample Initial Ingestion Job",
11   "compute_configuration_id": "351df95006b1350e597762a1d58e9319"
12 }'

Response

1 {
2   "job_type": "INITIAL",
3   "data_path": "/sample/initial/path",
4   "data_processing_configuration": {
5     "id": "3dd973d3fda8604fba040381bfcaab29",
6     "name": "Sample Ingestion Configuration"
7   },
8   "storage_connector": {
9     "id": "02910f90b30a5b94ad505efe83280dca",
10     "name": "Sample Storage Connector"
11   },
12   "datasource": {
13     "id": "1ea7e575defdf6bc3f26a3f127e98170",
14     "name": "Sample Datasource"
15   },
16   "id": "c716d98913540a5968891bed979d177e",
17   "name": "Sample Initial Ingestion Job",
18   "account": {
19     "id": "e268443e43d93dab7ebef303bbe9642f",
20     "name": "Sample Account"
21   },
22   "compute_configuration": {
23     "id": "351df95006b1350e597762a1d58e9319",
24     "name": "Sample Compute Configuration"
25   },
26   "status": "STARTING",
27   "maximum_runtime_seconds": 3600,
28   "created_at": "2026-03-15T10:15:30.000000Z",
29   "created_by": {
30     "id": "dad46b2058752ffde5eaf02f1eb6abd5",
31     "name": "Sample User"
32   },
33   "updated_at": "2026-03-15T10:15:30.000000Z",
34   "stage": "initializing",
35   "progress": 0,
36   "stage_progress": 0
37 }

The job references the storage connector, data path, ingestion configuration, and processing compute configuration. It also names the datasource that will be created.

Create an incremental job

An incremental ingestion job adds new data to an existing datasource.

POST

/data-processing-ingestion-jobs

1 curl -X POST https://api.hi.umnai.com/data-processing-ingestion-jobs \
2      -H "Authorization: Bearer <token>" \
3      -H "Content-Type: application/json" \
4      -d '{
5   "job_type": "INCREMENT",
6   "data_path": "/sample/incremental/path",
7   "datasource_id": "1ea7e575defdf6bc3f26a3f127e98170",
8   "storage_connector_id": "02910f90b30a5b94ad505efe83280dca",
9   "increment_label": "March update",
10   "name": "Sample Incremental Ingestion Job",
11   "compute_configuration_id": "351df95006b1350e597762a1d58e9319"
12 }'

Response

1 {
2   "job_type": "INCREMENT",
3   "data_path": "/sample/incremental/path",
4   "storage_connector": {
5     "id": "02910f90b30a5b94ad505efe83280dca",
6     "name": "Sample Storage Connector"
7   },
8   "datasource": {
9     "id": "1ea7e575defdf6bc3f26a3f127e98170",
10     "name": "Sample Datasource"
11   },
12   "increment_label": "March update",
13   "id": "5e5d0e4d86b54765b8ab4e7c3b4c9478",
14   "name": "Sample Incremental Ingestion Job",
15   "account": {
16     "id": "e268443e43d93dab7ebef303bbe9642f",
17     "name": "Sample Account"
18   },
19   "compute_configuration": {
20     "id": "351df95006b1350e597762a1d58e9319",
21     "name": "Sample Compute Configuration"
22   },
23   "status": "STARTING",
24   "maximum_runtime_seconds": 3600,
25   "created_at": "2026-03-16T10:15:30.000000Z",
26   "created_by": {
27     "id": "dad46b2058752ffde5eaf02f1eb6abd5",
28     "name": "Sample User"
29   },
30   "updated_at": "2026-03-16T10:15:30.000000Z",
31   "stage": "initializing",
32   "progress": 0,
33   "stage_progress": 0
34 }

Incremental jobs point to the existing datasource id and can include an increment_label to identify the update.

Data paths

The data_path points to data inside the connected bucket.

It can refer to a single CSV or Parquet file, or to a directory containing one or more CSV or Parquet files. A directory must contain at least one supported file and must not mix CSV and Parquet files.

Monitor jobs

Retrieve jobs to monitor ingestion progress or inspect previous runs.

GET

/data-processing-ingestion-jobs

1 curl https://api.hi.umnai.com/data-processing-ingestion-jobs \
2      -H "Authorization: Bearer <token>"

Response

1 {
2   "data": [
3     {
4       "id": "ea8b745ead39ec442fc3b4fba359bf98",
5       "name": "Sample Organisation",
6       "status": "ACTIVE",
7       "created_at": "2026-03-11T16:04:19.749000Z",
8       "updated_at": null
9     }
10   ],
11   "pagination": {
12     "total_items": 1,
13     "items_per_page": 10,
14     "current_page": 1,
15     "total_pages": 1
16   }
17 }

To inspect a specific job, retrieve it by id.

GET

/data-processing-ingestion-jobs/:dataProcessingIngestionJobId

1 curl https://api.hi.umnai.com/data-processing-ingestion-jobs/{dataProcessingIngestionJobId} \
2      -H "Authorization: Bearer <token>"

Response

1 {
2   "data": [
3     {
4       "id": "ea8b745ead39ec442fc3b4fba359bf98",
5       "name": "Sample Organisation",
6       "status": "ACTIVE",
7       "created_at": "2026-03-11T16:04:19.749000Z",
8       "updated_at": null
9     }
10   ],
11   "pagination": {
12     "total_items": 1,
13     "items_per_page": 10,
14     "current_page": 1,
15     "total_pages": 1
16   }
17 }

Job responses include the job status, overall progress, current stage, and stage progress.

Status	Meaning
`STARTING`	The job has been created and is being prepared.
`RUNNING`	The job is processing data.
`FINALIZING`	The job is completing final processing steps.
`COMPLETED`	The job finished successfully.
`FAILED`	The job stopped because of an error.
`STOPPED`	The job was stopped before completion.

Update a job

Update an ingestion job when you need to rename it.

PATCH

/data-processing-ingestion-jobs/:dataProcessingIngestionJobId

1 curl -X PATCH https://api.hi.umnai.com/data-processing-ingestion-jobs/dataProcessingIngestionJobId \
2      -H "Authorization: Bearer <token>" \
3      -H "Content-Type: application/json" \
4      -d '{
5   "name": "New Sample Ingestion Job"
6 }'

Response

1 {
2   "job_type": "INITIAL",
3   "data_path": "/sample/path",
4   "data_processing_configuration": {
5     "id": "3dd973d3fda8604fba040381bfcaab29",
6     "name": "Sample Ingestion Configuration"
7   },
8   "storage_connector": {
9     "id": "02910f90b30a5b94ad505efe83280dca",
10     "name": "Sample Storage Connector"
11   },
12   "datasource": {
13     "id": "1ea7e575defdf6bc3f26a3f127e98170",
14     "name": "Sample Datasource"
15   },
16   "id": "c716d98913540a5968891bed979d177e",
17   "name": "New Sample Ingestion Job",
18   "account": {
19     "id": "e268443e43d93dab7ebef303bbe9642f",
20     "name": "Sample Account"
21   },
22   "compute_configuration": {
23     "id": "351df95006b1350e597762a1d58e9319",
24     "name": "Sample Compute Configuration"
25   },
26   "status": "STARTING",
27   "maximum_runtime_seconds": 1,
28   "created_at": "2026-03-15T10:15:30.000000Z",
29   "created_by": {
30     "id": "dad46b2058752ffde5eaf02f1eb6abd5",
31     "name": "Sample User"
32   },
33   "updated_at": "2026-03-15T10:15:30.000000Z",
34   "stage": "sample_stage",
35   "progress": 0.5,
36   "stage_progress": 0.5
37 }

Delete a job

Delete an ingestion job when it should no longer be kept.

DELETE

/data-processing-ingestion-jobs/:dataProcessingIngestionJobId

1 curl -X DELETE https://api.hi.umnai.com/data-processing-ingestion-jobs/dataProcessingIngestionJobId \
2      -H "Authorization: Bearer <token>"