ModelsTraining

Jobs

Run training to create models.
View as Markdown

A training job runs a training configuration against a dataset.

It brings together a workspace, dataset, training configuration, compute configuration, and optional constraints. When the job completes, it creates the model that can be inspected, evaluated, and deployed.

Training jobs are asynchronous. Create a job, then retrieve it to monitor status, progress, current stage, and the model produced by the run.

Create a training job

Create a training job when the dataset, workspace, training configuration, and compute configuration are ready.

POST
/training-jobs
1curl -X POST https://api.hi.umnai.com/training-jobs \
2     -H "Authorization: Bearer <token>" \
3     -H "Content-Type: application/json" \
4     -d '{
5  "name": "Sample Training Job",
6  "dataset_id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
7  "training_configuration_id": "236f09a5d73018a9b8324105f1ee82df",
8  "compute_configuration_id": "351df95006b1350e597762a1d58e9319",
9  "model_name": "Sample Model Name",
10  "managed_spot_training": false
11}'
Response
1{
2  "id": "5647cb5d7800f6834be0f591838d5d9a",
3  "name": "Sample Training Job",
4  "dataset": {
5    "id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
6    "name": "Sample Dataset"
7  },
8  "account": {
9    "id": "e268443e43d93dab7ebef303bbe9642f",
10    "name": "Sample Account"
11  },
12  "workspace": {
13    "id": "1629dee48cc4e53161f9b2be8614e062",
14    "name": "Sample Workspace"
15  },
16  "training_configuration": {
17    "id": "236f09a5d73018a9b8324105f1ee82df",
18    "name": "Sample Training Configuration"
19  },
20  "compute_configuration": {
21    "id": "351df95006b1350e597762a1d58e9319",
22    "name": "Sample Compute Configuration"
23  },
24  "status": "STARTING",
25  "managed_spot_training": false,
26  "maximum_runtime_seconds": 3600,
27  "created_at": "2026-03-15T10:15:30.000000Z",
28  "model": {
29    "id": "20f35e630daf44dbfa4c3f68f5399d8c",
30    "name": "Sample Model"
31  },
32  "maximum_wait_time_seconds": 3600,
33  "model_constraints": [],
34  "model_constraint_groups": [],
35  "created_by": {
36    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
37    "name": "Sample User"
38  },
39  "updated_at": "2026-03-15T10:15:30.000000Z",
40  "stage": "sample_stage",
41  "progress": 0.5,
42  "stage_progress": 0.5
43}

The request names the model that will be created. It can also include individual model constraints, constraint groups, maximum runtime settings, and managed spot training.

Creating a training job requires an active workspace. Use the workspace id as the active workspace context when starting the job.

Managed spot training

managed_spot_training controls whether the job can use managed spot instances to reduce cost.

Checkpointing is not currently supported for managed spot training. If a spot instance is interrupted, training starts again from the beginning. This can lead to long waits or cause the job to time out, because spot availability and interruptions are unpredictable.

When managed spot training is enabled, maximum_wait_time_seconds controls how long the job may wait to start. maximum_runtime_seconds controls how long the job may run.

Monitor jobs

Retrieve jobs to monitor training progress or inspect previous runs.

GET
/training-jobs
1curl https://api.hi.umnai.com/training-jobs \
2     -H "Authorization: Bearer <token>"
Response
1{
2  "data": [
3    {
4      "id": "ea8b745ead39ec442fc3b4fba359bf98",
5      "name": "Sample Organisation",
6      "status": "ACTIVE",
7      "created_at": "2026-03-11T16:04:19.749000Z",
8      "updated_at": null
9    }
10  ],
11  "pagination": {
12    "total_items": 1,
13    "items_per_page": 10,
14    "current_page": 1,
15    "total_pages": 1
16  }
17}

To inspect a specific job, retrieve it by id.

GET
/training-jobs/:trainingJobId
1curl https://api.hi.umnai.com/training-jobs/{trainingJobId} \
2     -H "Authorization: Bearer <token>"
Response
1{
2  "data": [
3    {
4      "id": "ea8b745ead39ec442fc3b4fba359bf98",
5      "name": "Sample Organisation",
6      "status": "ACTIVE",
7      "created_at": "2026-03-11T16:04:19.749000Z",
8      "updated_at": null
9    }
10  ],
11  "pagination": {
12    "total_items": 1,
13    "items_per_page": 10,
14    "current_page": 1,
15    "total_pages": 1
16  }
17}

Job responses include the job status, overall progress, current stage, and stage progress.

StatusMeaning
STARTINGThe job has been created and is being prepared.
RUNNINGThe job is processing data.
FINALIZINGThe job is completing final processing steps.
COMPLETEDThe job finished successfully.
FAILEDThe job stopped because of an error.
STOPPEDThe job was stopped before completion.

Training stages

An induction training job can move through these stages:

  • Shard sampling
  • Interactions
  • Binning
  • Training
  • Build
  • ESM

Update a job

Update a training job when you need to rename it.

PATCH
/training-jobs/:trainingJobId
1curl -X PATCH https://api.hi.umnai.com/training-jobs/trainingJobId \
2     -H "Authorization: Bearer <token>" \
3     -H "Content-Type: application/json" \
4     -d '{
5  "name": "Sample Training Job"
6}'
Response
1{
2  "id": "5647cb5d7800f6834be0f591838d5d9a",
3  "name": "Sample Training Job",
4  "dataset": {
5    "id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
6    "name": "Sample Dataset"
7  },
8  "account": {
9    "id": "e268443e43d93dab7ebef303bbe9642f",
10    "name": "Sample Account"
11  },
12  "workspace": {
13    "id": "1629dee48cc4e53161f9b2be8614e062",
14    "name": "Sample Workspace"
15  },
16  "training_configuration": {
17    "id": "236f09a5d73018a9b8324105f1ee82df",
18    "name": "Sample Training Configuration"
19  },
20  "compute_configuration": {
21    "id": "351df95006b1350e597762a1d58e9319",
22    "name": "Sample Compute Configuration"
23  },
24  "status": "STARTING",
25  "managed_spot_training": false,
26  "maximum_runtime_seconds": 3600,
27  "created_at": "2026-03-15T10:15:30.000000Z",
28  "model": {
29    "id": "20f35e630daf44dbfa4c3f68f5399d8c",
30    "name": "Sample Model"
31  },
32  "maximum_wait_time_seconds": 3600,
33  "model_constraints": [],
34  "model_constraint_groups": [],
35  "created_by": {
36    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
37    "name": "Sample User"
38  },
39  "updated_at": "2026-03-15T10:15:30.000000Z",
40  "stage": "sample_stage",
41  "progress": 0.5,
42  "stage_progress": 0.5
43}

Delete a job

Delete a training job when it should no longer be kept.

DELETE
/training-jobs/:trainingJobId
1curl -X DELETE https://api.hi.umnai.com/training-jobs/trainingJobId \
2     -H "Authorization: Bearer <token>"