Jobs

Run training to create models.
View as Markdown

A training job runs a training configuration against a dataset.

It brings together a workspace, dataset, training configuration, compute configuration, and optional constraints. When the job completes, it creates the model that can be inspected, evaluated, and deployed.

Training jobs are asynchronous. Create a job, then retrieve it to monitor status, progress, current stage, and the model produced by the run.

Create a training job

Create a training job when the dataset, workspace, training configuration, and compute configuration are ready.

POST
/training-jobs
curl -X POST https://api.hi.umnai.com/training-jobs \
-H "X-Active-Workspace: X-Active-Workspace" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"name": "Sample Training Job",
"dataset_id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
"training_configuration_id": "236f09a5d73018a9b8324105f1ee82df",
"compute_configuration_id": "351df95006b1350e597762a1d58e9319",
"model_name": "Sample Model Name",
"managed_spot_training": false
}'
Response
{
"id": "5647cb5d7800f6834be0f591838d5d9a",
"name": "Sample Training Job",
"dataset": {
"id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
"name": "Sample Dataset"
},
"account": {
"id": "e268443e43d93dab7ebef303bbe9642f",
"name": "Sample Account"
},
"workspace": {
"id": "1629dee48cc4e53161f9b2be8614e062",
"name": "Sample Workspace"
},
"training_configuration": {
"id": "236f09a5d73018a9b8324105f1ee82df",
"name": "Sample Training Configuration"
},
"compute_configuration": {
"id": "351df95006b1350e597762a1d58e9319",
"name": "Sample Compute Configuration"
},
"status": "STARTING",
"managed_spot_training": false,
"maximum_runtime_seconds": 3600,
"created_at": "2026-03-15T10:15:30.000000Z",
"model": {
"id": "20f35e630daf44dbfa4c3f68f5399d8c",
"name": "Sample Model"
},
"maximum_wait_time_seconds": 3600,
"model_constraints": [],
"model_constraint_groups": [],
"created_by": {
"id": "dad46b2058752ffde5eaf02f1eb6abd5",
"name": "Sample User"
},
"updated_at": "2026-03-15T10:15:30.000000Z",
"stage": "sample_stage",
"progress": 0.5,
"stage_progress": 0.5
}

The request names the model that will be created. It can also include individual model constraints, constraint groups, maximum runtime settings, and managed spot training.

Creating a training job requires an active workspace. Use the workspace id as the active workspace context when starting the job.

Managed spot training

managed_spot_training controls whether the job can use managed spot instances to reduce cost.

Checkpointing is not currently supported for managed spot training. If a spot instance is interrupted, training starts again from the beginning. This can lead to long waits or cause the job to time out, because spot availability and interruptions are unpredictable.

When managed spot training is enabled, maximum_wait_time_seconds controls how long the job may wait to start. maximum_runtime_seconds controls how long the job may run.

Monitor jobs

Retrieve jobs to monitor training progress or inspect previous runs.

GET
/training-jobs
curl https://api.hi.umnai.com/training-jobs \
-H "Authorization: Bearer <token>"
Response
{
"data": [
{
"id": "5647cb5d7800f6834be0f591838d5d9a",
"name": "Sample Training Job",
"dataset": {
"id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
"name": "Sample Dataset"
},
"account": {
"id": "e268443e43d93dab7ebef303bbe9642f",
"name": "Sample Account"
},
"workspace": {
"id": "1629dee48cc4e53161f9b2be8614e062",
"name": "Sample Workspace"
},
"training_configuration": {
"id": "236f09a5d73018a9b8324105f1ee82df",
"name": "Sample Training Configuration"
},
"compute_configuration": {
"id": "351df95006b1350e597762a1d58e9319",
"name": "Sample Compute Configuration"
},
"status": "STARTING",
"managed_spot_training": false,
"maximum_runtime_seconds": 3600,
"created_at": "2026-03-15T10:15:30.000000Z",
"model": {
"id": "20f35e630daf44dbfa4c3f68f5399d8c",
"name": "Sample Model"
},
"maximum_wait_time_seconds": 3600,
"model_constraints": [],
"model_constraint_groups": [],
"created_by": {
"id": "dad46b2058752ffde5eaf02f1eb6abd5",
"name": "Sample User"
},
"updated_at": "2026-03-15T10:15:30.000000Z",
"stage": "sample_stage",
"progress": 0.5,
"stage_progress": 0.5
}
],
"pagination": {
"total_items": 2,
"items_per_page": 1,
"current_page": 1,
"total_pages": 1
}
}

To inspect a specific job, retrieve it by id.

GET
/training-jobs/:trainingJobId
curl https://api.hi.umnai.com/training-jobs/trainingJobId \
-H "Authorization: Bearer <token>"
Response
{
"id": "5647cb5d7800f6834be0f591838d5d9a",
"name": "Sample Training Job",
"dataset": {
"id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
"name": "Sample Dataset"
},
"account": {
"id": "e268443e43d93dab7ebef303bbe9642f",
"name": "Sample Account"
},
"workspace": {
"id": "1629dee48cc4e53161f9b2be8614e062",
"name": "Sample Workspace"
},
"training_configuration": {
"id": "236f09a5d73018a9b8324105f1ee82df",
"name": "Sample Training Configuration"
},
"compute_configuration": {
"id": "351df95006b1350e597762a1d58e9319",
"name": "Sample Compute Configuration"
},
"status": "STARTING",
"managed_spot_training": false,
"maximum_runtime_seconds": 3600,
"created_at": "2026-03-15T10:15:30.000000Z",
"model": {
"id": "20f35e630daf44dbfa4c3f68f5399d8c",
"name": "Sample Model"
},
"maximum_wait_time_seconds": 3600,
"model_constraints": [],
"model_constraint_groups": [],
"created_by": {
"id": "dad46b2058752ffde5eaf02f1eb6abd5",
"name": "Sample User"
},
"updated_at": "2026-03-15T10:15:30.000000Z",
"stage": "sample_stage",
"progress": 0.5,
"stage_progress": 0.5
}

Job responses include the job status, overall progress, current stage, and stage progress.

StatusMeaning
STARTINGThe job has been created and is being prepared.
RUNNINGThe job is processing data.
FINALIZINGThe job is completing final processing steps.
COMPLETEDThe job finished successfully.
FAILEDThe job stopped because of an error.
STOPPEDThe job was stopped before completion.

Training stages

An induction training job can move through these stages:

  • Shard sampling
  • Interactions
  • Binning
  • Training
  • Build
  • ESM

Update a job

Update a training job when you need to rename it.

PATCH
/training-jobs/:trainingJobId
curl -X PATCH https://api.hi.umnai.com/training-jobs/trainingJobId \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"name": "Sample Training Job"
}'
Response
{
"id": "5647cb5d7800f6834be0f591838d5d9a",
"name": "Sample Training Job",
"dataset": {
"id": "3c4d09e4ef50b370ae0efacdb43ec2dd",
"name": "Sample Dataset"
},
"account": {
"id": "e268443e43d93dab7ebef303bbe9642f",
"name": "Sample Account"
},
"workspace": {
"id": "1629dee48cc4e53161f9b2be8614e062",
"name": "Sample Workspace"
},
"training_configuration": {
"id": "236f09a5d73018a9b8324105f1ee82df",
"name": "Sample Training Configuration"
},
"compute_configuration": {
"id": "351df95006b1350e597762a1d58e9319",
"name": "Sample Compute Configuration"
},
"status": "STARTING",
"managed_spot_training": false,
"maximum_runtime_seconds": 3600,
"created_at": "2026-03-15T10:15:30.000000Z",
"model": {
"id": "20f35e630daf44dbfa4c3f68f5399d8c",
"name": "Sample Model"
},
"maximum_wait_time_seconds": 3600,
"model_constraints": [],
"model_constraint_groups": [],
"created_by": {
"id": "dad46b2058752ffde5eaf02f1eb6abd5",
"name": "Sample User"
},
"updated_at": "2026-03-15T10:15:30.000000Z",
"stage": "sample_stage",
"progress": 0.5,
"stage_progress": 0.5
}

Delete a job

Delete a training job when it should no longer be kept.

DELETE
/training-jobs/:trainingJobId
curl -X DELETE https://api.hi.umnai.com/training-jobs/trainingJobId \
-H "Authorization: Bearer <token>"