Deployment

Overview

Make trained models available for inference.
View as Markdown

Deployment makes a trained model available for inference and explanations.

Start from a model created by training, choose a deployment configuration, then create a deployed model. Once the deployed model is running, it can be queried and inspected through deployed model views.

How deployment works

1

Model

Deployment starts from a trained model.

2

Configuration

A deployment configuration defines how the model should be served.

3

Deployed model

Deploying the model creates a deployed model resource that can be monitored and managed.

4

Inference and explanations

When the deployed model is running, it can be used for inference and explanation workflows.

Deployment types

The platform supports two deployment types.

TypeUse
SERVERLESSUse for on-demand workloads where scaling to zero is useful and cold starts are acceptable.
REALTIMEUse for low-latency workloads where the model should stay ready for requests.

Serverless configurations use memory size and maximum concurrency. Real-time configurations use a serving machine type and machine count.

Deployed model lifecycle

A deployed model is created from a model and a deployment configuration.

Use the deployed model resource to monitor deployment status, rename the deployment, delete it when it is no longer needed, and access deployed model views.