> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.umnai.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.umnai.com/_mcp/server. # Evaluate models Evaluate a [running deployment](/ui-guides/deployment/deployed-models) using the appropriate data population. Review the metrics, plots and class counts together when assessing the result. ## Before you begin Confirm that the intended deployment is **RUNNING**. Decide whether you are inspecting training performance or independent test performance. **Test** requires observations labelled as test data during ingestion; the training/validation split used during training does not supply that test population. Keep the deployment and data selection fixed when comparing thresholds. If you compare different models, record which dataset and population each result uses. ## Select evaluation inputs Select the running deployment you want to evaluate. The example uses **adult_income_deployment** with **Test** data. A deployed model is the serving resource associated with a trained model; confirm that it is the intended deployment before calculating results. ![Model Evaluation inputs for adult\_income\_deployment using Test data.](/_fern-files/_img/f665be008d8952a7514db3b4862a79024b2670b540d9d76f8448393ccee4f0bd.webp) ## Evaluate a deployed model Open **Model Lab > Model Evaluation**. Evaluation runs against a **Deployed model**. If your model has not been deployed, [create a deployment configuration](/ui-guides/deployment/configuration) and [deploy the trained model](/ui-guides/deployment/deployed-models), then return here. 1. Select the intended **Deployed model**. 2. Check the automatically detected **Model type**. The example is **Classification**; regression models use a different set of evaluation metrics. 3. Choose **Evaluation data**: **Test**, **Train** or **All**. The example uses Test. Test evaluation requires test observations; All includes training data and is not an independent test set. 4. For classification, review **Decision threshold step size**. The example uses **0.05**. This value sets the resolution of the returned threshold results; the UI accepts values from **0.005** to **0.1**. 5. Select **Calculate** and wait for **Evaluation complete**. 6. For classification, review the decision threshold, metrics, ROC and PR plots, and confusion matrix. For regression, review MAE, MSE, MAPE, R2 Score and Maximum Error. The results below belong to **adult_income_deployment**, not the payments fraud model inspected earlier. The selected **Decision threshold** is **0.5**. Use this control to compare the returned threshold results and assess the trade-off between false positives and false negatives for your task. | **Metric** | **Example value** | | ---------- | ----------------- | | Accuracy | 0.85 | | F1 Score | 0.65 | | Precision | 0.76 | | Recall | 0.57 | | Log Loss | 0.32 | | ROC AUC | 0.9045 | | PR AUC | 0.7769 | Enable **Extended metrics** to display additional classification measures. When comparing evaluations, retain the deployment, evaluation data selection, decision threshold and threshold step size so that results can be interpreted on the same basis. ### Read the example metrics After **Evaluation complete**, review the metrics together. The example below uses decision threshold **0.5** with **Extended metrics** off. The values describe this evaluation and are not acceptance criteria for other models. ![Decision threshold 0.5 and the displayed classification metrics.](/_fern-files/_img/bb08a572954df46b37ccb9ad0853c40259d22e428697ff84f99ca360fc4a8b34.webp) ## Read evaluation plots and counts The **Receiver Operating Characteristic (ROC) Plot** displays True Positive Rate against False Positive Rate. The **Precision Recall (PR) Plot** displays Precision against Recall. Both show AUC values and a reference baseline in the example. Read them alongside class balance and the consequences of classification errors. The confusion matrix labels actual classes by row and predicted classes by column. Read the labels before interpreting the counts. | **Confusion matrix result** | **Example count** | | --------------------------- | ----------------- | | True Positive | 1360 | | False Negative | 1011 | | False Positive | 438 | | True Negative | 6896 | | Total Samples | 9705 | | Positive Class | 2371 | | Negative Class | 7334 | ### Read the example plots Read the actual and predicted class labels when interpreting the matrix. These results belong to the adult income evaluation example. The example reports **Class Balance** as **24.4%**. Use the class counts, evaluation population and task requirements together when assessing performance. Choose an acceptable decision threshold for your use case and compare the resulting false-positive and false-negative counts. ![ROC and PR plots with the labeled confusion matrix for the evaluation example.](/_fern-files/_img/9e651520401d6935dac6dbfe40fd4ed075dde5ac5efab75ec30cd008c41ea909.webp) ## Interpret classification metrics | Metric | What it tells you | | --------- | ---------------------------------------------------------------------------------------------------------------- | | Accuracy | Fraction of correct classifications. Read it alongside class balance, especially when one class is rare. | | Precision | Of the predicted positive cases, the fraction that is actually positive. | | Recall | Of the actual positive cases, the fraction identified as positive. | | F1 Score | Combines precision and recall in a single measure. | | Log Loss | Measures the quality of predicted probabilities; lower values indicate less loss for the evaluated observations. | | ROC AUC | Summarises discrimination across thresholds in the ROC plot. | | PR AUC | Summarises the precision-recall trade-off across thresholds. | The threshold changes how probabilities become class decisions. Raising or lowering it changes the confusion matrix and threshold-dependent metrics. **Decision threshold step size** controls which threshold results are calculated; **Decision threshold** selects a returned result to inspect. ## Interpret regression metrics Regression results use numeric prediction errors rather than a classification threshold. | Metric | What it tells you | | ------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | MAE | Average absolute prediction error, in the target's units. | | MSE | Average squared error; large errors have greater influence and units are squared. | | MAPE | Error relative to the actual value. Interpret cautiously when actual values are zero or close to zero. | | R2 Score | Fit relative to predicting a constant mean. A negative value indicates worse performance than that baseline on the evaluated data. | | Maximum Error | Largest absolute error in the evaluated population. | Review aggregate error together with the task's acceptable error range. A good average does not show whether important subgroups or extreme values are poorly predicted. ## Next steps Continue to [Explanation views](/ui-guides/explanations/overview).