Strengths and weaknesses | Umnai

Strengths and weaknesses, or SAW, describe confidence-related behaviour in an Umnai model.

SAW provides two measures: strength and uncertainty. Together, they help explain where the model is supported by strong evidence and where its behaviour may be less certain.

Strength

Strength reflects how strong the model is in the activated partitions.

It is based on:

Factor	Description
Training data support	The amount of training data in each activated partition.
Partition model fit	The quality of the partition’s polynomial model, or `then_expression`, fit.

A stronger partition is one where the model has more support from training data and a better local fit.

Uncertainty

Uncertainty reflects how certain the model is about its prediction behaviour.

It is based on:

Factor	Description
Estimator variance	Variation across estimators.
Partition prediction variance	Variation in the predictions produced by the partition’s polynomial model.

Higher uncertainty indicates that the model behaviour is less stable or less certain for the activated partitions.

Aggregation

Strength and uncertainty can be aggregated across activated partitions.

This allows SAW to produce confidence estimates at different levels:

Level	Description
Data segment	Confidence estimate for a selected segment of data.
Batch of predictions	Confidence estimate across a batch of prediction requests.
Dataset	Confidence estimate across the dataset as a whole.

At a global level, these aggregations help identify where the model is generally strong, where it is uncertain, and which areas may deserve further review.