Real-time explanations | Umnai

Real-time explanations are produced when a query is run through an Explanation Structure Model (ESM).

The ESM returns both the prediction and the explanation of how that prediction was formed. This means applications can inspect, review, or act on explanation data at the same time as they receive the model output.

Results view output

Real-time explanation output is returned through the Results view.

The Results view can include:

Output	Description
Query input	The input data submitted for inference.
Prediction	The model output for the query.
Feature attributions	Feature-level contributions to the prediction.
Module attributions	Module-level contributions from features and feature interactions.
Decompositions	Feature-level components of interaction attributions.
Activations	Activated partitions and rules for the query.
Model and dataset identifiers	Identifiers that connect the result to the model and dataset used.
Timestamps	Timing information associated with the result.

The output can be customised to the needs of the application.

Why real-time explanations are possible

Real-time explanations are possible because of the structure of ESMs and Explainable Neural Nets (XNNs).

The model already contains the structures needed to explain its output: modules, partitions, rules, attributions, decompositions, and activations. As a result, the explanation does not require a separate post-hoc process or additional calculation pipeline.

Instead, the explanation can be read from the model together with the prediction.

From prediction to decision

Real-time explanations make it possible to apply decision workflows immediately after inference.

A prediction can be evaluated using its explanation data to identify, assess, and resolve issues or non-idealities before the output is used as a final decision.

For example, a workflow may inspect attributions, activations, strengths and weaknesses, or bias checks before deciding whether to accept the prediction, adjust it, or route it for human review.