DisTreebution.UQ package

Submodules

DisTreebution.UQ.Conformalisation module

DisTreebution.UQ.Conformalisation.get_quantile(ls, n, q)

class DisTreebution.UQ.Conformalisation.Conformalisation(settings=None, params=None)

Bases: object

Handles conformalisation processes on tree-based models, including preprocessing of trees for training and test data, optionally with group information, and performing operations on leaf values.

Parameters:

settings dict

(optional) Configuration settings for the conformalisation process, such as the type of tree.

params dict

(optional) Additional parameters for the conformalisation process.

Methods

preprocess_trees(trees, x_train, x_test)

Preprocesses the given trees to extract and organize leaf values for each sample in the training and test sets.

Handles both standard and PQRT tree types.

preprocess_trees_with_groups(trees, x_train, x_test, max_depth_group)

Preprocesses the given trees to extract leaf values and group information for each sample in the training and test sets,

up to a specified maximum group depth.

operation_leaf(ls, q, weights=None)

Computes the q-th quantile value from a sorted list of leaf values, optionally using sample weights.

preprocess_trees(trees, x)

Preprocesses the given trees to extract and organize leaf values for each sample in the dataset. Handles both standard and PQRT tree types.

Parameters:

trees list

List of tree objects to preprocess.

x array-like

Input data for which to extract leaf values.

Returns dict:

A dictionary mapping tree IDs to another dictionary that maps sample IDs to sorted leaf values. Format: {tree_id: {sample_id: sorted_leaf_values}}

preprocess_trees_with_groups(trees, x_train, max_depth_group)

Preprocesses the given trees to extract leaf values and group information for each sample in the training set, up to a specified maximum group depth.

Parameters:

trees list

List of tree objects to preprocess.

x_train array-like

Training input data for which to extract leaf values and group information.

max_depth_group int

Maximum depth for group coverage in the trees.

Returns tuple:

• treeID2trainID2values (dict): A dictionary mapping tree IDs to another dictionary

that maps training sample IDs to sorted leaf values.

• treeID2trainID2group (dict): A dictionary mapping tree IDs to another dictionary

that maps training sample IDs to their group information.

operation_leaf(ls, q, weights=None)

Computes the q-th quantile value from a sorted list of leaf values, optionally using sample weights.

Parameters:

ls array-like

Sorted list or array of leaf values.

q float

Quantile to compute, between 0 and 1.

weights array-like

Weights for each value in ls. If None, uniform weights are used.

Returns float:

The q-th quantile value from the list.

DisTreebution.UQ.Conformalisation_CQR module

class DisTreebution.UQ.Conformalisation_CQR.Conformalisation_CQR(settings=None, params=None)

Bases: Conformalisation

Conformalized Quantile Regression (CQR) for tree-based models.

Implements CQR-style conformalization on tree ensembles producing quantile predictions. Extends Conformalisation with methods to compute global and group-wise conformity thresholds and to predict conformalized quantile sets on new inputs.

conformalize_split(trees, x_calib, y_calib, alpha, nominal_quantiles=None)

Compute CQR conformity thresholds for nominal quantiles.

The method computes conformity scores for each provided nominal quantile and sets self.conf_thresh to a list of thresholds corresponding to each nominal quantile.

Parameters:

trees list

List of trained tree models.

x_calib numpy.ndarray

Calibration input features.

y_calib numpy.ndarray

Calibration target values.

alpha float

Significance level for conformalization (e.g. 0.1 for 90% sets).

nominal_quantiles float or list of float

Nominal quantile(s) to conformalize. A single float will be converted to a list.

Returns:

None. Sets self.conf_thresh to a list of thresholds.

Return type:

None

Raises:

AssertionError – If nominal_quantiles is not provided.

predict_conformal_set_split(trees, x_test)

Predict conformalized quantile sets on test inputs.

Uses self.conf_thresh computed by conformalize_split() to produce, for each nominal quantile, a dictionary mapping test sample indices to conformal intervals [low, up].

Parameters:

trees list

List of trained tree models.

x_test numpy.ndarray

Test input features.

Returns:

Mapping from nominal quantile index to a mapping of test index to [low, up].

Return type:

dict

Raises:

AssertionError – If self.conf_thresh has not been computed.

conformalize_split_group_coverage(trees, x_calib, y_calib, alpha, nominal_quantiles=None, max_depth_group=None)

Compute group-wise CQR thresholds to guarantee group coverage.

For each nominal quantile and for each group defined by tree partitions up to max_depth_group, compute an empirical threshold stored in self.group2conf_thresh.

Parameters:

trees list

List of trained tree models.

x_calib numpy.ndarray

Calibration input features.

y_calib numpy.ndarray

Calibration target values.

alpha float

Significance level for conformalization.

nominal_quantiles float or list of float

Nominal quantile(s) to conformalize.

max_depth_group int

Maximum depth used to define groups in the tree partitioning.

Returns:

None. Sets self.group2conf_thresh.

Return type:

None

Raises:

AssertionError – If nominal_quantiles or max_depth_group is not provided.

predict_conformal_set_split_group_coverage(trees, x_test)

Predict group-aware conformalized quantile sets for test inputs.

Uses thresholds in self.group2conf_thresh to produce per-group conformal intervals for each nominal quantile.

Parameters:

trees list

List of trained tree models.

x_test numpy.ndarray

Test input features.

Returns:

Tuple (treeID2testID2group[0], sample2predset) where sample2predset maps nominal-quantile indices to per-sample [low, up].

Return type:

tuple

get_low_up_test_i(yj, qhat_low, qhat_up, qhat_low_test, qhat_up_test, qmedian, qmedian_test, t=None)

Compute conformalized lower and upper quantiles for one test sample.

Behavior depends on self.settings['nested_set'] and supports variants ‘CQR’, ‘CQR-m’, and ‘CQR-r’. If t is not provided the method searches for the smallest adjustment making the calibration value yj fall inside the adjusted interval.

Parameters:

yj float

Calibration output value.

qhat_low float

Predicted lower quantile on calibration data.

qhat_up float

Predicted upper quantile on calibration data.

qhat_low_test float

Predicted lower quantile for the test sample.

qhat_up_test float

Predicted upper quantile for the test sample.

qmedian float

Predicted median on calibration data.

qmedian_test float

Predicted median for the test sample.

t float or None, optional

Optional conformalization adjustment parameter; if None it will be computed.

Returns:

(low_test, up_test, t) where low_test and up_test are the adjusted bounds and t is the used threshold.

Return type:

tuple(float, float, float)

get_low_up_score(i, q, y_train, sample2calib_trees, treeID2trainID2values_low, treeID2testID2values_low, treeID2trainID2values_up, treeID2testID2values_up, treeID2trainID2values_median, treeID2testID2values_median, t_fixed=None)

Compute conformalized quantile predictions and conformity scores.

Aggregates tree-level quantile estimates according to self.settings['type_aggregation_trees'] and for each calibration sample returns a conformalized lower bound, upper bound and a conformity score index.

Parameters:

i int

Index of the test sample.

q float

Nominal quantile level (e.g. 0.1).

y_train list or numpy.ndarray

Calibration output values.

sample2calib_trees dict

Mapping from calibration sample index to list of tree IDs.

treeID2trainID2values_low dict

Mapping of tree ID to training sample IDs to lower quantile leaf values.

treeID2testID2values_low dict

Mapping of tree ID to test sample IDs to lower quantile leaf values.

treeID2trainID2values_up dict

Mapping of tree ID to training sample IDs to upper quantile leaf values.

treeID2testID2values_up dict

Mapping of tree ID to test sample IDs to upper quantile leaf values.

treeID2trainID2values_median dict

Mapping of tree ID to training sample IDs to median quantile leaf values.

treeID2testID2values_median dict

Mapping of tree ID to test sample IDs to median quantile leaf values.

t_fixed float or None, optional

Optional fixed conformalization threshold/index. If None the method will compute it per calibration sample.

Returns:

Tuple (lower, upper, conf_scores) where each is a list with one entry per calibration sample in y_train.

Return type:

tuple(list, list, list)

DisTreebution.UQ.Conformalisation_distributional module

class DisTreebution.UQ.Conformalisation_distributional.Conformalisation_distributional(settings=None, params=None)

Bases: Conformalisation

Conformalisation for distributional tree-based models.

This class implements conformalisation utilities for tree ensembles that produce distributional (quantile/interval) predictions. It extends Conformalisation and provides methods to compute conformity thresholds on a calibration set and to produce conformal prediction sets for test inputs.

Parameters:

settings dict, optional

Configuration settings for the conformalisation process, such as the type of tree.

params dict, optional

Hyperparameters for the conformalisation process.

conformalize_split(trees, x_calib, y_calib, alpha)

Compute conformity threshold from a calibration set.

The method computes conformity scores on the provided calibration dataset and sets self.conf_thresh to the empirical quantile corresponding to alpha.

Parameters:

trees list

List of trained tree models.

x_calib numpy.ndarray

Calibration input data.

y_calib numpy.ndarray

Calibration output data.

alpha float

Significance level for conformalization (e.g. 0.1 for 90% sets).

Returns:

None. Sets self.conf_thresh on success.

Return type:

None

predict_conformal_set_split(trees, x_test)

Predict conformal prediction sets for test inputs using a fixed threshold.

Parameters:

trees list

List of trained tree models.

x_test numpy.ndarray

Test input data.

Returns:

Mapping from test sample index to conformal prediction set [low, up].

Return type:

dict

Raises:

AssertionError – If self.conf_thresh has not been set by a previous calibration.

conformalize_split_group_coverage(trees, x_calib, y_calib, alpha, max_depth_group=None, get_res_on_calib=False)

Compute group-specific conformity thresholds from a calibration set.

For each group (as defined by the tree partitioning up to max_depth_group) this method computes an empirical conformity threshold and stores the results in self.group2conf_thresh.

Parameters:

trees list

List of trained tree models.

x_calib numpy.ndarray

Calibration input data.

y_calib numpy.ndarray

Calibration output data.

alpha float

Significance level for conformalization.

max_depth_group int, optional

Maximum depth for defining groups (tree levels).

get_res_on_calib bool, optional

If True, also return calibration set statistics.

Returns:

None. On success sets self.group2conf_thresh. If get_res_on_calib is True the method additionally computes and would return calibration group sizes and coverages (the caller should call the corresponding predict method to retrieve them).

Return type:

None

predict_conformal_set_split_group_coverage(trees, x_test, get_res_on_calib=False, x_calib=None, y_calib=None)

Predict conformal sets for test inputs with group-wise thresholds.

Uses self.group2conf_thresh (computed by conformalize_split_group_coverage()) to produce conformal sets for each test sample, adapting the threshold to the group that a sample belongs to.

Parameters:

trees list

List of trained tree models.

x_test numpy.ndarray

Test input data.

get_res_on_calib bool, optional

If True, also compute and return calibration-set statistics.

x_calib numpy.ndarray, optional

Calibration input data (required when get_res_on_calib is True).

y_calib numpy.ndarray, optional

Calibration output data (required when get_res_on_calib is True).

Returns:

When get_res_on_calib is True returns a tuple (group2sizecalib, group2covcalib, treeID2testID2group, sample2predset). Otherwise returns (treeID2testID2group[0], sample2predset).

Return type:

tuple

Raises:

AssertionError – If self.group2conf_thresh has not been computed.

get_low_up_test_i(yj, qhat_low, qhat_up, qhat_low_test, qhat_up_test, t=None)

Select lower/upper bounds for a test sample using conformity thresholds.

The method inspects arrays of calibration quantile estimates and their corresponding test quantile estimates and either uses the provided threshold index t or searches for the largest index where the calibration value covers yj.

Parameters:

yj float

Calibration sample true value.

qhat_low numpy.ndarray

Lower quantile estimates from calibration.

qhat_up numpy.ndarray

Upper quantile estimates from calibration.

qhat_low_test numpy.ndarray

Lower quantile estimates for the test sample.

qhat_up_test numpy.ndarray

Upper quantile estimates for the test sample.

t int or None, optional

Optional fixed threshold index to use. If None the method searches.

Returns:

(low_j, up_j, t_idx) where low_j and up_j are the selected bounds for the test sample and t_idx is the chosen threshold index.

Return type:

tuple(float, float, int)

get_low_up_score(i, y_train, sample2calib_trees, treeID2trainID2values, treeID2testID2values, t_fixed=None)

Compute lower/upper bounds and conformity scores for one test index.

This method aggregates quantile estimates across trees (according to the aggregation configured in self.settings['type_aggregation_trees']) and returns, for the test sample at index i, three lists: the lower bounds, the upper bounds and the conformity score indices.

Parameters:

i int

Index of the test sample.

y_train numpy.ndarray or list

Array of calibration/training output values used to compute scores.

sample2calib_trees dict

Mapping from sample indices to lists of tree IDs used for calibration.

treeID2trainID2values dict

Mapping from tree ID to a mapping of training sample IDs to leaf values.

treeID2testID2values dict

Mapping from tree ID to a mapping of test sample IDs to leaf values.

t_fixed int or float, optional

Optional fixed threshold/index to use when selecting bounds. If None the method will search.

Returns:

A tuple (lower, upper, conf_scores) where each entry is a list with one element per calibration sample in y_train.

Return type:

tuple(list, list, list)

DisTreebution.UQ.UQ module

class DisTreebution.UQ.UQ.UQ(type_tree='PQRT', nested_set='CQR', type_conformal='split', group_coverage=False, type_aggregation_trees='vr', params=None)

Bases: object

Uncertainty quantification and conformal prediction helper.

This class wraps training of forest-like quantile/regression trees, conformalization, and construction/analysis of predictive sets.

It centralizes configuration (settings), builds a non-conformal quantile estimator (noconf) for querying, selects a conformal backend (conf) based on settings, and exposes utilities to train trees, conformalize them, predict conformal sets, and extract quantile estimates.

Constructor Parameters

Parameters:

type_tree str

Type of tree learner used. Examples:

• 'PMQRT' – per-tree quantile regression

• 'CRPS' – CRPS-optimized regression

• 'RT' – quadratic regression tree

Other strings may be supported by external tree classes.

nested_set str

Strategy for nested prediction sets (e.g. 'CQR', 'distri'). Used to choose the conformal backend. - If it contains 'CQR' → Conformalisation_CQR backend. - If it contains 'distri' → Conformalisation_distributional backend.

type_conformal str or None

Controls conformalization mode. If None, a generic Conformalisation backend is instantiated; otherwise nested_set determines which backend is used.

group_coverage bool

Whether to use group-aware conformal methods and prediction routines (calls to conf.*_group_coverage).

type_aggregation_trees str

How per-tree quantiles are aggregated across trees. Examples: - 'vr' – quantile bagging: average quantiles across trees - 'vr-avg' – distributional bagging: aggregates conditional CDFs before querying

params dict

Miscellaneous hyperparameters, may include: - 'nTrees' – number of trees to train - 'treeID2quantiles_train' – mapping tree IDs → quantiles (for PMQRT) - 'max_depth' – maximum depth for trees - 'min_samples_split' – minimum samples to split - 'IG_biais_correction' – whether to use leave-one-out or Mallows Cp to correct the estimate of information gains - 'max_depth_group' – max depth for group coverage trees - 'list_distri_low_quantiles' – lower quantiles for distributional conformal prediction

Behavior / Side Effects

• self.settings is a dict of configuration options.

• self.params stores the params dictionary.

• self.noconf is always constructed as a Conformalisation_CQR instance with the same settings except nested_set='CQR' (used for querying quantiles without conformalization).

• self.conf is chosen based on type_conformal and nested_set.

Public Attributes

Variables:

settings dict

Configuration options described above.

params dict

Hyperparameters.

noconf

A Conformalisation_CQR instance for non-conformal queries.

conf

Selected conformalization backend instance (Conformalisation_CQR, Conformalisation, or Conformalisation_distributional).

Example:

uq = UQ(type_tree='CRPS', nested_set='CQR', params={'nTrees': 100, 'max_depth': 6})
trees, sample2calib = uq.train_trees(X_train, y_train)
uq.conformalize(trees, X_calib, y_calib, alpha=0.1)
sample2predset = uq.predict_conformal_set(trees, X_test)
widths, coverages = uq.compute_width_coverage(sample2predset, y_test)

compute_width_coverage(sample2predset, y_test)

Computes the total width of prediction sets and coverage indicators for each test sample.

For each test sample, this method calculates:

• The sum of the widths of all predicted intervals (sub-intervals) associated with the sample.

• Whether the true value falls within any of the predicted intervals (coverage).

Parameters:

sample2predset list[tuple]

A list where each element corresponds to a test sample and contains a tuple of the form (lower_bound, upper_bound).

y_test array-like

The true target values for the test samples.

Returns:

• widths (numpy.ndarray) – Total widths of the predicted intervals for each test sample.

• coverages (numpy.ndarray) – Binary indicators (1 if the true value is covered by any interval,

0 otherwise) for each test sample.

Return type:

tuple[numpy.ndarray, numpy.ndarray]

compute_group_width_coverage(sample2predset, y_test, testID2group)

Computes the average prediction interval width and empirical coverage per group.

Groups are defined by the testID2group mapping.

Parameters:

sample2predset list[list[tuple]]

A list where each element corresponds to a test sample and contains one or more tuples of the form (lower_bound, upper_bound) representing predicted intervals.

y_test array-like

The true target values for the test samples.

testID2group dict

A mapping from test sample indices to group identifiers.

Returns:

• group2width (dict) – Mapping group_id → average interval width across all samples in the group.

• group2cov (dict) – Mapping group_id → empirical coverage, i.e., the fraction of samples in the group

whose true value lies within any of the predicted intervals.

Return type:

tuple[dict, dict]

train_trees(x_train, y_train, max_depth_ref_tree=-1)

Trains an ensemble of regression trees and tracks calibration indices for Out-of-Bag (OOB) conformal methods.

Parameters:

x_train numpy.ndarray

Training feature matrix of shape (n_train, n_features).

y_train numpy.ndarray

Training target values of shape (n_train,).

max_depth_ref_tree int

If not equal to -1, a reference tree is constructed and fitted (used by certain tree-training routines). If -1, no reference tree is used.

Returns:

• trees (list) – List of fitted tree objects, ordered by training loop.

• sample2calib_trees (dict) – Mapping sample_index → list of tree indices for which

the sample was placed in the calibration set (i.e., excluded from that tree’s training set). Useful for split-conformal calibration.

Return type:

tuple[list, dict]

conformalize(trees, x_calib, y_calib, alpha, **kwargs)

Runs conformalization (split-conformal) on the provided trees and calibration data.

Parameters:

trees list

List of fitted tree objects (the same trees returned by train_trees()).

x_calib numpy.ndarray

Calibration feature matrix.

y_calib numpy.ndarray

Calibration target values.

alpha float

Miscoverage level (e.g., 0.1 for 90% coverage).

kwargs

Additional keyword arguments forwarded to the conformal backend. For CQR nested sets, the nominal_quantiles parameter should be provided.

Behavior:

• If self.settings['group_coverage'] is True, calls

self.conf.conformalize_split_group_coverage(...). - Otherwise, calls self.conf.conformalize_split(...). - The conformal backend is expected to update its internal state used by predict_conformal_set().

predict_conformal_set(trees, x_test, return_testID2group=False, **kwargs)

Predict conformalized sets for test points using the conformal backend.

Parameters:

trees list

Trained trees (same as passed to conformalize()).

x_test numpy.ndarray

Test feature matrix of shape (n_test, n_features).

return_testID2group bool

When group_coverage is True, optionally return the mapping from test indices to groups alongside the prediction sets.

kwargs

Additional keyword arguments forwarded to the conformal backend’s prediction routine.

Returns:

• If group_coverage is False:

sample2predset – Sequence-like (length n_test) of (low, up) prediction interval pairs.

• If group_coverage is True:

• If return_testID2group is False:

  sample2predset as above.

• If return_testID2group is True:

  Tuple (testID2group, sample2predset) where testID2group is the mapping used.

Notes:

• Delegates to either predict_conformal_set_split() or

predict_conformal_set_split_group_coverage() on self.conf.

get_quantile_estimate(trees, x_test, quantiles)

Query aggregate quantile estimates for each test sample from the (non- or pre-) conformal tree predictions.

Parameters:

trees list

Trained tree objects.

x_test numpy.ndarray

Test features matrix of shape (n_test, n_features).

quantiles iterable

List or array of quantile levels to estimate (values in [0, 1]).

Returns:

sample2quantiles (dict) – Mapping from sample index to list of estimated quantile values. The inner list has the same ordering as the input quantiles argument.

Behavior / Implementation details:

• Calls self.conf.preprocess_trees(trees, x_test) to obtain a structure

treeID2testID2values mapping tree IDs to test sample leaf/value indexes.

• For each requested quantile q, finds the nearest available query quantile

in self.conf.quantiles_query (via argmin on absolute distance) and obtains the set of tree IDs associated with that query from self.conf.quantile_query2treeIDs.

DisTreebution.UQ.datasets module

DisTreebution.UQ.datasets.level2idx(n, q)

DisTreebution.UQ.datasets.sine_skew(n=100, x=None, levels=None)

DisTreebution.UQ.datasets.griewank(n=100, x=None, levels=None)

DisTreebution.UQ.datasets.michalewicz(n=100, x=None, levels=None)

DisTreebution.UQ.datasets.candes_sesia(n=100, d=4, x=None, levels=None)

DisTreebution.UQ.datasets.bimodal(n=100, x=None, levels=None)

DisTreebution.UQ.datasets.asymmetric(n=100, x=None, levels=None)

DisTreebution.UQ.datasets.homoscedastic(n=100, x=None, levels=None)

DisTreebution.UQ.datasets.heteroscedastic(n=100, x=None, levels=None)

DisTreebution.UQ.datasets.sales(n=1000, x=None, levels=None)

DisTreebution.UQ.utils module

DisTreebution.UQ.utils.filter_dict(dict, keys_to_keep)

DisTreebution.UQ.utils.crps_from_quantiles(quantiles, tau, y)

Compute CRPS from predicted quantiles.

Parameters: - quantiles: List or numpy array of predicted quantiles. - tau: List or numpy array of corresponding quantile levels (e.g., [0.1, 0.2, …, 0.9]). - y: The observed value.

Returns: - crps: The computed CRPS score.

DisTreebution.UQ.utils.compute_crps(sample2predset, y_test, ls_q)

DisTreebution.UQ.utils.load_dataset(root_path, dataset)

Module contents