Overview of the different settings possible#

  1. Choosing type_tree

    • ‘PMQRT’ : train quantile regression trees (multi-quantile support). Use when you want direct quantile estimates.

    • ‘RT’ : standard regression tree with squared-error splitting (uses RegressionTreeQuadratic).

    • ‘CRPS’ : specialized tree optimized for CRPS (Continuous Ranked Probability Score) available in CRPSRT/RegressionTree.py.

  2. Choosing type_conformal

    • ‘None’ : in case you don’t plan to use conformal prediction methods

    • ‘split’ : if you want to use split conformal prediction

  3. Conformalization nested_set options

    • ‘CQR’ / ‘CQR-m’ / ‘CQR-r’ : variants of Conformalized Quantile Regression implemented in Conformalisation_CQR.py. They require you pass nominal_quantiles to conformalize (see example below).

    • ‘distributional’ : distributional conformalization implemented in Conformalisation_distributional.py. It uses a list of nested low quantiles (param list_distri_low_quantiles) and supports group coverage.

  4. Group coverage

    • Set group_coverage=True when instantiating UQ to enable group-aware conformalization.

  5. Choose aggregation type

    • ‘vr’ : corresponds to quantile bagging: simple average of quantile obtained from single trees in the forest

    • ‘vr-avg’ : corresponds to distributional bagging: aggregates conditional CDF from all trees before querying the desired quantile

  6. Parameters:

    • ‘nTrees’ : number of trees to train

    • ‘treeID2quantiles_train’ : dict mapping tree IDs to quantiles to train for (used in PMQRT)

    • ‘max_depth’ : maximum depth passed to tree constructors

    • ‘min_samples_split’ : minimum samples to split passed to tree constructors

    • ‘IG_biais_correction’ : whether tree fit uses leave-one-out or Mallows correction of information gains

    • ‘max_depth_group’ : maximum depth for group coverage trees

    • ‘list_distri_low_quantiles’: list of lower quantiles for distributional conformal prediction

  7. Main methods:

    • Use uq.train_trees(X_train, y_train) to train a forest

    1. Query quantiles

      • Use uq.get_quantile_estimate(trees, X_test, quantiles) to query raw quantiles aggregated across trees.

      • Use uq.compute_width_coverage(sample2predset, y_test) to compute width and coverage if sample2predset contains confidence intervals for each test sample.

    2. Conformal prediction

      • Use uq.conformalize(trees, X_calib, y_calib, alpha, nominal_quantiles=[2*alpha]) to conformalize the your forest. Note that nominal_quantiles is only needed if you trained PMQRTs.

      • Use uq.predict_conformal_set(trees, X_test) to predict the conformal sets on the test samples.

      • Use uq.compute_width_coverage(sample2predset, y_test) to get the width and coverage of your predictions if sample2predset contains the conformalized sets. Note that if you used PMQRTs, sample2predset actually contains different conformal sets based on the different nominal_quantiles you considered when conformalizing the method. In our example, you specify a single nominal quantile and those we would get access to the conformal sets on the test set using sample2predset|0]. Therefore, one should use uq.compute_width_coverage(sample2predset[0], y_test).

Show Source