macrosynergy.learning.model_selection#

class ModelAveragingRegressor(estimators, scoring=make_scorer(r2_score, response_method='predict'), cv=ExpandingKFoldPanelSplit(min_n_splits=None, n_splits=5), temperature='max-min', min_weight=0.0, error_score=nan)[source]#

Bases: BaseEstimator, RegressorMixin

Ensemble of regression models weighted by their cross-validation performance.

Note

This class is still experimental: the predictions and the API might change without any deprecation cycle.

Parameters:

  • estimators (list of tuples) – List of (name, estimator, param_grid) tuples where: - name: str, name of the estimator - estimator: scikit-learn regressor object - param_grid: dict, hyperparameter grid for GridSearchCV

  • scoring (callable, default=make_scorer(r2_score, greater_is_better=True)) – Scikit-learn compatible scorer to evaluate the performance of each estimator during cross-validation.

  • cv (cross-validation class, default=ExpandingKFoldPanelSplit(n_splits=5)) – Cross-validation splitting strategy. Can be any scikit-learn compatible cross-validation class.

  • temperature (str or float, default="max-min") – Method to scale the cross-validation scores for the softmax weighting. Options are: - “max-min”: scale by the range (max - min) of the scores - “std”: scale by the standard deviation of the scores - “mad”: scale by the median absolute deviation of the scores - “iqr”: scale by the interquartile range of the scores - float: a custom scaling factor

  • min_weight (float, default=0.0) – Minimum weight to be included in the final ensemble. Weights below this threshold will be set to zero.

  • error_score ("raise" or numeric, default=np.nan) – Value to assign to the score if an error occurs in estimator fitting. If set to ” raise”, the error is raised. If numeric, the score is set to this value.

Notes

Model averaging uses cross-validation to estimate a probability distribution over a discrete space of models. The final prediction is a weighted average of the predictions from each model. The weights correspond to the estimated probability of each model being the “optimal” model, based on estimated out-of-sample performance.

One issue with the cross-validation estimator is that maximising the scores amplifies noise in the estimation, resulting in a “winners curse” effect. When a large number of models are considered, this selection bias is more pronounced and can result in detrimental trading signals, simply by flickering between similar models across time periods, even if those models have very similar out-of-sample performance. The dynamics caused by this model selection overfitting impacts the stability of the trading signal, not to mention transaction costs.

fit(X, y)[source]#
predict(X)[source]#
set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → ModelAveragingRegressor#

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Parameters:

sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.

Returns:

self – The updated object.

Return type:

object

Submodules#