Metric Utils

raimitigations.utils.metric_utils.get_metrics(y: Union[ndarray, list], y_pred: Union[ndarray, list, DataFrame], regression: bool = False, best_th_auc: bool = True, fixed_th: Optional[float] = None, return_th_list: bool = False)

Evaluates the performance of a prediction array based on its true values. This function computes a set of metrics and return a dictionary with the following metrics (depends if it is a regression or classification problem):

  • Classification: ROC AUC, Precision, Recall, F1, accuracy, log loss, threshold used, and final classes predicted (using the probabilities with the threshold);

  • Regression: MSE, RMSE, MAE, and R2

Parameters

  • y – an array with the true labels;

  • y_pred – an array with the predicted values. For classification problems, this array must contain the probabilities for each class, with shape = (N, C), where N is the number of rows and C is the number of classes;

  • regression – if True, regression metrics are computed. If False, only classification metrics are computed;

  • best_th_auc – if True, the best threshold is computed using ROC graph. If False, the threshold is computed using the precision x recall graph;

  • fixed_th – a value between [0, 1] that should be used as the threshold for classification tasks;

  • return_th_list – returns the list of thresholds tested (be it using the ROC curve, or the precision and recall curve).

Returns

a dictionary with the following metrics:

  • Classification: ROC AUC, Precision, Recall, F1, accuracy, log loss, threshold used, and final classes predicted (using the probabilities with the threshold);

  • Regression: MSE, RMSE, MAE, and R2

Return type

dict