Skip to main content

automl.ml

sklearn_metric_loss_score

def sklearn_metric_loss_score(metric_name: str, y_predict, y_true, labels=None, sample_weight=None, groups=None)

Loss using the specified metric.

Arguments:

  • metric_name - A string of the metric name, one of 'r2', 'rmse', 'mae', 'mse', 'accuracy', 'roc_auc', 'roc_auc_ovr', 'roc_auc_ovo', 'roc_auc_weighted', 'roc_auc_ovo_weighted', 'roc_auc_ovr_weighted', 'log_loss', 'mape', 'f1', 'ap', 'ndcg', 'micro_f1', 'macro_f1'.
  • y_predict - A 1d or 2d numpy array of the predictions which can be used to calculate the metric. E.g., 2d for log_loss and 1d for others.
  • y_true - A 1d numpy array of the true labels.
  • labels - A list or an array of the unique labels.
  • sample_weight - A 1d numpy array of the sample weight.
  • groups - A 1d numpy array of the group labels.

Returns:

  • score - A float number of the loss, the lower the better.

norm_confusion_matrix

def norm_confusion_matrix(y_true: Union[np.array, Series], y_pred: Union[np.array, Series])

normalized confusion matrix.

Arguments:

  • estimator - A multi-class classification estimator.
  • y_true - A numpy array or a pandas series of true labels.
  • y_pred - A numpy array or a pandas series of predicted labels.

Returns:

A normalized confusion matrix.

multi_class_curves

def multi_class_curves(y_true: Union[np.array, Series], y_pred_proba: Union[np.array, Series], curve_func: Callable)

Binarize the data for multi-class tasks and produce ROC or precision-recall curves.

Arguments:

  • y_true - A numpy array or a pandas series of true labels.
  • y_pred_proba - A numpy array or a pandas dataframe of predicted probabilites.
  • curve_func - A function to produce a curve (e.g., roc_curve or precision_recall_curve).

Returns:

A tuple of two dictionaries with the same set of keys (class indices). The first dictionary curve_x stores the x coordinates of each curve, e.g., curve_x[0] is an 1D array of the x coordinates of class 0. The second dictionary curve_y stores the y coordinates of each curve, e.g., curve_y[0] is an 1D array of the y coordinates of class 0.