DOC add metrics documentation in user guide (scikit-learn-contrib#375)

Author: glemaitre

doc/metrics.rst

@@ -0,0 +1,44 @@
+.. _metrics:
+
+=======
+Metrics
+=======
+
+.. currentmodule:: imblearn.metrics
+
+Currently, scikit-learn only offers the
+``sklearn.metrics.balanced_accuracy_score`` (in 0.20) as metric to deal with
+imbalanced datasets. The module :mod:`imblearn.metrics` offers a couple of
+other metrics which are used in the literature to evaluate the quality of
+classifiers.
+
+.. _sensitivity_specificity:
+
+Sensitivity and specificity metrics
+-----------------------------------
+
+Sensitivity and specificity are metrics which are well known in medical
+imaging. Sensitivity (also called true positive rate or recall) is the
+proportion of the positive samples which is well classified while specificity
+(also called true negative rate) is the proportion of the negative samples
+which are well classified. Therefore, depending of the field of application,
+either the sensitivity/specificity or the precision/recall pair of metrics are
+used.
+
+Currently, only the `precision and recall metrics
+<http://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html>`_
+are implemented in scikit-learn. :func:`sensitivity_specificity_support`,
+:func:`sensitivity_score`, and :func:`specificity_score` add the possibility to
+use those metrics.
+
+.. _imbalanced_metrics:
+
+Additional metrics specific to imbalanced datasets
+--------------------------------------------------
+
+The :func:`geometric_mean_score` is the root of the product of class-wise
+sensitivity. This measure tries to maximize the accuracy on each of the classes
+while keeping these accuracies balanced.
+
+The :func:`make_index_balanced_accuracy` can wrap any metric and give more
+importance to a specific class using the parameter ``alpha``.

doc/user_guide.rst

@@ -14,5 +14,6 @@ User Guide
  under_sampling.rst
  combine.rst
  ensemble.rst
+ metrics.rst
  Dataset loading utilities <datasets/index.rst>
  developers_utils.rst

doc/whats_new/v0.0.4.rst

@@ -12,6 +12,9 @@ Enhancement
 - Add a documentation node to create a balanced random forest from a balanced
  bagging classifier. :issue:`372` by :user:`Guillaume Lemaitre <glemaitre>`.
 
+- Document the metrics to evaluate models on imbalanced dataset. :issue:`367`
+ by :user:`Guillaume Lemaitre <glemaitre>`.
+
 Bug fixes
 .........
 

imblearn/metrics/classification.py

@@ -60,6 +60,8 @@ def sensitivity_specificity_support(y_true,
  returns the average sensitivity and specificity if ``average``
  is one of ``'weighted'``.
 
+ Read more in the :ref:`User Guide <sensitivity_specificity>`.
+
  Parameters
  ----------
  y_true : ndarray, shape (n_samples, )
@@ -277,6 +279,8 @@ def sensitivity_score(y_true,
 
  The best value is 1 and the worst value is 0.
 
+ Read more in the :ref:`User Guide <sensitivity_specificity>`.
+
  Parameters
  ----------
  y_true : ndarray, shape (n_samples, )
@@ -376,6 +380,8 @@ def specificity_score(y_true,
 
  The best value is 1 and the worst value is 0.
 
+ Read more in the :ref:`User Guide <sensitivity_specificity>`.
+
  Parameters
  ----------
  y_true : ndarray, shape (n_samples, )
@@ -486,6 +492,8 @@ class is unrecognized by the classifier, G-mean resolves to zero. To
  unrecognized classes can be "corrected" to be a user specified value
  (instead of zero). This option works only if ``average == 'multiclass'``.
 
+ Read more in the :ref:`User Guide <imbalanced_metrics>`.
+
  Parameters
  ----------
  y_true : ndarray, shape (n_samples, )
@@ -649,6 +657,8 @@ def make_index_balanced_accuracy(alpha=0.1, squared=True):
  balanced accuracy. ``y_score`` cannot be used since the dominance
  cannot be computed.
 
+ Read more in the :ref:`User Guide <imbalanced_metrics>`.
+
  Parameters
  ----------
  alpha : float, optional (default=0.1)