Phi Coefficient - Wikipedia

Statistical measure of association for two binary variables

In statistics, the phi coefficient, or mean square contingency coefficient, denoted by φ or rφ, is a measure of association for two binary variables.

In machine learning, it is known as the Matthews correlation coefficient (MCC) and used as a measure of the quality of binary (two-class) classifications, introduced by biochemist Brian W. Matthews in 1975.[1]

Introduced by Karl Pearson,[2] and also known as the Yule phi coefficient from its introduction by Udny Yule in 1912[3] this measure is similar to the Pearson correlation coefficient in its interpretation.

In meteorology, the phi coefficient,[4] or its square (the latter aligning with M. H. Doolittle's original proposition from 1885[5]), is referred to as the Doolittle Skill Score or the Doolittle Measure of Association.

Definition

[edit]

A Pearson correlation coefficient estimated for two binary variables will return the phi coefficient.[6]

Two binary variables are considered positively associated if most of the data falls along the diagonal cells. In contrast, two binary variables are considered negatively associated if most of the data falls off the diagonal.

If we have a 2×2 table for two random variables x and y

y = 1	y = 0	total
x = 1	n 11 {\displaystyle n_{11}}	n 10 {\displaystyle n_{10}}	n 1 ∙ {\displaystyle n_{1\bullet }}
x = 0	n 01 {\displaystyle n_{01}}	n 00 {\displaystyle n_{00}}	n 0 ∙ {\displaystyle n_{0\bullet }}
total	n ∙ 1 {\displaystyle n_{\bullet 1}}	n ∙ 0 {\displaystyle n_{\bullet 0}}	n {\displaystyle n}

where n11, n10, n01, n00, are non-negative counts of numbers of observations that sum to n, the total number of observations. The phi coefficient that describes the association of x and y is

φ = n 11 n 00 − n 10 n 01 n 1 ∙ n 0 ∙ n ∙ 0 n ∙ 1 . {\displaystyle \varphi ={\frac {n_{11}n_{00}-n_{10}n_{01}}{\sqrt {n_{1\bullet }n_{0\bullet }n_{\bullet 0}n_{\bullet 1}}}}.}

Phi is related to the point-biserial correlation coefficient and Cohen's d and estimates the extent of the relationship between two variables (2×2).[7]

The phi coefficient can also be expressed using only n {\displaystyle n} , n 11 {\displaystyle n_{11}} , n 1 ∙ {\displaystyle n_{1\bullet }} , and n ∙ 1 {\displaystyle n_{\bullet 1}} , as

φ = n n 11 − n 1 ∙ n ∙ 1 n 1 ∙ n ∙ 1 ( n − n 1 ∙ ) ( n − n ∙ 1 ) . {\displaystyle \varphi ={\frac {nn_{11}-n_{1\bullet }n_{\bullet 1}}{\sqrt {n_{1\bullet }n_{\bullet 1}(n-n_{1\bullet })(n-n_{\bullet 1})}}}.}

Maximum values

[edit]

Although computationally the Pearson correlation coefficient reduces to the phi coefficient in the 2×2 case, they are not in general the same. The Pearson correlation coefficient ranges from −1 to +1, where ±1 indicates perfect agreement or disagreement, and 0 indicates no relationship. The phi coefficient has a maximum value that is determined by the distribution of the two variables if one or both variables can take on more than two values.[further explanation needed] See Davenport and El-Sanhury (1991) [8] for a thorough discussion.

Machine learning

[edit]

The MCC is defined identically to phi coefficient, introduced by Karl Pearson,[2][9] also known as the Yule phi coefficient from its introduction by Udny Yule in 1912.[3] Despite these antecedents which predate Matthews's use by several decades, the term MCC is widely used in the field of bioinformatics and machine learning.

The coefficient accounts for true and false positives and negatives and is generally regarded as a balanced measure which can be used even if the classes are of very different sizes.[10] The MCC is in essence a correlation coefficient between the observed and predicted binary classifications; it returns a value between −1 and +1. A coefficient of +1 represents a perfect prediction, 0 no better than random prediction and −1 indicates total disagreement between prediction and observation. However, if MCC equals neither −1, 0, or +1, it is not a reliable indicator of how similar a predictor is to random guessing because MCC is dependent on the dataset.[11] MCC is closely related to the chi-square statistic for a 2×2 contingency table

| MCC | = χ 2 n {\displaystyle |{\text{MCC}}|={\sqrt {\frac {\chi ^{2}}{n}}}}

where n is the total number of observations.

While there is no perfect way of describing the confusion matrix of true and false positives and negatives by a single number, the Matthews correlation coefficient is generally regarded as being one of the best such measures.[12] Other measures, such as the proportion of correct predictions (also termed accuracy), are not useful when the two classes are of very different sizes. For example, assigning every object to the larger set achieves a high proportion of correct predictions, but is not generally a useful classification.

The MCC can be calculated directly from the confusion matrix using the formula:

MCC = T P × T N − F P × F N ( T P + F P ) ( T P + F N ) ( T N + F P ) ( T N + F N ) {\displaystyle {\text{MCC}}={\frac {{\mathit {TP}}\times {\mathit {TN}}-{\mathit {FP}}\times {\mathit {FN}}}{\sqrt {({\mathit {TP}}+{\mathit {FP}})({\mathit {TP}}+{\mathit {FN}})({\mathit {TN}}+{\mathit {FP}})({\mathit {TN}}+{\mathit {FN}})}}}}

In this equation, TP is the number of true positives, TN the number of true negatives, FP the number of false positives and FN the number of false negatives. If exactly one of the four sums in the denominator is zero, the denominator can be arbitrarily set to one; this results in a Matthews correlation coefficient of zero, which can be shown to be the correct limiting value. In case two or more sums are zero (e.g. both labels and model predictions are all positive or negative), the limit does not exist.

The MCC can be calculated with the formula:

MCC = P P V × T P R × T N R × N P V − F D R × F N R × F P R × F O R {\displaystyle {\text{MCC}}={\sqrt {{\mathit {PPV}}\times {\mathit {TPR}}\times {\mathit {TNR}}\times {\mathit {NPV}}}}-{\sqrt {{\mathit {FDR}}\times {\mathit {FNR}}\times {\mathit {FPR}}\times {\mathit {FOR}}}}}

using the positive predictive value, the true positive rate, the true negative rate, the negative predictive value, the false discovery rate, the false negative rate, the false positive rate, and the false omission rate.

The original formula as given by Matthews was:[1]

N = T N + T P + F N + F P S = T P + F N N P = T P + F P N MCC = T P / N − S × P P S ( 1 − S ) ( 1 − P ) {\displaystyle {\begin{aligned}N&={\mathit {TN}}+{\mathit {TP}}+{\mathit {FN}}+{\mathit {FP}}\\[6pt]S&={\frac {{\mathit {TP}}+{\mathit {FN}}}{N}}\\[6pt]P&={\frac {{\mathit {TP}}+{\mathit {FP}}}{N}}\\[6pt]{\text{MCC}}&={\frac {{\mathit {TP}}/N-S\times P}{\sqrt {PS(1-S)(1-P)}}}\end{aligned}}}

This is equal to the formula given above. As a correlation coefficient, the Matthews correlation coefficient is the geometric mean of the regression coefficients of the problem and its dual. The component regression coefficients of the Matthews correlation coefficient are markedness (Δp) and Youden's J statistic (informedness or Δp′).[12][13] Markedness and informedness correspond to different directions of information flow and generalize Youden's J statistic, the δ p {\displaystyle \delta p} statistics, while their geometric mean generalizes the Matthews correlation coefficient to more than two classes.[12]

Some scientists claim the Matthews correlation coefficient to be the most informative single score to establish the quality of a binary classifier prediction in a confusion matrix context.[14][15]

Example

[edit]

Given a sample of 12 pictures, 8 of cats and 4 of dogs, where cats belong to class 1 and dogs belong to class 0,

actual = [1,1,1,1,1,1,1,1,0,0,0,0],

assume that a classifier that distinguishes between cats and dogs is trained, and we take the 12 pictures and run them through the classifier, and the classifier makes 9 accurate predictions and misses 3: 2 cats wrongly predicted as dogs (first 2 predictions) and 1 dog wrongly predicted as a cat (last prediction).

prediction = [0,0,1,1,1,1,1,1,0,0,0,1]

With these two labelled sets (actual and predictions) we can create a confusion matrix that will summarize the results of testing the classifier:

PredictedclassActual class	Cat	Dog
Cat	6	2
Dog	1	3

In this confusion matrix, of the 8 cat pictures, the system judged that 2 were dogs, and of the 4 dog pictures, it predicted that 1 was a cat. All correct predictions are located in the diagonal of the table (highlighted in bold), so it is easy to visually inspect the table for prediction errors, as they will be represented by values outside the diagonal.

In abstract terms, the confusion matrix is as follows:

PredictedclassActual class	P	N
P	TP	FN
N	FP	TN

where P = positive; N = negative; TP = truepositive; FP = false positive; TN = true negative; FN = false negative.

Plugging the numbers from the formula:

MCC = 6 × 3 − 1 × 2 ( 6 + 1 ) × ( 6 + 2 ) × ( 3 + 1 ) × ( 3 + 2 ) = 16 1120 ≈ 0.478 {\displaystyle {\text{MCC}}={\frac {6\times 3-1\times 2}{\sqrt {(6+1)\times (6+2)\times (3+1)\times (3+2)}}}={\frac {16}{\sqrt {1120}}}\approx 0.478}

Confusion matrix

[edit] Main article: Confusion matrix

Let us define an experiment from P positive instances and N negative instances for some condition. The four outcomes can be formulated in a 2×2 contingency table or confusion matrix, as follows:

Predicted condition		Sources: [16][17][18][19][20][21][22][23] view talk edit
Total population = P + N	Predicted positive	Predicted negative	Informedness, bookmaker informedness (BM) = TPR + TNR − 1	Prevalence threshold (PT) = ⁠√TPR × FPR − FPR/TPR − FPR⁠
Actual condition	Real Positive (P) [a]	True positive (TP), hit[b]	False negative (FN), miss, underestimation	True positive rate (TPR), recall, sensitivity (SEN), probability of detection, hit rate, power = ⁠TP/P⁠ = 1 − FNR	False negative rate (FNR), miss rate type II error [c] = ⁠FN/P⁠ = 1 − TPR
	Real Negative (N)[d]	False positive (FP), false alarm, overestimation	True negative (TN), correct rejection[e]	False positive rate (FPR), probability of false alarm, fall-out type I error [f] = ⁠FP/N⁠ = 1 − TNR	True negative rate (TNR), specificity (SPC), selectivity = ⁠TN/N⁠ = 1 − FPR
Prevalence = ⁠P/P + N⁠	Positive predictive value (PPV), precision = ⁠TP/TP + FP⁠ = 1 − FDR	False omission rate (FOR) = ⁠FN/TN + FN⁠ = 1 − NPV	Positive likelihood ratio (LR+) = ⁠TPR/FPR⁠	Negative likelihood ratio (LR−) = ⁠FNR/TNR⁠
Accuracy (ACC) = ⁠TP + TN/P + N⁠	False discovery rate (FDR) = ⁠FP/TP + FP⁠ = 1 − PPV	Negative predictive value (NPV) = ⁠TN/TN + FN⁠ = 1 − FOR	Markedness (MK), deltaP (Δp) = PPV + NPV − 1	Diagnostic odds ratio (DOR) = ⁠LR+/LR−⁠
Balanced accuracy (BA) = ⁠TPR + TNR/2⁠	F1 score = ⁠2 PPV × TPR/PPV + TPR⁠ = ⁠2 TP/2 TP + FP + FN⁠	Fowlkes–Mallows index (FM) = √PPV × TPR	phi or Matthews correlation coefficient (MCC) = √TPR × TNR × PPV × NPV - √FNR × FPR × FOR × FDR	Threat score (TS), critical success index (CSI), Jaccard index = ⁠TP/TP + FN + FP⁠

1. ^ the number of real positive cases in the data
2. ^ A test result that correctly indicates the presence of a condition or characteristic
3. ^ Type II error: A test result which wrongly indicates that a particular condition or attribute is absent
4. ^ the number of real negative cases in the data
5. ^ A test result that correctly indicates the absence of a condition or characteristic
6. ^ Type I error: A test result which wrongly indicates that a particular condition or attribute is present

Multiclass case

[edit]

The Matthews correlation coefficient has been generalized to the multiclass case. The generalization called the R K {\displaystyle R_{K}} statistic (for K different classes) was defined in terms of a K × K {\displaystyle K\times K} confusion matrix C {\displaystyle C} [24] .[25]

MCC = ∑ k ∑ l ∑ m C k k C l m − C k l C m k ∑ k ( ∑ l C k l ) ( ∑ k ′ | k ′ ≠ k ∑ l ′ C k ′ l ′ ) ∑ k ( ∑ l C l k ) ( ∑ k ′ | k ′ ≠ k ∑ l ′ C l ′ k ′ ) {\displaystyle {\text{MCC}}={\frac {\sum _{k}\sum _{l}\sum _{m}C_{kk}C_{lm}-C_{kl}C_{mk}}{{\sqrt {\sum _{k}\left(\sum _{l}C_{kl}\right)\left(\sum _{k'|k'\neq k}\sum _{l'}C_{k'l'}\right)}}{\sqrt {\sum _{k}\left(\sum _{l}C_{lk}\right)\left(\sum _{k'|k'\neq k}\sum _{l'}C_{l'k'}\right)}}}}}

When there are more than two labels the MCC will no longer range between −1 and +1. Instead the minimum value will be between −1 and 0 depending on the true distribution. The maximum value is always +1.

This formula can be more easily understood by defining intermediate variables:[26]

• i {\displaystyle i} is the actual value index
• j {\displaystyle j} is the predicted value index
• K {\displaystyle K} is the total number of classes
• t k = ∑ i C i k {\displaystyle t_{k}=\sum _{i}C_{ik}} the number of times class k truly occurred,
• p k = ∑ j C k j {\displaystyle p_{k}=\sum _{j}C_{kj}} the number of times class k was predicted,
• c = ∑ k C k k {\displaystyle c=\sum _{k}C_{kk}} the total number of samples correctly predicted,
• s = ∑ i ∑ j C i j {\displaystyle s=\sum _{i}\sum _{j}C_{ij}} the total number of samples. This allows the formula to be expressed as:

MCC = c s − t → ⋅ p → s 2 − p → ⋅ p → s 2 − t → ⋅ t → {\displaystyle {\text{MCC}}={\frac {cs-{\vec {t}}\cdot {\vec {p}}}{{\sqrt {s^{2}-{\vec {p}}\cdot {\vec {p}}}}{\sqrt {s^{2}-{\vec {t}}\cdot {\vec {t}}}}}}}

PredictedclassActual class	Cat	Dog	Sum
Cat	6	2	8
Dog	1	3	4
Sum	7	5	12

Using above formula to compute MCC measure for the dog and cat example discussed above, where the confusion matrix is treated as a 2 × Multiclass example:

MCC = ( 6 + 3 ) × 12 − 5 × 4 − 7 × 8 12 2 − 5 2 − 7 2 12 2 − 4 2 − 8 2 = 32 4480 ≈ 0.478 {\displaystyle {\text{MCC}}={\frac {(6+3)\times {\color {green}12}\;-\;{\color {blue}5}\times {\color {brown}4}\;-\;{\color {purple}7}\times {\color {maroon}8}}{{\sqrt {{\color {green}12}^{2}-{\color {blue}5}^{2}-{\color {purple}7}^{2}}}{\sqrt {{\color {green}12}^{2}-{\color {brown}4}^{2}-{\color {maroon}8}^{2}}}}}={\frac {32}{\sqrt {4480}}}\approx 0.478}

An alternative generalization of the Matthews Correlation Coefficient to more than two classes was given by Powers [12] by the definition of Correlation as the geometric mean of Informedness and Markedness.

Several generalizations of the Matthews Correlation Coefficient to more than two classes along with new Multivariate Correlation Metrics for multinary classification have been presented by P Stoica and P Babu.[27]

See also

[edit]

• Cohen's kappa
• Contingency table
• Cramér's V, a similar measure of association between nominal variables.
• F1 score
• Fowlkes–Mallows index
• Polychoric correlation (subtype: Tetrachoric correlation), when variables are seen as dichotomized versions of (latent) continuous variables

References

[edit]

1. ^ a b Matthews, B. W. (1975). "Comparison of the predicted and observed secondary structure of T4 phage lysozyme". Biochimica et Biophysica Acta (BBA) - Protein Structure. 405 (2): 442–451. doi:10.1016/0005-2795(75)90109-9. PMID 1180967.
2. ^ a b Cramer, H. (1946). Mathematical Methods of Statistics. Princeton: Princeton University Press, p. 282 (second paragraph). ISBN 0-691-08004-6 https://archive.org/details/in.ernet.dli.2015.223699
3. ^ a b Yule, G. Udny (1912). "On the Methods of Measuring Association Between Two Attributes". Journal of the Royal Statistical Society. 75 (6): 579–652. doi:10.2307/2340126. JSTOR 2340126.
4. ^ Hogan, Robert J.; Mason, Ian B. (December 16, 2011). "Deterministic Forecasts of Binary Events". In Jolliffe, Ian T.; Stephenson, David B. (eds.). Forecast Verification: A Practitioner's Guide in Atmospheric Science, Second Edition. John Wiley & Sons. pp. 31–59. doi:10.1002/9781119960003.ch3. ISBN 978-0-470-66071-3.
5. ^ Armistead, Timothy W. (2016). "Misunderstood and Unattributed: Revisiting M. H. Doolittle's Measures of Association, With a Note on Bayes' Theorem". The American Statistician. 70 (1): 63–73. doi:10.1080/00031305.2015.1086686. JSTOR 45118274.
6. ^ Guilford, J. (1936). Psychometric Methods. New York: McGraw–Hill Book Company, Inc.
7. ^ Aaron, B., Kromrey, J. D., & Ferron, J. M. (1998, November). Equating r-based and d-based effect-size indices: Problems with a commonly recommended formula. Paper presented at the annual meeting of the Florida Educational Research Association, Orlando, FL. (ERIC Document Reproduction Service No. ED433353)
8. ^ Davenport, E.; El-Sanhury, N. (1991). "Phi/Phimax: Review and Synthesis". Educational and Psychological Measurement. 51 (4): 821–8. doi:10.1177/001316449105100403.
9. ^ Date unclear, but prior to his death in 1936.
10. ^ Boughorbel, S.B (2017). "Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric". PLOS ONE. 12 (6) e0177678. Bibcode:2017PLoSO..1277678B. doi:10.1371/journal.pone.0177678. PMC 5456046. PMID 28574989.
11. ^ Chicco, D.; Tötsch, N.; Jurman, G. (2021). "The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation". BioData Mining. 14 (1): 13. doi:10.1186/s13040-021-00244-z. PMC 7863449. PMID 33541410.
12. ^ a b c d Powers, David M. W. (10 October 2020). "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation". arXiv:2010.16061 [cs.LG].
13. ^ Perruchet, P.; Peereman, R. (2004). "The exploitation of distributional information in syllable processing". J. Neurolinguistics. 17 (2–3): 97–119. doi:10.1016/s0911-6044(03)00059-9. S2CID 17104364.
14. ^ Chicco D (December 2017). "Ten quick tips for machine learning in computational biology". BioData Mining. 10 (35) 35. doi:10.1186/s13040-017-0155-3. PMC 5721660. PMID 29234465.
15. ^ Chicco D, Jurman G (February 2023). "The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification". BioData Min. 16 (1) 4. doi:10.1186/s13040-023-00322-4. PMC 9938573. PMID 36800973.
16. ^ Fawcett, Tom (2006). "An Introduction to ROC Analysis" (PDF). Pattern Recognition Letters. 27 (8): 861–874. doi:10.1016/j.patrec.2005.10.010. S2CID 2027090.
17. ^ Provost, Foster; Tom Fawcett (2013-08-01). "Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking". O'Reilly Media, Inc.
18. ^ Powers, David M. W. (2011). "Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation". Journal of Machine Learning Technologies. 2 (1): 37–63.
19. ^ Ting, Kai Ming (2011). Sammut, Claude; Webb, Geoffrey I. (eds.). Encyclopedia of machine learning. Springer. doi:10.1007/978-0-387-30164-8. ISBN 978-0-387-30164-8.
20. ^ Brooks, Harold; Brown, Barb; Ebert, Beth; Ferro, Chris; Jolliffe, Ian; Koh, Tieh-Yong; Roebber, Paul; Stephenson, David (2015-01-26). "WWRP/WGNE Joint Working Group on Forecast Verification Research". Collaboration for Australian Weather and Climate Research. World Meteorological Organisation. Retrieved 2019-07-17.
21. ^ Chicco D, Jurman G (January 2020). "The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation". BMC Genomics. 21 (1): 6-1–6-13. doi:10.1186/s12864-019-6413-7. PMC 6941312. PMID 31898477.
22. ^ Chicco D, Toetsch N, Jurman G (February 2021). "The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation". BioData Mining. 14 (13): 13. doi:10.1186/s13040-021-00244-z. PMC 7863449. PMID 33541410.
23. ^ Tharwat A. (August 2018). "Classification assessment methods". Applied Computing and Informatics. 17: 168–192. doi:10.1016/j.aci.2018.08.003.
24. ^ Gorodkin, Jan (2004). "Comparing two K-category assignments by a K-category correlation coefficient". Computational Biology and Chemistry. 28 (5): 367–374. doi:10.1016/j.compbiolchem.2004.09.006. PMID 15556477.
25. ^ Gorodkin, Jan. "The Rk Page". The Rk Page. Retrieved 28 December 2016.
26. ^ "Matthew Correlation Coefficient". scikit-learn.org.
27. ^ Stoica P and Babu P (2024), Pearson–Matthews correlation coefficients for binary and multinary classification, Elsevier Signal Processing, 222, 109511, doi = https://doi.org/10.1016/j.sigpro.2024.109511

v t e Statistics
Outline Index
Descriptive statistics Continuous data Center Mean Arithmetic Arithmetic-Geometric Contraharmonic Cubic Generalized/power Geometric Harmonic Heronian Heinz Lehmer Median Mode Dispersion Average absolute deviation Coefficient of variation Interquartile range Percentile Range Standard deviation Variance Shape Central limit theorem Moments Kurtosis L-moments Skewness Count data Index of dispersion Summary tables Contingency table Frequency distribution Grouped data Dependence Partial correlation Pearson product-moment correlation Rank correlation Kendall's τ Spearman's ρ Scatter plot Graphics Bar chart Biplot Box plot Control chart Correlogram Fan chart Forest plot Histogram Pie chart Q–Q plot Radar chart Run chart Scatter plot Stem-and-leaf display Violin plot
Data collection Study design Effect size Missing data Optimal design Population Replication Sample size determination Statistic Statistical power Survey methodology Sampling Cluster Stratified Opinion poll Questionnaire Standard error Controlled experiments Blocking Factorial experiment Interaction Random assignment Randomized controlled trial Randomized experiment Scientific control Adaptive designs Adaptive clinical trial Stochastic approximation Up-and-down designs Observational studies Cohort study Cross-sectional study Natural experiment Quasi-experiment
Statistical inference Statistical theory Population Statistic Probability distribution Sampling distribution Order statistic Empirical distribution Density estimation Statistical model Model specification Lp space Parameter location scale shape Parametric family Likelihood (monotone) Location–scale family Exponential family Completeness Sufficiency Statistical functional Bootstrap U V Optimal decision loss function Efficiency Statistical distance divergence Asymptotics Robustness Frequentist inference Point estimation Estimating equations Maximum likelihood Method of moments M-estimator Minimum distance Unbiased estimators Mean-unbiased minimum-variance Rao–Blackwellization Lehmann–Scheffé theorem Median unbiased Plug-in Interval estimation Confidence interval Pivot Likelihood interval Prediction interval Tolerance interval Resampling Bootstrap Jackknife Testing hypotheses 1- & 2-tails Power Uniformly most powerful test Permutation test Randomization test Multiple comparisons Parametric tests Likelihood-ratio Score/Lagrange multiplier Wald Specific tests Z-test (normal) Student's t-test F-test Goodness of fit Chi-squared G-test Kolmogorov–Smirnov Anderson–Darling Lilliefors Jarque–Bera Normality (Shapiro–Wilk) Likelihood-ratio test Model selection Cross validation AIC BIC Rank statistics Sign Sample median Signed rank (Wilcoxon) Hodges–Lehmann estimator Rank sum (Mann–Whitney) Nonparametric anova 1-way (Kruskal–Wallis) 2-way (Friedman) Ordered alternative (Jonckheere–Terpstra) Van der Waerden test Bayesian inference Bayesian probability prior posterior Credible interval Bayes factor Bayesian estimator Maximum posterior estimator
Correlation Regression analysis Correlation Pearson product-moment Partial correlation Confounding variable Coefficient of determination Regression analysis Errors and residuals Regression validation Mixed effects models Simultaneous equations models Multivariate adaptive regression splines (MARS) Template:Least squares and regression analysis Linear regression Simple linear regression Ordinary least squares General linear model Bayesian regression Non-standard predictors Nonlinear regression Nonparametric Semiparametric Isotonic Robust Homoscedasticity and Heteroscedasticity Generalized linear model Exponential families Logistic (Bernoulli) / Binomial / Poisson regressions Partition of variance Analysis of variance (ANOVA, anova) Analysis of covariance Multivariate ANOVA Degrees of freedom
Categorical / multivariate / time-series / survival analysis Categorical Cohen's kappa Contingency table Graphical model Log-linear model McNemar's test Cochran–Mantel–Haenszel statistics Multivariate Regression Manova Principal components Canonical correlation Discriminant analysis Cluster analysis Classification Structural equation model Factor analysis Multivariate distributions Elliptical distributions Normal Time-series General Decomposition Trend Stationarity Seasonal adjustment Exponential smoothing Cointegration Structural break Granger causality Specific tests Dickey–Fuller Johansen Q-statistic (Ljung–Box) Durbin–Watson Breusch–Godfrey Time domain Autocorrelation (ACF) partial (PACF) Cross-correlation (XCF) ARMA model ARIMA model (Box–Jenkins) Autoregressive conditional heteroskedasticity (ARCH) Vector autoregression (VAR) (Autoregressive model (AR)) Frequency domain Spectral density estimation Fourier analysis Least-squares spectral analysis Wavelet Whittle likelihood Survival Survival function Kaplan–Meier estimator (product limit) Proportional hazards models Accelerated failure time (AFT) model First hitting time Hazard function Nelson–Aalen estimator Test Log-rank test
Applications Biostatistics Bioinformatics Clinical trials / studies Epidemiology Medical statistics Engineering statistics Chemometrics Methods engineering Probabilistic design Process / quality control Reliability System identification Social statistics Actuarial science Census Crime statistics Demography Econometrics Jurimetrics National accounts Official statistics Population statistics Psychometrics Spatial statistics Cartography Environmental statistics Geographic information system Geostatistics Kriging
Category Mathematics portal Commons WikiProject

v t e Machine learning evaluation metrics
Regression	MSE MAE sMAPE MAPE MASE MSPE RMS RMSE/RMSD R2 MDA MAD
Classification	F-score P4 Accuracy Precision Recall Kappa MCC AUC ROC Sensitivity and specificity Logarithmic loss
Clustering	Silhouette Calinski–Harabasz index Davies–Bouldin index Dunn index Hopkins statistic Jaccard index Rand index Similarity measure SMC DBCV index
Ranking	MRR NDCG AP
Computer vision	PSNR SSIM IoU
NLP	Perplexity BLEU MAUVE
Deep learning	Inception score FID
Recommender system	Coverage Intra-list similarity
Similarity	Cosine similarity Euclidean distance Pearson correlation coefficient
Confusion matrix