Effect Size Calculators

Effect Size Calculators

Refer to this page for formulae and citations.

Two groups	ANOVA, OLS & HLM
One-sample	Partial eta-squared (Fixed effects)
Independent-samples	R-squared (OLS)
Paired-samples	Intraclass Correlation Coefficient
Odds/risk/absolute ratios & NNT	HLM / multilevel Pseudo R-squared's

Developed by James Uanhoro, when I was a graduate student within the Quantitative Research, Evaluation & Measurement program @ OSU.

I have run out of resources to sustain fitting the multilevel models, so for now, the ICC and multilevel R-squared sections are down.

All other components of this page have very low computational cost, so they will continue to function for the forseeable future.

One-sample t-test

Inputs

Sample mean:	Population mean:
Sample standard deviation:	Sample size:
Confidence Interval: %
Calculate	Clear

Results (CI using noncentral t distribution)

Cohen's d:
Lower limit on d:	Upper limit on d:
Clear

Formula

where x_bar is the sample mean, mu is the population mean, and s is the sample standard deviation.

See here for additional details.

Independent-samples t-test

Inputs

Sample 1	Sample 2
Mean:	Mean:
Standard deviation:	Standard deviation:
Sample size:	Sample size:
Confidence Interval: %
Calculate	Clear

Results (CI using noncentral t distribution)

Hedges' g (Unbiased):	Lower limit on d:
Conversion from g to r:	Upper limit on d:
Clear

Formula

where x_bar_1 and x_bar_2 are sample means, n_1 and n_2 are sample sizes, SD_1 and SD_2 are sample standard deviations, and N is the sum of n_1 and n_2.

See here for additional details.

Paired-samples t-test

Inputs

Sample 1	Sample 2
Mean:	Mean:
Standard deviation:	Standard deviation:
Number of pairs:	r:
Confidence Interval: %
Calculate	Clear

Average	Repeated measures

Results (CI using noncentral t distribution)

Hedges's g - average (recommended):	Lower limit on d:
Hedges's g - repeated measures:	Upper limit on d:
Clear

Formula

For average:

For repeated measures:

For both:

where x_bar_1 and x_bar_2 are sample means, n_pairs is the number of pairs, SD_1 and SD_2 are sample standard deviations, and r is the correlation between the scores.

See here for additional details.

Odds/risk/absolute ratios & Number needed to treat

Inputs

Outcome Frequency
Yes		No
Treatment
Control
Method (Odds-ratio): Median-unbiased estimation (mid-p)Conditional maximum likelihood estimation (Fisher)Unconditional maximum likelihood estimation (Wald)Small sample adjustment (small)
Method (Relative-risk): Unconditional maximum likelihood estimation (Wald)Small sample adjustment (small)Bootstrap estimation (boot)
Compute relative risk reduction in place of relative risk?: YesNo
Confidence Interval: %
Calculate		Clear

Results

Odds ratio:	Risk ratio/Relative risk:
Lower limit on odds ratio:	Lower limit on risk ratio:
Upper limit on odds ratio:	Upper limit on risk ratio:
Number needed to treat:	Absolute risk:
Clear

Recommendations for methods

If the outcome is negative, such that a reduction is desired, select yes to compute the relative risk reduction (RRR) and the absolute risk reduction (ARR).

I use the short name for the methods (contained in parenthesis in the dropdown menu) in these recommendations. Given a large sample size, the Wald method suffices (Jewell, 2004).

One can use the small method as a diagnostic. If it produces markedly different results in the point estimates and the CI from Wald, then the sample size is not large enough for Wald (Jewell, 2004, p. 85).

When this occurs for the odds ratio, you can use the Fisher method (Jewell, 2004, p. 85), although it may be highly conservative (Agresti, 2013, p. 93). A better alternative might be mid-p, the default option, which is recommended by Agresti (2013, p. 94).

Agresti, A. (2013). Inference for Contingency Tables. In Categorical data analysis (pp. 70-114). Wiley-Interscience.

Jewell, N. P. (2004). Estimation and Inference for Measures of Association. In Statistics for epidemiology (pp. 76-97). Chapman & Hall/CRC.

Partial eta-squared (Fixed effects)

Inputs

F-value:	Confidence Interval: %
Numerator degrees of freedom:	Denominator degrees of freedom:
Calculate	Clear

It is recommended that you use the 90% CI if you have an alpha level of 5%.

Results (CI using noncentral F distribution)

Partial eta-squared:	Lower limit on partial eta-squared:
Partial omega-squared:	Upper limit on partial eta-squared:
Cohen's f:	Lower limit on Cohen's f:
Upper limit on Cohen's f:
Clear

Formula

where F is the F-statistic, and df_1 and df_2 are the numerator and denominator degrees of freedom respectively.

Lower and upper limits on Cohen's f use the same formula for f on lower and upper limits of partial eta-squared.

See here for additional details.

Partial eta-squared and omega-squared calculated here should only be interpreted if all your factors are manipulated not observed (such as gender), and you have no covariates.Additionally, the confidence intervals produced here will differ from the confidence intervals produced in the OLS section. The calculation for the intervals returned here assumes the predictors are planned/fixed as in an experiment.

R-squared (OLS)

Inputs

R-squared:	Confidence Interval: %
Numerator degrees of freedom:	Denominator degrees of freedom:
Calculate	Clear

It is recommended that you use the 90% CI if you have an alpha level of 5%.

Results

Lower limit on R-squared:	Upper limit on R-squared:
Clear

I have run out of resources to sustain fitting the multilevel models, so for now, the ICC and multilevel R-squared sections are down.

Intraclass Correlation Coefficient

Inputs

Export all your variables into a csv file. The first row has to be the variable names - without spaces within variable names. To minimize problems, files should be ASCII and should not contain missing values.

Method:ANOVAREMLFEMLCalculate

Results - CI are always calculated from One-Way ANOVA (95% CI)

Estimate of ICC:
Variance between:	Variance within:
Lower limit on ICC:	Upper limit on ICC:
Clusters analyzed:	Average per cluster (k):
Design effect (DEFF):	Root DEFF (DEFT):
Clear

Formula

where tau_00 is the variance between clusters, sigma_squared is the variance within clusters, a is the number of clusters, and x is the number of cases in each cluster.

See here for additional details.

Note: Average per cluster is less than mean for unbalanced designs.

I have run out of resources to sustain fitting the multilevel models, so for now, the ICC and multilevel R-squared sections are down.

HLM / multilevel Pseudo R-squared

Inputs

Export all your variables into a csv file. The first row has to be the variable names - without spaces within variable names. To minimize problems, files should be ASCII and should not contain missing values.

Optimization Method: Nelder-MeadPowellcgbfgsCalculate

Results

Marginal R-squared:	Conditional R-squared:
Variance between:	Variance within:
Average random effect:	Residual ICC:
Clusters analyzed:	Model converged?:
Clear

Brief background

These are pseudo-R-squared's as they attempt to recreate the properties of R-squared from OLS. These measures achieve those properties to varying degrees.The marginal R-squared attempts to capture the variance explained by the fixed effects in the model, and the conditional R-squared attempts to capture the variance explained by both the fixed effects and random effects.

Formula

      

where R^2m, R^2_c are the marginal and conditional R-squared's respectively, var_re is the average of the random effects variance, sigma_squared is the variance within clusters, and var_fixed is the variance explained by the fixed effects in the model.

See here for additional details.

Check the results for convergence. If they do not converge, try another optimization method from the drop down menu above.

Please contact me with questions and suggestions at [email protected].

Pull requests welcome on repo, where formulae alongside sources can be found.

If you would like to cite this website, you can use the citation below, it's APA. Thank you.

Uanhoro, J. O. (2017). Effect size calculators. Available online at: https://effect-size-calculator.herokuapp.com/.