Chương 14 Phân Tích Lực Thống Kê - Bookdown

14.1 Fixed-Effect Model

To determine the power of a meta-analysis under the fixed-effect model, we have to specify a distribution which represents that our alternative hypothesis is correct. To do this, however, it is not sufficient to simply say that \(\theta \neq 0\) (i.e. that some effect exists). We have to assume a specific true effect that we want to be able to detect with sufficient (80%) power. For example SMD = 0.29.

We already covered previously (see Chapter 8.1.2) that dividing an effect size through its standard error creates a \(z\) score. These \(z\) scores follow a standard normal distribution, where a value of \(|z| \geq\) 1.96 means that the effect is significantly different from zero (\(p<\) 0.05). This is exactly what we want to achieve in our meta-analysis: no matter how large the exact effect size and standard error of our result, the value of \(|z|\) should be at least 1.96, and thus statistically significant:

\[\begin{equation} z = \frac{\theta}{\sigma_{\theta}}~~~\text{where}~~~|z| \geq 1.96. \tag{14.2} \end{equation}\]

The value of \(\sigma_{\theta}\), the standard error of the pooled effect size, can be calculated using this formula:

\[\begin{equation} \sigma_{\theta}=\sqrt{\frac{\left(\frac{n_1+n_2}{n_1n_2}\right)+\left(\frac{\theta^2}{2(n_1+n_2)}\right)}{K}} \tag{14.3} \end{equation}\]

Where \(n_1\) and \(n_2\) stand for the sample sizes in group 1 and group 2 of a study, where \(\theta\) is the assumed effect size (expressed as a standardized mean difference), and \(K\) is the total number of studies in our meta-analysis. Importantly, as a simplification, this formula assumes that the sample sizes in both groups are identical across all included studies.

The formula is very similar to the one used to calculate the standard error of a standardized mean difference, with one exception. We now divide the the standard error by \(K\). This means that the standard error of our pooled effect is reduced by factor \(K\), representing the total number of studies in our meta-analysis. Put differently, when assuming a fixed-effect model, pooling the studies leads to a \(K\)-fold increase in the precision of our overall effect69.

After we defined \(\theta\) and calculated \(\sigma_{\theta}\), we end up with a value of \(z\). This \(z\) score can be used to obtain the power of our meta-analysis, given a number of studies \(K\) with group sizes \(n_1\) and \(n_2\):

\[\begin{align} \text{Power} &= 1-\beta \notag \\ &= 1-\Phi(c_{\alpha}-z)+\Phi(-c_{\alpha}-z) \notag \\ &= 1-\Phi(1.96-z)+\Phi(-1.96-z). \tag{14.4} \end{align}\]

Where \(c_{\alpha}\) is the critical value of the standard normal distribution, given a specified \(\alpha\) level. The \(\Phi\) symbol represents the cumulative distribution function (CDF) of a standard normal distribution, \(\Phi(z)\). In R, the CDF of the standard normal distribution is implemented in the pnorm function.

We can now use these formulas to calculate the power of a fixed-effect meta-analysis. Imagine that we expect \(K=\) 10 studies, each with approximately 25 participants in both groups. We want to be able to detect an effect of SMD = 0.2. What power does such a meta-analysis have?

# Define assumptions theta <- 0.2 K <- 10 n1 <- 25 n2 <- 25 # Calculate pooled effect standard error sigma <- sqrt(((n1+n2)/(n1*n2)+(theta^2/(2*n1+n2)))/K) # Calculate z z = theta/sigma # Calculate the power 1 - pnorm(1.96-z) + pnorm(-1.96-z) ## [1] 0.6059151

We see that, with 60.6%, such a meta-analysis would be underpowered, even though 10 studies were included. A more convenient way to calculate the power of a (fixed-effect) meta-analysis is to use the power.analysis function.

The “power.analysis” Function

The power.analysis function is included in the {dmetar} package. Once {dmetar} is installed and loaded on your computer, the function is ready to be used. If you did not install {dmetar}, follow these instructions:

1. Access the source code of the function online.
2. Let R “learn” the function by copying and pasting the source code in its entirety into the console (bottom left pane of R Studio), and then hit “Enter”.
3. Make sure that the {ggplot2} package is installed and loaded.

The power.analysis function contains these arguments:

• d. The hypothesized, or plausible overall effect size, expressed as the standardized mean difference (SMD). Effect sizes must be positive numeric values.

• OR. The assumed effect of a treatment or intervention compared to control, expressed as an odds ratio (OR). If both d and OR are specified, results will only be computed for the value of d.

• k. The expected number of studies included in the meta-analysis.

• n1, n2. The expected average sample size in group 1 and group 2 of the included studies.

• p. The alpha level to be used. Default is \(\alpha\)=0.05.

• heterogeneity. The level of between-study heterogeneity. Can either be "fixed" for no heterogeneity (fixed-effect model), "low" for low heterogeneity, "moderate" for moderate-sized heterogeneity, or "high" for high levels of heterogeneity. Default is "fixed".

Let us try out this function, using the same input as in the example from before.

library(dmetar) power.analysis(d = 0.2, k = 10, n1 = 25, n2 = 25, p = 0.05) ## Fixed-effect model used. ## Power: 60.66%