This vignette replicates several of the examples found in the G*Power manual (version 3.1). It is not meant to be exhaustive but instead demonstrates how power analyses could be computed and extended using simulation methodology by either editing the default functions found within the package or by creating a new user-defined function for yet-to-be-defined statistical analysis contexts.

Correlation

Power associated with the hypotheses \[H_0:\, \rho-\rho_0=0\] \[H_1:\, \rho-\rho_0\ne 0\]

where \(\rho\) is the population correlation and \(\rho_0\) the null hypothesis constant.

Example 3.3; Difference from constant (one sample case)

Sample size estimate to reject \(H_0:\, \rho_0=.60\) in correlation analysis with \(1-\beta=.95\) probability when \(\rho=.65\).

p_r(n = NA, r = .65, rho = .60) |> 
    Spower(power = .95, interval=c(500,3000))

## 
## Design conditions: 
## 
## # A tibble: 1 × 5
##       n     r   rho sig.level power
##   <dbl> <dbl> <dbl>     <dbl> <dbl>
## 1    NA  0.65   0.6      0.05  0.95
## 
## Estimate of n: 1940.8
## 95% Prediction Interval: [1896.2, 1979.6]

G*power estimates \(n\) to be 1929 using the same Fisher z-transformation approximation.

Test against constant \(\rho_0=0\)

Unlike the previous section, this is the more canonical version of the hypotheses involving correlation coefficients. Power associated with \(\rho = .3\) with 100 pairs of observations, tested against \(\rho_0=0\).

p_r(n = 100, r = .3) |> Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 4
##       n     r sig.level power
##   <dbl> <dbl>     <dbl> <dbl>
## 1   100   0.3      0.05    NA
## 
## Estimate of power (1 - beta): 0.860
## 95% Confidence Interval: [0.854, 0.867]

Sample size estimate to reject \(H_0:\, \rho_0=0\) in correlation analysis with \(1-\beta=.95\) probability when \(\rho=.3\).

p_r(n = NA, r = .3) |> 
    Spower(power = .95, interval=c(50,1000))

## 
## Design conditions: 
## 
## # A tibble: 1 × 4
##       n     r sig.level power
##   <dbl> <dbl>     <dbl> <dbl>
## 1    NA   0.3      0.05  0.95
## 
## Estimate of n: 138.5
## 95% Prediction Interval: [136.2, 140.7]

Compare to approximate result from pwr package.

pwr::pwr.r.test(r=.3, power=.95, n=NULL)

## 
##      approximate correlation power calculation (arctangh transformation) 
## 
##               n = 137.8
##               r = 0.3
##       sig.level = 0.05
##           power = 0.95
##     alternative = two.sided

Example 27.3; Correlation - inequality of two independent Pearson r’s

n <- 206
n2_n1 <- 51/n
p_2r(n=n, r.ab1=.75, r.ab2=.88, n2_n1=n2_n1) |> Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 4
##   r.ab1 r.ab2 sig.level power
##   <dbl> <dbl>     <dbl> <dbl>
## 1  0.75  0.88      0.05    NA
## 
## Estimate of power (1 - beta): 0.730
## 95% Confidence Interval: [0.721, 0.739]

G*power gives power of .726.

Example 16.3; Point-biserial correlation

# solution per group
out <- p_t.test(r = .25, n = NA, two.tailed=FALSE) |> 
    Spower(power = .95, interval=c(50, 200))
out

## 
## Design conditions: 
## 
## # A tibble: 1 × 5
##       r     n two.tailed sig.level power
##   <dbl> <dbl> <lgl>          <dbl> <dbl>
## 1  0.25    NA FALSE           0.05  0.95
## 
## Estimate of n: 80.9
## 95% Prediction Interval: [79.7, 81.9]

# total sample size required
ceiling(out$n) * 2

## [1] 162

G*power gives the result \(N=164\).

Relatedly, one can specify \(d\), Cohen’s standardized mean-difference effect size, instead of \(r\) since \(d\) is easily converted to \(r\).

Example 31.3; tetrachoric correlation

F <- matrix(c(203, 186, 167, 374), 2, 2)
N <- sum(F)
(marginal.x <- colSums(F)/N)

## [1] 0.4183 0.5817

(marginal.y <- rowSums(F)/N)

## [1] 0.3978 0.6022

# converted to intercepts
tauX <- qnorm(marginal.x)[2]
tauY <- qnorm(marginal.y)[2]
c(tauX, tauY)

## [1] 0.2063 0.2589

G*power gives \(n=463\), though uses the SE value at the null (Score test). p_r.cat(), on the other hand, can also use the Wald approach where the SE is at the MLE. To switch, use score=FALSE, though note that this requires twice as many computations.

p_r.cat(n=NA, r=0.2399846, tauX=tauX, tauY=tauY, 
        score=FALSE, two.tailed=FALSE) |> 
    Spower(power = .95, interval=c(100, 500), 
           parallel=TRUE, check.interval=FALSE)

## 
## Design conditions: 
## 
## # A tibble: 1 × 8
##       n     r  tauX  tauY score two.tailed sig.level power
##   <dbl> <dbl> <dbl> <dbl> <lgl> <lgl>          <dbl> <dbl>
## 1    NA 0.240 0.206 0.259 FALSE FALSE           0.05  0.95
## 
## Estimate of n: 462.9
## 95% Prediction Interval: [458.5, 466.6]

Proportions

Example 4.3; One sample proportion tests

One sample, one-tailed proportions test given data generated from a population tested against \(\pi_0 = .65\) with \(g=.15\) (hence, \(\pi = .65+g=.80\)) with \(n=20\).

PI <- .65
g <- .15
p <- PI + g

p_(n=20, prop=p, pi=PI, two.tailed=FALSE) |>
    Spower()

## Error in names(CI) <- paste0("CI_", c(alpha/2, predCI + alpha/2) * 100): 'names' attribute [2] must be the same length as the vector [0]

G*power gives the estimate \(1-\beta=.4112\).

Fisher’s exact test is also supported by using the argument exact = TRUE.

# Fisher exact test
p_(n=20, prop=p, pi=PI, exact=TRUE, 
            two.tailed=FALSE) |> Spower()

## Error in names(CI) <- paste0("CI_", c(alpha/2, predCI + alpha/2) * 100): 'names' attribute [2] must be the same length as the vector [0]

Example 22.1; Sign test

Normal as the parent distribution.

p_wilcox.test(n=649, d=.1, type='one.sample', two.tailed=FALSE) |> 
    Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 6
##       n     d type       two.tailed sig.level power
##   <dbl> <dbl> <chr>      <lgl>          <dbl> <dbl>
## 1   649   0.1 one.sample FALSE           0.05    NA
## 
## Estimate of power (1 - beta): 0.799
## 95% Confidence Interval: [0.791, 0.807]

G*power gives the power estimate of .800.

A one-sample sign test with Laplace distribution as the parent:

library(VGAM)

# generate data with scale 0-1 for d effect size to be same as mean
parent <- function(n, d, scale=sqrt(1/2)) 
    VGAM::rlaplace(n, d, scale=scale)

p_wilcox.test(n=11, d=.8, parent1=parent, type='one.sample',
              two.tailed=FALSE, correct = FALSE) |> Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 7
##       n     d type       two.tailed correct sig.level power
##   <dbl> <dbl> <chr>      <lgl>      <lgl>       <dbl> <dbl>
## 1    11   0.8 one.sample FALSE      FALSE        0.05    NA
## 
## Estimate of power (1 - beta): 0.805
## 95% Confidence Interval: [0.798, 0.813]

G*power gives the estimate .830.

Example 5.3; Two dependent proportions test (McNemar’s test)

obrien2002 <- matrix(c(.54, .32, .08, .06), 2, 2,
                     dimnames = list('Treatment' = c('Yes', 'No'),
                                    'Standard' = c('Yes', 'No')))
obrien2002

##          Standard
## Treatment  Yes   No
##       Yes 0.54 0.08
##       No  0.32 0.06

p_mcnemar.test(n=50, prop=obrien2002, two.tailed=FALSE) |>
        Spower(replications=30000)

## 
## Design conditions: 
## 
## # A tibble: 1 × 4
##       n two.tailed sig.level power
##   <dbl> <lgl>          <dbl> <dbl>
## 1    50 FALSE           0.05    NA
## 
## Estimate of power (1 - beta): 0.840
## 95% Confidence Interval: [0.836, 0.844]

G*Power gives .839 (\(\alpha = .032\)). Slightly more power can be achieved when not using the continuity correction, though in general this is not recommended in practice.

p_mcnemar.test(n=50, prop=obrien2002, two.tailed=FALSE, correct=FALSE) |> 
    Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 5
##       n two.tailed correct sig.level power
##   <dbl> <lgl>      <lgl>       <dbl> <dbl>
## 1    50 FALSE      FALSE        0.05    NA
## 
## Estimate of power (1 - beta): 0.884
## 95% Confidence Interval: [0.878, 0.890]

Multiple Linear Regression (Random IVs)

Example 7.3

p_lm.R2(n=95, R2=.1, k=5, fixed.X=FALSE) |> Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 6
##       n    R2     k fixed.X sig.level power
##   <dbl> <dbl> <dbl> <lgl>       <dbl> <dbl>
## 1    95   0.1     5 FALSE        0.05    NA
## 
## Estimate of power (1 - beta): 0.666
## 95% Confidence Interval: [0.657, 0.675]

G*power gives 0.662, though note that G*power states that this is a one-tailed test.

Multiple Linear Regression (Fixed IVs)

Example 13.1

p_lm.R2(n=95, R2=.1, k=5) |> Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 5
##       n    R2     k sig.level power
##   <dbl> <dbl> <dbl>     <dbl> <dbl>
## 1    95   0.1     5      0.05    NA
## 
## Estimate of power (1 - beta): 0.668
## 95% Confidence Interval: [0.659, 0.677]

G*power gives \(1-\beta = .673\).

Example 14.3

Note that k is total IVs, k.R2_0 is number of IVs for baseline model.

p_lm.R2(n=90, R2=.3, k=9, R2_0=.25, k.R2_0=5) |> Spower(sig.level=.01)

## 
## Design conditions: 
## 
## # A tibble: 1 × 7
##       n    R2     k  R2_0 k.R2_0 sig.level power
##   <dbl> <dbl> <dbl> <dbl>  <dbl>     <dbl> <dbl>
## 1    90   0.3     9  0.25      5      0.01    NA
## 
## Estimate of power (1 - beta): 0.232
## 95% Confidence Interval: [0.224, 0.241]

G*power gives \(1-\beta = .241\). Solving the sample size to achieve 80% power

p_lm.R2(n=NA, R2=.3, R2_0 = .25, k=9, k.R2_0=5) |> 
        Spower(sig.level=.01, power=.8, interval=c(100, 400))

## 
## Design conditions: 
## 
## # A tibble: 1 × 7
##       n    R2  R2_0     k k.R2_0 sig.level power
##   <dbl> <dbl> <dbl> <dbl>  <dbl>     <dbl> <dbl>
## 1    NA   0.3  0.25     9      5      0.01   0.8
## 
## Estimate of n: 243.0
## 95% Prediction Interval: [240.8, 245.3]

G*power gives \(n = 242\).

Example 14.3b

Compare model with 12 IVs to model with 9 IVs.

p_lm.R2(n=200, R2=.16, R2_0 = .1, k=12, k.R2_0=9, R2.resid=.8) |> 
    Spower(sig.level=.01)

## 
## Design conditions: 
## 
## # A tibble: 1 × 8
##       n    R2  R2_0     k k.R2_0 R2.resid sig.level power
##   <dbl> <dbl> <dbl> <dbl>  <dbl>    <dbl>     <dbl> <dbl>
## 1   200  0.16   0.1    12      9      0.8      0.01    NA
## 
## Estimate of power (1 - beta): 0.760
## 95% Confidence Interval: [0.752, 0.768]

G*power gives \(1-\beta = .767\).

Fixed effects ANOVA - One way (F-test)

Example 10.3

p_anova.test(n=NA, k=10, f=.25) |>  Spower(power=.95, interval=c(20, 300))

## 
## Design conditions: 
## 
## # A tibble: 1 × 5
##       n     k     f sig.level power
##   <dbl> <dbl> <dbl>     <dbl> <dbl>
## 1    NA    10  0.25      0.05  0.95
## 
## Estimate of n: 38.1
## 95% Prediction Interval: [37.3, 39.1]

G*power gives the estimate \(n=39\).

Fixing \(n=200\) in total (hence, \(n=200/k=20\)) and performing a compromise analysis assuming \(q=\frac{\beta}{\alpha}=1\),

p_anova.test(n=20, k=10, f=.25) |> Spower(beta_alpha=1, replications=30000)

## 
## Design conditions: 
## 
## # A tibble: 1 × 6
##       n     k     f sig.level power beta_alpha
##   <dbl> <dbl> <dbl>     <dbl> <dbl>      <dbl>
## 1    20    10  0.25        NA    NA          1
## 
## Estimate of Type I error rate (alpha/sig.level): 0.160
## Estimate of Type II error rate (beta): 0.160
## 95% Confidence Interval: [0.155, 0.164]
## 
## Estimate of power (1-beta): 0.840
## 95% Confidence Interval: [0.836, 0.845]

G*Power gives \(\alpha=\beta=0.159\).

Variance tests

Example 26.3; Difference from constant (one sample case)

# solve n for variance ratio of 1/1.5 = 2/3, one.tailed, 80% power
p_var.test(n=NA, sds=sqrt(1), sigma=sqrt(1.5), two.tailed=FALSE) |> 
    Spower(power=.80, interval=c(10, 200))

## Error in which(sapply(expr[-1], function(x) {: argument to 'which' is not logical

G*power gives sample size of 81.

Example 15.3; Two-sample variance test

For a two-sample equality of variance test with equal sample sizes,

# solve n for variance ratio of 1/1.5 = 2/3, two.tailed, 80% power
p_var.test(n=NA, sds=sqrt(c(1, 1.5)), two.tailed=TRUE) |> 
    Spower(power=.80, interval=c(50, 300))

## Error in which(sapply(expr[-1], function(x) {: argument to 'which' is not logical

G*Power gives estimate of 193 per group.

t-tests

Estimate sample size (\(n\)) per group in independent samples \(t\)-test, one-tailed, medium effect size (\(d=0.5\)), \(\alpha=0.05\), 95% power (\(1-\beta = 0.95\)), equal sample sizes (\(\frac{n_2}{n_1}=1\)).

(out <- p_t.test(n = NA, d = .5, two.tailed=FALSE) |>  
               Spower(power = .95, interval=c(10,500)))

## 
## Design conditions: 
## 
## # A tibble: 1 × 5
##       n     d two.tailed sig.level power
##   <dbl> <dbl> <lgl>          <dbl> <dbl>
## 1    NA   0.5 FALSE           0.05  0.95
## 
## Estimate of n: 88.0
## 95% Prediction Interval: [85.5, 90.4]

G*power estimate is 88 per group, Spower estimate is 87.988 with the 95% CI [85.5198, 90.4206].

Example 19.3; Paired samples t-test

p_t.test(n=50, d=0.421637, type = 'paired') |> Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 5
##       n       d type   sig.level power
##   <dbl>   <dbl> <chr>      <dbl> <dbl>
## 1    50 0.42164 paired      0.05    NA
## 
## Estimate of power (1 - beta): 0.847
## 95% Confidence Interval: [0.840, 0.854]

G*power gives power estimate of .832.

Example 20.3; One-sample t-test

p_t.test(n=NA, d=.625, two.tailed=FALSE, type='one.sample') |>  
    Spower(power=.95, interval=c(10, 100))

## 
## Design conditions: 
## 
## # A tibble: 1 × 6
##       n     d two.tailed type       sig.level power
##   <dbl> <dbl> <lgl>      <chr>          <dbl> <dbl>
## 1    NA 0.625 FALSE      one.sample      0.05  0.95
## 
## Estimate of n: 28.7
## 95% Prediction Interval: [28.0, 29.7]

G*power gives sample size of \(n=30\).

p_t.test(n=NA, d=.1, type='one.sample') |>  
    Spower(power=.9,sig.level=.01, interval=c(100,2000))

## 
## Design conditions: 
## 
## # A tibble: 1 × 5
##       n     d type       sig.level power
##   <dbl> <dbl> <chr>          <dbl> <dbl>
## 1    NA   0.1 one.sample      0.01   0.9
## 
## Estimate of n: 1492.1
## 95% Prediction Interval: [1474.4, 1513.8]

G*power gives sample size of \(n=1492\).

Wilcox tests

Example 22.3; One-sample test with normal distribution

p_wilcox.test(n=649, d=.1, type='one.sample', two.tailed=FALSE) |> Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 6
##       n     d type       two.tailed sig.level power
##   <dbl> <dbl> <chr>      <lgl>          <dbl> <dbl>
## 1   649   0.1 one.sample FALSE           0.05    NA
## 
## Estimate of power (1 - beta): 0.809
## 95% Confidence Interval: [0.802, 0.817]

G*power provide power estimate of .800.

Two-sample test with Laplace distributions

library(VGAM)

parent1 <- function(n, d) VGAM::rlaplace(n, d, scale=sqrt(1/2))
parent2 <- function(n, d) VGAM::rlaplace(n, scale=sqrt(1/2))

nr <- 134/67
p_wilcox.test(n=67, n2_n1=nr, d=0.375, parent1=parent1, parent2=parent2) |> 
    Spower()

## 
## Design conditions: 
## 
## # A tibble: 1 × 4
##       n     d sig.level power
##   <dbl> <dbl>     <dbl> <dbl>
## 1    67 0.375      0.05    NA
## 
## Estimate of power (1 - beta): 0.845
## 95% Confidence Interval: [0.838, 0.852]

G*power gives power of .847.

G*Power examples evaluated with Spower

Phil Chalmers

2025-02-21