Ronny Gunnarsson. Sample size estimation [in Science Network TV]. Available at: http://science-network.tv/sample-size-estimation/. Accessed November 19, 2017.

In a research project analysing numbers you first collect data and then process them before statistical calculations (=inferential statistics) are made. The statistical calculations looks at your data and produces results such as p-values, odds ratios, hazard ratios, etc. Is the reason for not reaching statistical significance that there are no correlation / no difference between groups or is the reason that your sample size was too small. To avoid ending up with the latter problem it is recommended to do a sample size estimation before data collection is made. It has gradually become more common that ethics committees require a sample size estimation before approving a project.

Table of Contents

- 1 Different approaches to sample size estimation
- 2 Videos showing examples of sample size calculations using G*Power (click to watch)
- 3 Videos showing examples of sample size calculations using PASS (click to watch)
- 4 Unadjusted or adjusted multivariate sample size calculation
- 5 The difference between level of significance (alpha) and the p-value
- 6 Sample size estimation in clustered studies
- 7 Example of how to write the sample size section in a study protocol (click to read)
- 8 Useful links
- 9 References

## Different approaches to sample size estimation

- Get a convenient sample and hope it is enough
- See how many observations other published projects included and imitate them
- Follow a rule of thumb
- Make a calculation based on your best assumptions.

Hope is good in many situations except this one. Imitate others is also not a good advice. What if the others did an underpowered study? Why replicate their mistake? There are some rules of thumb such as:

- For group comparisons of means (t-test) have at least 30 in each group.
- For group comparisons of proportions (chi-square) have at least 5 in each cell.
- For standard linegressions/correlations have at least 20 observations for each independent variable.
- For logistic regression have have at least 10 times more events / end points than independent variables .
- For Cox regression have at least 10 times more events / end points than independent variables . For example: you have four independent predictor variables in the model and the proportion of positive cases in the population is expected to be 0.30 (30%) the minimum number of cases required would be 133.

However, these rule of thumb are quite rudimentary because they do not consider the magnitude of the effect size or correlation you are looking for. They just give the bare minimum number you should have to avoid violating underlying mathematical assumptions but they do not consider your particular situation. The best approach to estimate the size of the sample is to do a proper sample size calculation considering the situation in your study. This is done by first making four important decisions:

- Decide what statistical method is going to be used for the inferential statistics.
- Decide what effect size / correlation you are looking for. It is best if this can be estimated using data from previous publications. You have to make a qualified guess if no prior publications exists.
- Decide what would be an acceptable safety margin to avoid doing a type one error (claiming a statistical finding that is not true). This safety margin is labelled alpha or level of significance and is commonly set to 0.05. This means that you have a one in twenty chance of doing a type one error.
- Decide what power your study should have. This is the same as the inverse of the risk of doing a type two error (not identifying an effect/correlation that is true). The power is often set to something between 0.80-0.95 which corresponds to a 5-20% chance of doing a type two error.

The rest is quite easy once we have made these four decisions. We put in our decisions in a software that does the statistical calculation backwards and states how large sample we need. Example of such software are G*Power and PASS. G*Power is free but PASS is quite expensive. G*Power can manage most situations except Cox regression.

## Videos showing examples of sample size calculations using G*Power (click to watch)

Example 1 of sample size calculation for comparing two groups – T-test and Mann-Whitneys test## Videos showing examples of sample size calculations using PASS (click to watch)

Example of sample size calculation for Cox regression## Unadjusted or adjusted multivariate sample size calculation

You may plan for a multivariate regression as your preferred final statistical analysis. There are a few approaches to this situation:

- Make one sample size calculation for each independent variable as if you are going to do univariate (unadjusted) regressions. You will get one sample size for each independent variable. Pick the one with the highest number as your preferred sample size (and perhaps add a margin of 20% extra). This is the most common strategy and the one used in the videos above.
- In case you are only interested in one independent variable and want to add a few more only to adjust for them (as confounding variables) try to estimate the contribution from the covariates (R square other X in G*Power) and add it in G*Power together with the expected information around your main independent variable to calculate the sample size required. Finding the right value on the “R square other X” is tricky and might be impossible. Either make a reasonable guess or go with strategy a above.
- There may be many independent variables in an exploratory study and none are initially more important than another. The simplest solutions is to use strategy a above. It may be difficult to sort out how the variables may relate in a multivariate model without making a lot of guesses.
- Calculating sample size for interaction variables in a regression is tricky for two reasons. The first one it is often difficult to find support for the assumptions you need to make so you may be left with some wild guessing. Secondly you would need more advanced software than G*Power and a statistician who has experience of this advanced calculation (not all statisticians would have that).

## The difference between level of significance (alpha) and the p-value

A low p-value says it is unlikely that we would get the observed observations if the effect / correlation we’re looking for in reality is zero. A low P value indicates that the null hypothesis can be rejected and the alternative hypothesis is the most likely. How low must the p-value be for us to believe that our alternative hypothesis is the most plausible? This should be determined from case to case. Read more about this on the page describing the level of significance or alpha.

## Sample size estimation in clustered studies

Observations are often grouped (clustered). A typical example can be that observations are clustered in different primary health care centres (GP clinics) or different hospitals. These clusters will add a random variation between clusters that makes your vision slightly blurred. It means that you must increase your sample size to maintain your ability to find what you are looking for. It can be shown that it is better to have many clusters contributing with a few observations compared to having a few clusters contributing with many observations. To estimate this calculate the required sample size as if there was no cluster effect. After that use the calculator below to estimate the effect on required sample size different cluster designs will have.

The impact of clusters is measured with Intra Class Correlation (ICC). You need to find a suitable assumption for ICC to put in below. The ideal situation is if you find a publication with a study similar to yours stating the ICC. If that is the case use that. Otherwise make a reasonable guess to estimate ICC. In a hospital setting common values if ICC are 0.02-0.1 . In a primary care setting common estimates of ICC are 0.1-0.2 although estimates up to 0.3 may occasionally be seen .

## Example of how to write the sample size section in a study protocol (click to read)

An observational study establishing prevalence of symptoms## Useful links

- Power and sample size calculators
- Calculator for confidence interval for proportions – 1
- Calculator for confidence interval for proportions – 2
- Power and Sample Size Primer (a video)
- Sample size calculator for non-inferiority studies if the variable is binary
- Sample size calculator for non-inferiority studies if the variable is contonous
- Tips around using G*Power
- Calculating sample size for survival analysis (such as Cox regression)
- Power and sample size calculators from HyLown Consulting LLC
- PS: Power and Sample Size calculator from Vanderbilt University in the US

## References

Ronny Gunnarsson. Sample size estimation [in Science Network TV]. Available at: http://science-network.tv/sample-size-estimation/. Accessed November 19, 2017.