There is an updated version of the EPV-calculator found here: https://infovoice.se/epv-calculator/
| Suggested pre-reading | What this web page adds |
|---|---|
| This web-page will give you an understanding of what pilot studies (feasibility studies) are. It will also clarify the difference between a planned pilot study and an underpowered study. |
This page aim to clarify the difference between a pilot study and other studies. This discussion is most often relevant in relation to studies taking on an empirical-atomistic approach aiming to evaluate the effect of a drug or any other intervention. However, it can be used also for observational studies. The rest of this page will focus the discussion to the situation of pilot studies preparing the way for a larger study to establish effect of an intervention.
It is a common misconception that small underpowered studies trying to prove effect of an intervention compared to placebo or another intervention are pilot studies . This is not true. If they aim to prove effect and are underpowered it means that they either failed to recruit participants up to their pre-calculated target or it was simply a badly designed study, not a pilot study. A pilot study aim to answer questions such as:
- Will a large or small proportion of individuals asked to participate accept participation? Will randomisation make potential participants to hesitate?
- Will the measurement of effect be practical and feasible? Which survey is suitable in this situation? Should a new survey tool be constructed?
- Is the intervention reasonably easy to apply given the resources we could expect to have in a larger study?
- Will a large or small proportion of patients drop out of the study for various reasons?
- What level of random variability and rough changes or effects can be seen after intervention? (to be used as a basis for sample size estimation before the larger adequately powered study)
- How common are unforeseen problems (that may warrant exclusion of a patient or worse termination of the study)?
A pilot study aim to provide answers to some or all of these questions but is not powered to answer the big question is there an effect of the intervention. Hence, the reporting of a true pilot study should completely refrain from trying to reject a null hypothesis concerning possible effect of the studies intervention. Effect size may be calculated only to serve as a guide for a sample size calculation before the final study.
| Pilot study | “The real thing” | |
|---|---|---|
| Sample size calculation before data collection | Rarely (see below) | Should always be done |
| Number of included participants | Usually very small | Usually fairly large |
| Randomization | Often if it is an interventional study | Should be done if the focus is to evaluate effect of something |
| Aim to establish effect | Never ! (but may make a preliminary estimation of effect size to inform a sample size calculation for the final study.) | Establishing / comparing effect is the main aim (may include superiority, equivalence and non-inferiority trials) |
| Aim to establish practical feasibility | Clarifying the practical feasibility is the main aim of a pilot study | Practical feasibility should be sorted out before the real study is done. |
| Inferential statistics used | Is only done to inform a sample size calculation for the coming final study. | Should always be used to answer the research questions. |
Sample size in pilot studies
Various recommendations point to a number between 10-50 or higher. However, a proper sample size calculation can only be done for research question VI above . There is no sensible way to make a proper sample size calculation for research question I-V.
Effectiveness and efficacy
Efficacy is a measure of effect of the intervention under ideal circumstances while effectiveness is a measure of the effect if implemented in reality. Pilot studies does not aim to estimate efficacy or effectiveness.
