Pilot studies (feasibility studies)

You should cite this article if you use its information in other circumstances. An example of citing this article is:
Ronny Gunnarsson. Pilot studies (feasibility studies) [in Science Network TV]. Available at: http://science-network.tv/pilot-studies/. Accessed March 25, 2017.

Chief investigators in a pilot study?
(Photo by Alex Pereslavtsev)

Recommended reading before reading this page:

This page aim to clarify the difference between a pilot study and other studies. This discussion is most often relevant in relation to studies taking on an empirical-atomistic approach aiming to evaluate the effect of a drug or any other intervention. However, it can be used also for observational studies. The rest of this page will focus the discussion to the situation of pilot studies preparing the way for a larger study to establish effect of an intervention.

It is a common misconception that small underpowered studies trying to prove effect of an intervention compared to placebo or another intervention are pilot studies. This is not true. If they aim to prove effect and are underpowered it means that they either failed to recruit participants up to their pre-calculated target or it was simply a badly designed study, not a pilot study. A pilot study aim to answer questions such as:

  • Will a large or small proportion of individuals asked to participate accept participation?
  • Will the measurement of effect be practical and feasible? Which survey is suitable in this situation? Should a new survey tool be constructed?
  • Is the intervention reasonably easy to apply given the resources we could expect to have in a larger study?
  • Will a large or small proportion of patients drop out of the study for various reasons?
  • What level of random variability and rough changes or effects can be seen after intervention? (to be used as a basis for sample size estimation before the larger adequately powered study)

A pilot study aim to provide answers to these questions but is not powered to answer the big question is there an effect of the intervention. Hence, the reporting of a true pilot study should completely refrain from calculating p-values and effect size because they are simply not designed to collect enough data for this purpose.

Pilot study “The real thing”
Sample size calculation before data collection Never Often (should always be done)
Number of included participants Usually very small Usually fairly large
Randomization Sometimes Should be done if the focus is to evaluate effect of something
Aim to establish effect Never ! Establishing / comparing effect is the main aim (may include superiority, equivalence and non-inferiority trials)
Aim to establish practical feasibility Clarifying the practical feasibility is the main aim of a pilot study Practical feasibility should be sorted out before the real study is done.
Inferential statistics used Should rarely be used Should always be used
You should cite this article if you use its information in other circumstances. An example of citing this article is:
Ronny Gunnarsson. Pilot studies (feasibility studies) [in Science Network TV]. Available at: http://science-network.tv/pilot-studies/. Accessed March 25, 2017.

Comments are closed.