Study design (in studies using statistics)

You should cite this article if you use its information in other circumstances. An example of citing this article is:
Ronny Gunnarsson. Study design (in studies using statistics) [in Science Network TV]. Available at: Accessed January 16, 2018.


This text was first put up on our Swedish sister site. It has been revised many times since
first appearing in 1999.

Relaxed study design? (Photo by Ergoquest)

The content on this page is relevant if you engage in a research project using an empiric-atomistic approach (any study using statistics). The equivalent information for empiric-holistic approach (not using any statistics) is presented on the page introduction to gualitative studies.

Start early to think about study design before submitting an application to the ethics committee. Reading this page will give you a bird’s perspective of the different options and their pros and cons. Pilot studies is a separate topic and is dealt with on the page Pilot studies (feasibility studies).

A brief overview of research design

  1. Observational studies = Non experimental studies
    1. Prospective studies (observations does not exists and needs to be collected – data on an individual level can be collected)
      1. Longitudinal studies = Cohort studies (One or several groups are followed to see what happens. An example would be to follow one group of non-smokers and compare their risk for lung cancer with another cohort of smokers.)
      2. Cross-sectional studies
    2. Retrospective studies (observations on an individual level already exists in databases, charts or archives. They just needs to be compiled – data on an individual level can be collected)
      1. Longitudinal studies
        1. Case-control studies (Smokers and non-smokers are followed in a cohort study to see if there later is a difference in risk for lung cancer. In a case-control study it is the other way around. The proportion of smokers are compared between patients with  diagnosed lung cancer and a group of healthy individuals. Hence you start with the disease and look for exposition.)
        2. Historic cohort studies (You have a database or register enabling you to identify people who in the past were smokers and non-smokers. You compare this with other existing data to see if there is a difference in risk for lung cancer. Hence, a historic cohort study is similar to a cohort study with the difference that all data already resides somewhere in a historic cohort study.)
      2. Cross-sectional studies
    3. Ecological studies (data on an individual level does not exist, only aggregated data for large groups of individuals)
      1. Geographical (comparing health and/or exposure between geographical areas)
      2. Longitudinal (assessing changes in health and/or other confounding factor over time in one population)
      3. Migration (focusing on health and/or exposure in different population types by studying migrant populations)
  2. Experimental studies (are always prospective and longitudinal – data on an individual level can be collected)
    1. Single Case Research Experimental Design – SCRED = zero factor design (Intervention in one group with no comparison group.)
    2. Controlled Clinical Trial – CCT (Using a comparison group but without randomization.)
    3. Randomized Controlled Trial – RCT (Using randomization to intervention or control.)
      1. One factor design (only one factor used for group allocation)
        1. Unmatched groups (most common with only two groups)
        2. Matched pairs design
        3. Cross-over design
      2. N-factor design (two or more factors used for group allocation)
        1. Unmatched N-factor design
        2. Matched N-factor design = Block trial
        3. Latin square (cross-over for an N-factor design)

An individual is only observed once in a cross-sectional study in contrast to longitudinal studies where the same individual is observed (measured) more than one time with a short or long time period in between. A non randomized controlled clinical trial done prospectively should be labelled controlled clinical trial (CCT). If it is done retrospectively it would be logical to label it a historic cohort study.

Observational studies

There is no attempt to tamper with the reality in observational studies (no intervention). There are different types of study design within observational studies (see brief overview above).

Case studies / case series

One or a few patients are described without using any inferential statistics. This is an observational study that most often is retrospective in its nature.

Cohort studies

Following a group of individuals over a period of time (often a long period) to see how disease develops is labelled a cohort study. It is common to follow several different groups (called cohorts) to see if there is any difference between the different groups. For example, smokers compared with nonsmokers.

Case-Control studies

In case-control studies a group of individuals with a particular disease, such as lung cancer, and their exposure to something, such as smoking, is compared with a control group that do not have the disease. Case-control studies can be matched or unmatched. A cohort study usually provides more reliable conclusions than a case-control study.

Historic cohort studies

The historical cohort study is similar to the case-control study, but the difference is that this is based on a group of individuals with a particular feature, such as being a smoker, following them to see what happened to them. What percentage of them develop lung cancer? Case-control studies are based on a group of individuals with a particular disease (or other outcome) and we are looking for an association to different exposures. A historical cohort study do the opposite and is based on individuals exposed to a risk factor. The most common scenario where the historic cohort design is used is the retrospective chart review. This design has a few pitfalls, the most dangerous one is probably Simpson’s statistical paradox.

Ecological studies

In some situations there are no individual data, only aggregated data for large groups such as prevalence, incidence etc . These data are often already compiled and published. Hence accessing data is usually relatively simple and cheap. Data is usually analysed using regression techniques to adjust for confounding factors, preferably using multilevel techniques . Despite this ecological studies have potential problems unique for ecological studies and is named the ecological fallacy .

Experimental studies

Experimental studies are always prospective longitudinal. In most experimental studies (also in SCRED) individuals are in one sense their own controls, that is the statistics is calculated on the individual’s change in an outcome variable rather than measurements at last follow up. We do not label this as a matched pairs design.

The most common variant of experimental study is the one factor design using randomization between one study group and one control group.  Patients are randomized to one of two groups in this scenario. The greater the number of patients included, the less random variation (and the greater the accuracy of the statistics). Matched pairs design is often a little better than an unmatched study but at the cost of much more complicated administration. Matched pairs design may be appropriate if all individuals are collected at once, instead of consecutively being included in the study, one by one where matched pairs design is less suitable.

We talk about studies using zero factor design, one factor design or n factor design. This relates to the number of variables used to determine group allocation. A zero factor design does not have a variable for group allocation because all participants belong to the one and only group. The common situation where participants are randomised to one of two treatment groups is a one factor design.

Let us take an example. We aim to evaluate a physiotherapy intervention for patients (intervention group = IG) just exposed to a whiplash trauma compared to a control group (CG) . Let us assume that we also want to evaluate if it matters if the patient get the treatment early or with some delay. This would give four different groups; group 1 (IG early), group 2 (CG early), group 3 (IG late) and group 4 (CG late). We could create one single variable and for each patient state the group allocation 1-4. Doing so would analyse data as a one factor design even if we have four separate treatment groups. The other alternative would be to create one variable for treatment (IG or CG) and one variable for timing (early or late). By doing the latter we could analyse data using a two-factor design. There are a few significant advantages by using a two-factor design rather than a one factor design in this case:

  • A two factor design would allow estimation of interaction between intervention and timing. Is the combination of IG and early as if 1+1=2 or could it be like 1+1=5?
  • A two factor design uses the data better and would give a more reliable answer to the question if type of intervention or timing matters.

I guess you now see that the number of variables used for group allocation decides the N in N-factor design. A one or two factor design is not very complicated but anything more than a two-factor design usually requires collaboration with statisticians and other personnel with prior experience of N-factor design.

Single Case Research Experimental Design – SCRED

Single Case Research Experimental Design (SCRED) is also known as Single Subject Design or Single-case experimental design (SCED). In SCRED all individuals receive the same intervention. SCRED is not a cross-over study in which treatment options are compared.

SCRED is useful in intervention studies where it is very difficult to recruit sufficient numbers of patients for a randomized controlled trial such as when studying rare diseases. A randomized controlled trial is always a much better option if there are enough participants. SCRED should not be used simply because you don’t have the resources to do a proper randomized controlled trial.

One common cause of systematic errors in SCRED is if the outcome variable is not stable over time such as in diseases with a substantial spontaneous healing or in children that by nature always change. Hence, SCRED is especially unsuitable in children and when you want to study a disease that is not stable over a reasonable time period.

Scred can be made as AB, ABA and ABAB design. With A referring to a period without intervention and B a period of intervention. The more periods / cycles showing a change only in the B-periods, the more likely that this changes is actually caused by the intervention. An intervention with lasting effect has as a consequence that the deterioration is not seen in a A period that follows a B period. If you see a continued improvement in an A period following a B period it implies that the improvement is part of a spontaneous recovery rather than caused by the intervention.

There are two traditions when evaluating the results of a SCRED. One involves making a graph with a line for each individual. On the y-axis is the variable of interest and on the x-axis time. You then look at the lines and decide if the trend indicates improvement, deterioration or no change in the B-periods. A more accurate method is to calculate the individual change between the various periods and then with appropriate statistical tests decide if the change during B periods are statistically significant. The latter method is considered safer than mere visual inspection of a chart. You should make a prior estimation of sample size if you decide to evaluate the effect with statistical tests. It is important to report these studies properly, preferably following SCRIBE .

Clinical trials

A clinical trial is a pre-planned clinical study of the safety, efficacy and optimal dosing schedule of one or more diagnostic, therapeutic or prophylactic drugs, devices, or techniques, performed on humans selected according to predefined criteria and observed for predefined evidence of favorable or unfavorable effects. It may also be used for veterinary studies that meet the above criteria. A clinical trial is an experimental study, even if you rarely use the latter term.
Clinical trials are trials usually divided into Phase I, II, III and IV trials. Phase I is the first time a drug is tested on humans, usually a small group of healthy individuals. Phase II is when testing the agent on a larger group of healthy people (a couple of hundreds) and often also on a small group of patients, among other things, to see which dose is best to use in further studies. Phase III is when a large group of patients is being enrolled and the outcome is compared with a control group. Phase I trials often last around one year. Phase II trials for about two years. Phase III trials are longer, often for three years or more. Phase IV studies are conducted after the product is approved for general sale to get a better grip on efficacy and less common side effects.
What makes it all a bit messy is that a randomized controlled study, sometimes at the same time can be classified as an experimental study, a clinical trial, an intervention study and an epidemiological study (see below).

Cross-over and latin square

Random variation are like dirt on your glasses. It means that observations spread out from the group mean and it makes it harder to see the details that are there (such as a difference between groups). Reducing random variation increases the chances of detecting something that is there. We have several types of random variation such as variation within individuals, variation between individuals and random variation in measurements. The variation between individuals is often the largest random variation in most scenarios.

One way to remove the inter-individual variation is to have the same individual in the treatment group and the control group. The same individual may of course not have multiple treatments simultaneously but may take one followed by the other. If all first got active treatment and placebo, then it could be any time-bound phenomena that affect our reading. This phenomenon could result in an incorrect conclusion.

Vitamin C

Imagine that someone wants to investigating vitamin C’s ability to prevent colds. Assume that 100 patients are given two grams of vitamin C daily for six months. Thereafter, patients are without vitamin C for 6 months and the number of colds during this period is recorded and compared with the previous period. If the first period falls during the summer period and the other during the winter period, this can lead to the incorrect conclusion that vitamin C prevents colds. One way to solve this problem is to form pairs of individuals. Randomisation decides who in the couple start with placebo and  the other individual in the matched pair starts with vitamin C. They switch after half time. This is known as a cross-over trial.

If more than two groups are involved, that is if more than one new treatment is to be evaluated we label this cross-over trial a latin square. In Latin square, as for block design with more than two individuals, each block requires one individual more than the number of new treatments to be examined. The additional individual in the block serve as a control and this extra “treatment” is placebo or an established treatment that the new therapies should be compared with.

Stepped wedge cluster randomised trial

The stepped wedge cluster randomised trial is a pragmatic trial where all participants in the end gets the intervention. It makes long term follow up impossible. Early names on this design were “waiting list designs” or “phased implementations” .

Epidemiological studies

The word epidemiology originally comes from the word epidemic. Initially the focus was on infectious diseases. Today epidemiology embraces teaching and research of the occurrence of diseases in different populations and their causal factors. Epidemiological research are studies designed to investigate correlations and if possible to also make hypothesis about causality. Common purposes with epidemiological research are:

  • Establish factors associated with a disease of interest. These factors are labelled risk factors or predictors. Combinations of risk factors can be used to create prediction models predicting presence of a disease. Some kind of regression would usually be used to establish this.
  • Establish if any of the identified risk factors is also a causal factor. If possible also to clarify the exact relation between exposure to the risk factor and subsequent disease.
  • Clarify transmission pathways for infectious diseases.

The most common types of epidemiological studies are case-control studies, cohort studies and cross-sectional studies. Epidemiological studies aiming to establish causality can sometimes be experimental and is then called intervention studies. Hence, epidemiological research can sometimes be experimental although most epidemiological research is observational in its nature.


You should cite this article if you use its information in other circumstances. An example of citing this article is:
Ronny Gunnarsson. Study design (in studies using statistics) [in Science Network TV]. Available at: Accessed January 16, 2018.

Comments are closed.