Keywords: sample size planning, clinical study, clinical trial
Secondary keywords: sample size, sample size calculation
1 Introduction
When planning a clinical trial, sample size planning plays an important role. This determines how many test subjects must be included in order to demonstrate a relevant effect - and thus ultimately the success or failure of a study. What considerations play a role here?
To demonstrate the effectiveness of any clinical trial, e.g. B. PMCF or in registration studies, hypotheses are tested based on a primary endpoint. A hypothesis to be proven (called alternative hypothesis) can e.g. B. the superiority of a product over a standard therapy. The confirmation or rejection of a hypothesis is assessed based on collected data and the results are then transferred to the population, i.e. to the entire target group. For this to be meaningful, there must be enough data from observations from the target group. If there are too few observations, actual treatment effects cannot be demonstrated and the study fails. On the other hand, a large sample size leads to high costs, is difficult to justify ethically, ties up resources and extends the duration of the study.
The sample size planning is used to determine the minimum number of patients or test subjects to be included in order to prove an actual effect. A number of preliminary considerations are crucial for this.
2. Reasons for case number planning
The aim of every confirmatory clinical trial is to statistically prove a hypothesis. If the sample size is too small, an actual difference between two treatment groups cannot be demonstrated. The result is a non-significant statistical test, even though effects actually exist.
On the other hand, data collection requires a lot of time, human resources are tied up and costs arise for each additional patient included. If too many patients are recruited, this also leads to even small, medically irrelevant effects being detected.
Sample number planning for a clinical trial ensures that:
- An effect present in the target group is detected with the statistical test, i.e. the test delivers a significant result
- If the statistical test does not show a significant result, a sufficient sample size ensures that there is no effect in the target group (population) with a sufficiently high degree of certainty.
The need for sample size planning in the planning phase of clinical trials is also required by law and is reviewed by the ethics committee. Calculating the sample size is an essential part of the clinical trial plan and the statistical analysis plan.
For prospective study designs, sample size planning before the start of the study is essential, but in pilot studies or retrospective studies it should also be considered in advance how high the minimum sample size must be.
Aspects of caseload planning
Doctors, investigators, statisticians and CRO work closely together when planning the number of cases. The starting point is always the primary endpoint and the hypothesis of the clinical study to be tested.
3. Selection of statistical test
On the one hand, the type of question is essential for selecting the appropriate statistical test. Depending on whether superiority or equivalence of a treatment is to be demonstrated, different testing procedures are required. The scale level of the primary target variable also plays a crucial role. Different methods are used for nominal characteristics (yes/no, success/no success) than for ordinal (e.g. Likert scale) or continuous characteristics (e.g. visual analogue scale (VAS), sum scores, etc.).
3.1 Effect size
The effect size indicates the relevant difference to be detected. Depending on the test method used, different measurements are used. The best-known effect size for continuous variables is Cohen's d, which indicates the difference between two independent groups in relation to the common variance.
For categorical endpoints, the effect size W is used, which is the root of the squared relative difference in proportions.
According to Cohen (1988), the following general rules of thumb apply:
Effect size ≈ 0.2: small effect
Effect size ≈ 0.5: medium effect
Effect size ≈ 0.8: large effect
To determine the effect size, preliminary information that is as precise as possible from the literature or our own pilot studies is required. The medically and practically relevant difference that can be proven is also taken into account. A mean blood pressure reduction of a few mmHg, i.e. a very small effect size, can be statistically proven with a sufficiently large sample size, but is practically irrelevant for patients and doctors.
3.2 Significance level of the statistical test
The significance level a must be determined in advance and written down in the study protocol and the statistical analysis plan (SAP). The significance level indicates the probability of obtaining a statistically significant test result if there is actually no effect in the target group. A further distinction is made as to whether the test is carried out on one or two sides. One-sided tests test superiority hypotheses. Two-sided questions that compare the difference between two therapies are common. The value a = 5% has been established as the significance level; if the question is one-sided, a = 2.5% is often used.
3.3 Power or might
The power of the study is also determined in the planning phase. This refers to the probability that a statistical test will prove the actual difference, i.e. deliver a significant p-value. The power of a study should therefore be as high as possible. Values between 80% and 90% are common here. The higher the power of a study, the higher the resulting number of cases.
4. Example from our NOVUSTAT consulting practice
As part of a clinical trial, the improvement in quality of life, measured by the score of the “Physical Functioning” scale of the SF-36 questionnaire, should be demonstrated after 3 months of therapy. The scale ranges from 0 to 100 points. The measuring instrument is well documented, validated and there are numerous publications with this measuring instrument. From the standard value table of the Federal Health Survey [1] it can be seen that healthy people in the age range 40-70 years have a mean score of 80-90 with a standard deviation of around 20 score points. For the study population, this physical function at inclusion (before therapy) will be 50 score points (standard deviation 25 score points), as results of a pilot study have shown. After three months of therapy, the goal is to achieve an improvement in physical functioning by 30 score points, so that the average functional ability after therapy corresponds to that of healthy people of the same age. A low value of 0.2 is expected for the correlation between the first measurement before therapy and the second measurement after 3 months of therapy (and confirmed with the data from the pilot study) due to the time interval.
If you enter these values into G*Power, a software for calculating the sample size, you get the following result:
Fig. 1 Calculating the effect size
Based on the information and prior information, you get an effect size of 0.949, i.e. around 1. This information is now needed to calculate the minimum sample size required to detect an effect of d = 0.949.
For a normally distributed characteristic, a two-tailed t-test for paired samples can be used to demonstrate this. With a 5% significance level and a power of 90%, at least 14 observations are required for detection (see Figure 2).
Fig. 2 Sample size calculation for a two-tailed t-test with paired samples.
Taking into account a drop-out rate of 10%, at least 1.1*14 = 15.4, i.e. 16 patients, must be recruited.
As part of a sensitivity analysis, we check how sensitively the number of cases reacts to deviations from the assumptions. On the one hand, the effect size can be varied within reasonable limits, and on the other hand, the sample size can also be carried out using a non-parametric alternative. Reducing the power results in a reduction in the number of cases required.
A graphical sensitivity analysis can be seen in Figure 3.
Fig. 3 Sensitivity analysis: number of cases depending on the effect size and power of the study
5. Sources/literature
- Case number planning in clinical trials
- Chow S, Shao J, Wang H. 2008. Sample Size Calculations in Clinical Research. 2nd Ed. Chapman & Hall/CRC Biostatistics Series.
- Bock J., Determining sample sizes for biological experiments and controlled clinical trials. Oldenbourg 1998
6. What we can do for you
Before the start of a clinical trial, sample size planning is an important part of the preparation. The sample number calculation ensures that the actual effect can be proven. Professional sample size planning ensures that the sample size remains as small as possible. The sample size planning is tailored to the respective test, taking into account the study design, the primary target variable, the hypothesis to be proven and the required security.
That's why our study planning basically and always includes case number planning as a first step. The entire study concept is based on this. And thus further planning (e.g. How many test centers are needed? How long do I need to recruit? etc.) can build on this.
At this point we would like to thank our partner Novustat for the guest article, as we think that this topic in particular is often underestimated.
About the author: "Dr. Robert Grünwald has been self-employed with the statistical consultancy Novustat for 6 years and and his team primarily advises customers from the pharmaceutical, medical technology and industrial sectors on all questions relating to statistical evaluations."
Statistics consultancy Novustat
8. Outlook
In one of the next blog posts we will take up the topic of “study types” again and take a closer look at the approval study according to MDR Article 62.
9. How we can help you
At medXteam we clarify whether and if so which clinical trial needs to be carried out under what conditions and according to what requirements during the pre-study phase: In 3 steps we determine the correct and cost-effective strategy in relation to the clinical trial required in your case Data collection.
Do you already have some initial questions?
You can get a free initial consultation here: free initial consultation
[1] https://www.thieme.de/statics/documents/thieme/final/de/documents/zw_das-gesundheitswesen/gesu-suppl_klein.pdf