The quality and significance of scientific studies are often judged by study design and how a given statistical analysis plan has incorporated methods to reduce the risk of bias and unbalanced datasets. This includes the randomization procedures for participants, the inclusion of appropriate control groups, using a well-demonstrated methodology, and recruiting a substantial sample size to establish statistical significance. Broadly speaking, a randomized control trial can be classified as the gold standard study design for intervention-based studies according to the GRADE criteria (Guyett et al., 2008).
Randomized control trial vs Synthetic control arm
In a randomized control trial (RCT) study design, participants are distributed into two or more groups, typically an intervention group (Drug, Exercise, or Therapy) and a control group which can be a placebo or standard-of-care group. In the context of oncology, interventions using a control group across different studies have led to a huge increase in the cost of clinical trials, highlighting the need to find alternative solutions.
Another important consideration for RCT study design and the successful completion of an RCT for the drug or treatment under study is whether participants are not disposed to be administered with the new drug candidate that could possibly cure their disease, being willing to stay within the trial even if they are in this control group. This may lead to higher drop-out rates conflating the risk of bias, reducing the statistical power of the study, and extending the length of a trial with substantial financial implications (Groth 2010).
A synthetic control group derived from patient data may offer a novel and economical solution to these issues. In rare diseases or ever-complex disease phenotypes, it may allow clinical trials to be conducted successfully (Thorlund et al., 2020). This approach based on the generation of Real-World Evidence (RWE) has received backing from the US Food and Drug Agency (FDA), European Medical Agency (EMA), and National Institute for Clinical Excellence in the United Kingdom.
Using a synthetic control arm alongside advanced data mining, such as artificial intelligence (AI) in medical imaging technologies, offers the ability to rapidly decrease clinical trial duration, increase the feasibility of trials, make them more financially viable, and reduce reporting bias.
What is a synthetic control arm?
The synthetic control method can be used to compare the effectiveness of an intervention using external control data. The synthetic control arm can be added as an independent control group or as an adjuvant to a control group. External control arm data may be derived primarily from observational, real-world data (RWD), and previous clinical trial data. The validity and robustness of this approach has already been appropriately analyzed and accepted (Thorlund et al., 2020).
The most significant advantage of a synthetic control group emerges when low numbers of patients with specific inclusion criteria are required as per sample size calculations to evaluate putative differences between the new treatment and the standard-of-care. In this scenario, finding appropriate patients for the control group would be impractical, time-consuming, and expensive. A synthetic control arm can potentially overcome these issues.
The successful use of a synthetic arm depends on the quality of data used, which will be specific for every study. Some considerations to be made when choosing this strategy include:
-
Is the control group relevant to the population being studied?
-
What is the source of the synthetic control group?
-
How is the synthetic control group matched to the intervention group?
-
Are new treatment vs. synthetic control group populations similar? Factors include ethnicity, socio-economic environment, physical activity levels, and geographical region.
-
Is it of substantial sample size and with the appropriate number of co-variates/confounders measured?
-
In the case of adjuvant synthetic control groups, what is the hybrid ratio between synthetic and “normal” controls?
-
Does usual care from synthetic control data reflect current clinical and imaging practice?
-
Do medical imaging acquisition protocols used in synthetic control groups match closely with those applied in the intervention groups exams?
Each of these considerations will influence the validity of using synthetic control groups, and reporting how this may influence results, may be considered best practice. Machine learning can increase the quality and validity of these groups. The use of synthetic control arms is on the rise, including recent Phase II/III oncology trials using data from the MAMMOTH and SCHOLAR-1 trials (Banerjee et al., 2022).
Furthermore, some studies that have used an external control arm as a supportive analysis have allowed accelerated regulatory approval of the drug (i.e., blinatumomab for patients with precursor B-cell acute lymphoblastic leukemia in complete remission with detectable minimal residual disease, BLAST Study) (Gökbuget N et al., 2018)
What is the role of quantitative imaging and AI in synthetic control arms?
The analysis of clinical studies’ endpoints from medical imaging poses a challenge for managing the consistency of examination protocols across the institutions involved. Clinical imaging techniques show a wide variation in acquisition parameters, such as image spatial resolution, administration of contrast agents, kVp, and mAs (among others) for Computed Tomography (CT), echo time, repetition time, and many other sequence parameters for Magnetic Resonance Imaging (MRI). Furthermore, different vendors offer different reconstruction algorithms.
All these variables, known as “scanner effects” can hinder the detection of biological and pathological information. This issue is even more prominent with MRI because of the absence of a standard intensity scale. Therefore in RCTs, centralized management and control of the appropriateness of medical imaging exams and their suitability with regards to the study protocol must be carefully undertaken by central reading labs (aka imaging core labs).
The advance towards the use of synthetic control arms in clinical trials requires more systematic techniques to accurately find patients that best match the synthetic control group. AI can be used to identify patients in hospital imaging repositories that have similar imaging protocols (by automatically generating a similarity score based on DICOM file metadata) to the ones required in the treatment arm of the study. Moreover, the AI-based harmonization solution can be integrated to deal with the large variabilities seen across the images coming from different imaging centers with a wide variation in acquisition parameters.
The use of self-supervised learning, as a paradigm shift in AI where models learn without ground truth, is a perfect fit for this problem because a model can be generated by learning the different patterns between the image acquisition parameters (metadata) and the resulting image quality (pixel data). Therefore, the chance to generate scanner-invariant representations of imaging data (MR, CT, PET/CT) for extraction of harmonized features and parameters as best ground-truth outputs from the synthetic control group, without loss or alteration of relevant image information, will enable causal inference from real-world data despite the inherent levels of uncertainty.
Furthermore, imaging biomarkers and radiomics features extraction will allow the quantification of properties related to angiogenesis, cell density, tumor heterogeneity, aggressiveness, etc. Quantifying these indicators traditionally requires manual delineations of anatomy or lesions, but AI and convolutional neural networks (CNN) automate these analysis pipelines. The quantitative features obtained can be viewed as a virtual tissue profiling technology for matching the external control group not only at a general level but for searching similar tissue signatures.
Conclusion
Each study using synthetic control groups will have its own unique challenges and constraints with regards to the generalizability of results, interpretation, and associated statistical methodology. Utilizing expertise from specialists in data and imaging fields from project onset can help to improve the validity and reproducibility of this emerging methodology. The inclusion of hybrid control groups mixing synthetic and traditional control groups offers an effective, economical, and patient-friendly option for clinical trials.
References
- Anderson M, Naci H, Morrison D, Osipenko L, Mossialos E. A review of NICE appraisals of pharmaceuticals 2000–2016 found variation in establishing comparative clinical effectiveness. J Clin Epidemiol. 2018;105:50–59.
- Banerjee, R, Shonali M, Kelkar AH, Goodman A, Prasad V, and Mohyuddin GR. Synthetic control arms in studies of multiple myeloma and diffuse large B-cell lymphoma. Br J Haematol. 2022;196 (5): 1274–1277.
- Gökbuget N, Dombret H, Bonifacio M, Reichle A, Graux C, et al. Blinatumomab for minimal residual disease in adults with B-cell precursor acute lymphoblastic leukemia. Blood. 2018 Apr 5;131(14):1522-1531.
- Groth SW. Honorarium or coercion: use of incentives for participants in clinical research. J N Y State Nurses Assoc. 2010;41(1):11.
- Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ. What is “quality of evidence” and why is it important to clinicians? BMJ (Clinical research ed). 2008;336(7651):995-998.
- Thorlund K, Dron L, Park JJH, Mills EJ. Synthetic and external controls in clinical trials – A primer for researchers. Clin Epidemiol. 2020; 12:457–467.
