The quality and significance of scientific studies are often judged by study design and how a given statistical analysis plan has incorporated methods to reduce the risk of bias and unbalanced datasets. This includes the randomization procedures for participants, the inclusion of appropriate control groups, using a well-demonstrated methodology, and recruiting a substantial sample size to establish statistical significance. Broadly speaking, a randomized control trial can be classified as the gold standard study design for intervention-based studies according to the GRADE criteria (Guyett et al., 2008).
In a randomized control trial (RCT) study design, participants are distributed into two or more groups, typically an intervention group (Drug, Exercise, or Therapy) and a control group which can be a placebo or standard-of-care group. In the context of oncology, interventions using a control group across different studies have led to a huge increase in the cost of clinical trials, highlighting the need to find alternative solutions.
Another important consideration for RCT study design and the successful completion of an RCT for the drug or treatment under study is whether participants are not disposed to be administered with the new drug candidate that could possibly cure their disease, being willing to stay within the trial even if they are in this control group. This may lead to higher drop-out rates conflating the risk of bias, reducing the statistical power of the study, and extending the length of a trial with substantial financial implications (Groth 2010).
A synthetic control group derived from patient data may offer a novel and economical solution to these issues. In rare diseases or ever-complex disease phenotypes, it may allow clinical trials to be conducted successfully (Thorlund et al., 2020). This approach based on the generation of Real-World Evidence (RWE) has received backing from the US Food and Drug Agency (FDA), European Medical Agency (EMA), and National Institute for Clinical Excellence in the United Kingdom.
Using a synthetic control arm alongside advanced data mining, such as artificial intelligence (AI) in medical imaging technologies, offers the ability to rapidly decrease clinical trial duration, increase the feasibility of trials, make them more financially viable, and reduce reporting bias.
The synthetic control method can be used to compare the effectiveness of an intervention using external control data. The synthetic control arm can be added as an independent control group or as an adjuvant to a control group. External control arm data may be derived primarily from observational, real-world data (RWD), and previous clinical trial data. The validity and robustness of this approach has already been appropriately analyzed and accepted (Thorlund et al., 2020).
The most significant advantage of a synthetic control group emerges when low numbers of patients with specific inclusion criteria are required as per sample size calculations to evaluate putative differences between the new treatment and the standard-of-care. In this scenario, finding appropriate patients for the control group would be impractical, time-consuming, and expensive. A synthetic control arm can potentially overcome these issues.
The successful use of a synthetic arm depends on the quality of data used, which will be specific for every study. Some considerations to be made when choosing this strategy include:
Is the control group relevant to the population being studied?
What is the source of the synthetic control group?
How is the synthetic control group matched to the intervention group?
Are new treatment vs. synthetic control group populations similar? Factors include ethnicity, socio-economic environment, physical activity levels, and geographical region.
Is it of substantial sample size and with the appropriate number of co-variates/confounders measured?
In the case of adjuvant synthetic control groups, what is the hybrid ratio between synthetic and “normal” controls?
Does usual care from synthetic control data reflect current clinical and imaging practice?
Do medical imaging acquisition protocols used in synthetic control groups match closely with those applied in the intervention groups exams?
Each of these considerations will influence the validity of using synthetic control groups, and reporting how this may influence results, may be considered best practice. Machine learning can increase the quality and validity of these groups. The use of synthetic control arms is on the rise, including recent Phase II/III oncology trials using data from the MAMMOTH and SCHOLAR-1 trials (Banerjee et al., 2022).
Furthermore, some studies that have used an external control arm as a supportive analysis have allowed accelerated regulatory approval of the drug (i.e., blinatumomab for patients with precursor B-cell acute lymphoblastic leukemia in complete remission with detectable minimal residual disease, BLAST Study) (Gökbuget N et al., 2018)
As we have been developing, external and synthetic control arms are alternative approaches used in clinical trials to compare the effectiveness of an intervention without always resorting to a traditional control group. Although both methods seek to reduce the number of patients needed in the control group, their differences lie in how the control data are selected and used:
An external control arm is composed of actual data obtained from patients who participated in previous clinical studies or who were treated under normal clinical conditions. These data typically come from medical records, real-world databases or previous trials. By using these data directly, investigators can compare patients in the intervention group with those who have been previously treated with standard of care or placebo, thus avoiding the creation of a new control group in the current study. This speeds up the trial process and can reduce costs, but carries the risk that the data used may not perfectly match the characteristics of the current population, either because treatment standards have evolved or because there are differences in medical care between studies.
On the other hand, the synthetic control arm generates an artificial control group using advanced statistical models. Unlike the external control, where data are used as-is, the synthetic control combines different data sources, including previous trials and real-world observational data, to create a control group that simulates as closely as possible the intervention group. These models allow adjustment for patient characteristics, such as age, disease status and other demographic factors, to make the synthetic group comparable to the group receiving the new treatment. This flexibility is especially useful in studies where it is difficult to recruit a sufficient number of patients, such as in rare diseases or in studies with strict inclusion criteria. However, the success of a synthetic arm depends to a large extent on the quality of the data used and the ability of the models to generate valid comparisons. If the data are insufficient or the fit is not accurate, the synthetic control may introduce bias or may not be a reliable representation of actual clinical behavior.
Both approaches have advantages and disadvantages. The external control arm uses real, well-documented data, but may not be fully comparable to the current study context, whereas the synthetic arm offers more flexibility and adaptability, but at the cost of greater statistical complexity and dependence on data quality. External control arms are more common when sufficient historical data are available, while synthetic arms are employed in situations where traditional studies would be impractical or costly. Both methods are increasingly being used in areas such as oncology, where the need for faster and less expensive assays is critical.
The analysis of clinical studies’ endpoints from medical imaging poses a challenge for managing the consistency of examination protocols across the institutions involved. Clinical imaging techniques show a wide variation in acquisition parameters, such as image spatial resolution, administration of contrast agents, kVp, and mAs (among others) for Computed Tomography (CT), echo time, repetition time, and many other sequence parameters for Magnetic Resonance Imaging (MRI). Furthermore, different vendors offer different reconstruction algorithms.
All these variables, known as “scanner effects” can hinder the detection of biological and pathological information. This issue is even more prominent with MRI because of the absence of a standard intensity scale. Therefore in RCTs, centralized management and control of the appropriateness of medical imaging exams and their suitability with regards to the study protocol must be carefully undertaken by central reading labs (aka imaging core labs).
The advance towards the use of synthetic control arms in clinical trials requires more systematic techniques to accurately find patients that best match the synthetic control group. AI can be used to identify patients inhospital imaging repositories that have similar imaging protocols (by automatically generating a similarity score based on DICOM file metadata) to the ones required in the treatment arm of the study. Moreover, the AI-based harmonization solution can be integrated to deal with the large variabilities seen across the images coming from different imaging centers with a wide variation in acquisition parameters.
The use of self-supervised learning, as a paradigm shift in AI where models learn without ground truth, is a perfect fit for this problem because a model can be generated by learning the different patterns between the image acquisition parameters (metadata) and the resulting image quality (pixel data). Therefore, the chance to generate scanner-invariant representations of imaging data (MR, CT, PET/CT) for extraction of harmonized features and parameters as best ground-truth outputs from the synthetic control group, without loss or alteration of relevant image information, will enable causal inference from real-world data despite the inherent levels of uncertainty.
Furthermore, imaging biomarkers and radiomics features extraction will allow the quantification of properties related to angiogenesis, cell density, tumor heterogeneity, aggressiveness, etc. Quantifying these indicators traditionally requires manual delineations of anatomy or lesions, but AI and convolutional neural networks (CNN) automate these analysis pipelines. The quantitative features obtained can be viewed as a virtual tissue profiling technology for matching the external control group not only at a general level but for searching similar tissue signatures.
Each study using synthetic control groups will have its own unique challenges and constraints with regards to the generalizability of results, interpretation, and associated statistical methodology. Utilizing expertise from specialists in data and imaging fields from project onset can help to improve the validity and reproducibility of this emerging methodology. The inclusion of hybrid control groups mixing synthetic and traditional control groups offers an effective, economical, and patient-friendly option for clinical trials.
References
