The exponential growth in the publication of systematic reviews and meta-analyses (SRs/MAs) over the past two decades has radically transformed the landscape of scientific evidence. In fields like psychiatry, this proliferation is particularly striking: in 2023 alone, more SRs/MAs were indexed in PubMed under psychiatry than randomized controlled trials (RCTs), a reversal of the traditional pattern. For conditions such as depression, anxiety, and schizophrenia, new SRs/MAs emerge daily. This overwhelming and abundant volume, while rich in data, presents a profound challenge: how can clinicians, researchers, and policymakers efficiently interpret and integrate these often divergent findings?
Tertiary evidence synthesis, particularly in the form of umbrella reviews, has emerged as a methodological response to this saturation. Tertiary synthesis refers to the integration and analysis of data derived from SRs/MAs, thus operating one level above primary and secondary studies. Rather than synthesizing original trials or observational studies, tertiary synthesis draws exclusively from previously synthesized evidence. Its goal is to provide a comprehensive, high-level overview that consolidates findings across multiple SRs/MAs addressing similar questions, highlighting consistencies, discrepancies, and knowledge gaps [1–4]. However, traditional umbrella reviews (TURs) are slow, labor-intensive, and frequently outdated by the time they are published. Moreover, they tend to limit their scope to descriptive mapping, rarely performing any statistical aggregation of results. Methodological authorities such as the Joanna Briggs Institute [5] advise against aggregating outcomes across multiple SRs/MAs, while Cochrane overviews allow re-analysis only from primary data and under strict conditions [6]. These approaches, while methodologically cautious, may limit the practical utility of tertiary syntheses in fast-evolving fields.

The Umbrella Collaboration® (TU®) offers a novel model for tertiary synthesis, designed from the ground up to address these, and other, structural limitations. This commentary follows the recent publication of our validation study in JMIR Formative Research, which systematically evaluated the performance of TU® in comparison to traditional umbrella review methods across eight projects in geriatrics [7]. While that study focused on methodological validation, the present work aims to explore broader conceptual and practical implications of TU® when applied to another evidence-intensive discipline: psychiatry. TU® is a semi-automated system that facilitates human-guided synthesis across all phases of the process. Unlike most tools developed to support secondary synthesis, which only assist with specific stages (e.g., screening or data extraction), to our knowledge, TU® is the first tool specifically designed for tertiary evidence synthesis and encompasses the entire workflow. The study referenced above thus represents the first published evaluation of a tool purpose-built for tertiary synthesis. While there are several studies assessing the performance of automation tools in secondary synthesis, no comparable evaluation exists for tertiary synthesis applications. It integrates automation to enhance speed and reproducibility, yet retains human oversight to ensure contextual accuracy, prevent algorithmic bias, and maintain interpretative depth.

This semi-automated design is particularly important given the known limitations of full automation in evidence synthesis. While automation can efficiently handle structured quantitative data, it struggles with qualitative nuance, and its lack of transparency can generate "black-box" concerns. TU® acknowledges these risks and implements safeguards: human reviewers supervise all stages, confirm eligibility, and validate results. The system does not aim to replace expert judgment but to augment it. Semi-automation enables a balance between scalability and rigor, making it feasible to generate robust syntheses without forgoing critical reasoning.

The central role of automation in evidence synthesis is now well established, with recent overviews detailing how AI-enabled tools improve screening, data extraction, and updating while requiring human oversight for reliability [8,9]. While existing tools aim to accelerate specific steps of the systematic review process, such as literature screening or risk-of-bias assessment, few have attempted to orchestrate a comprehensive, continuously updating synthesis. TU® fills this gap by offering a semi-automated system that encompasses the entire workflow of tertiary synthesis, from literature identification to result aggregation, result visualization, and updating. Notably, it performs literature surveillance every 24 hours, well beyond the standard monthly frequency recommended for Living Systematic Reviews by Cochrane [10].

The application of this model to psychiatry is particularly promising. Mental health research is characterized by a high volume of publications, substantial heterogeneity in outcomes, and considerable methodological diversity. Additionally, psychiatric research frequently includes subjective endpoints, population subgroups, and intervention variability, which complicates aggregation and interpretation. The challenge is not just to collect evidence, but to navigate its complexity in a way that is actionable and comprehensible. TU® addresses this by allowing synthesis projects to be built around defined PICO questions and updated with minimal effort, ensuring continued relevance in areas with high publication velocity.

The rationale for tertiary synthesis in psychiatry is further reinforced by the increasing rate of discordance between SRs/MAs answering the same question. Numerous studies have documented significant discrepancies in effect sizes, conclusions, or certainty ratings between reviews covering the same intervention or exposure [11–13]. These divergences arise from varying eligibility criteria, analytical approaches, selective reporting, or even conflicts of interest [14]. While much of the evidence for such discordance comes from other fields of medicine, there is reason to believe these challenges may be even more pronounced in psychiatry due to the inherent complexity of mental health conditions, variability in outcome measures, and subjective interpretations of clinical endpoints. TU® addresses this by treating SRs/MAs as the unit of analysis and calculating a weighted average of the reported effect sizes. This approach does not aim to overwrite individual interpretations, but to offer a centralized, cumulative signal.

Moreover, TU® aligns with international efforts to bridge the gap between research and practice. The WHO [15], along with the European Council [16], has emphasized the importance of knowledge translation in health as a means to facilitate equitable, evidence-based decision-making [17,18]. By providing results in a dynamic, interactive format, TU® enables end-users with varying degrees of statistical literacy to access, interpret, and act upon high-level evidence. This democratization of access is vital in mental health, where decisions often involve multidisciplinary teams and shared decision-making with patients and caregivers.

In the recent validation study conducted in the field of geriatrics [7], TU® was compared directly with TURs across eight projects. The findings demonstrated that TU® successfully replicated 85% (73 of 86) of the outcomes of interest identified by TURs. More importantly, it identified an additional 337 outcomes of interest, reflecting a 4.77-fold increase in the total volume of synthesized evidence. This amplification in detection was not achieved at the cost of quality: full concordance in effect size classification was observed in 50% of cases, and consistent concordance (defined as full agreement plus a one-level deviation) was achieved in 94% of comparisons. The strength of association, measured by Cramér’s V, was moderate (0.339), while Cohen’s kappa indicated fair agreement (K = 0.357). These findings illustrate that TU® not only accelerates and expands synthesis but also maintains methodological rigor in line with accepted standards.

Execution time is another critical dimension where TU® offers transformative potential. While conventional TURs may require 6 to 12 months to complete, TU® projects in the validation study were executed in under ten hours. This extreme reduction in synthesis time creates the possibility of integrating tertiary evidence synthesis into real-time decision-making workflows. In psychiatry, where clinical paradigms evolve rapidly and evidence for new interventions or risk factors emerges daily, this capability may represent a structural advantage.

One point that has drawn particular attention is TU’s use of sentiment analysis to estimate the certainty of evidence. Naturally, this cannot replace the nuanced deliberations of a GRADE assessment [19–21]. But it does offer something else: a standardized, replicable, and scalable approximation. In our validation, these sentiment-based scores showed moderate, statistically significant correlation with GRADE. This tells us that TU may be useful not just for synthesis, but for prioritization, particularly when resources are limited or timeliness is critical.

Likewise, TU’s proprietary effect size metric (R_TU) has evolved from a conceptual solution to a functional instrument. In our study, R_TU demonstrated high categorical concordance with standardized metrics like Cohen’s d. While the correlation at the numerical level was modest, this reflects, in part, the fact that TU works with abstract-level data only. In future versions, we plan to incorporate weighting and contextual modifiers, but even now, R_TU seems to perform well as a first-pass indicator of magnitude.

Several technical features distinguish TU® from traditional methods. The platform performs searches, extracts outcomes, estimates effect sizes, assesses direction and statistical significance, and updates certainty assessments using sentiment analysis. It incorporates filters and dynamic tables that enable users to explore results across subgroups or compare related outcomes. All of this is achieved without abandoning human judgment, reviewers remain involved in validation, interpretation, and final inclusion of outcomes.

It is also important to note that TU® does not attempt to replicate every function of a traditional systematic review. Instead, it streamlines those functions that can be standardized while preserving flexibility where expert input is needed. In this way, it reflects the design philosophy of rapid reviews [22–26], where scope is pragmatically defined to yield timely, actionable insights. What differentiates TU® is that it applies these principles at the tertiary level, across a meta-layer of evidence, and does so with an unprecedented level of automation and speed.

The greatest strength of TU® is not its speed, nor even its breadth, but its ability to remain active and keep synthesis projects continuously updated. Once a synthesis project is initiated, TU® continues to monitor the literature, automatically every 24 hours or upon request by the project reviewer, flagging new SRs/MAs that meet the original inclusion criteria. With minimal reviewer input, these findings can be integrated into the live project, maintaining its relevance over time.

This idea, that evidence synthesis can be 'living' without becoming unmanageable, has long been discussed in theory [27–29]. TU® makes it operational. Although this dynamic updating capacity was not explored in our initial validation study, as it requires a different methodological framework and longitudinal design, it represents a central focus for future research. This is planned for future work, as it requires extended reviewer engagement and dedicated monitoring. Nevertheless, the platform already allows for real-time incorporation of new SRs/MAs as they are published, making it one of the few systems capable of functioning as a Living Tertiary Review.

In parallel, we are currently exploring its usability and acceptance among expert reviewers. Early findings suggest that TU® is especially well received by users with experience in tertiary synthesis and by those whose areas of interest produce a high volume of SRs/MAs, such as psychiatry. These users appreciate the ability to interact with evidence in a structured, visual, and continuously evolving environment, something traditional methods cannot offer. Future studies will further characterize patterns of use, barriers to adoption, and potential integration into clinical guideline development.

The justification for tertiary synthesis in psychiatry is no longer merely conceptual; it is a practical necessity. Although the volume of SRs/MAs has grown across all fields of medicine, the pace in psychiatry is particularly intense, with conditions such as depression and anxiety each generating several new SRs/MAs daily. This surge in secondary synthesis has naturally led to an increase in tertiary synthesis publications as a response to the challenges of fragmentation and interpretative inconsistency. Between 2009 and 2020, the number of tertiary syntheses published annually increased eightfold, reaching a frequency of one publication per day by 2020 [30]. This trend underscores both the need and the opportunity for platforms like TU®.

Despite the advances in automation, the human element remains essential. The interpretation of psychiatric evidence often requires nuanced clinical reasoning and contextual awareness. TU® was built with this reality in mind. It does not pretend to automate thought but to assist it. Its design recognizes the importance of supervised automation, the only kind that can be trusted in fields where the consequences of misinterpretation are substantial [9,31].

Recent years have seen a rapid expansion of tertiary evidence in psychiatry across exposures, predictors, and interventions. Umbrella reviews have mapped predictors of treatment response across mental disorders [32], examined psychotherapy and pharmacotherapy efficacy with stricter risk-of-bias filters [33], and synthesized wide-ranging mental-health impacts of environmental exposures, including air pollution and climate change [34]. Other umbrella syntheses in youth populations and during the COVID-19 period further illustrate the field’s breadth and heterogeneity [35,36]. Against this backdrop, our commentary adds a platform-oriented perspective: The Umbrella Collaboration® operationalizes a semi-automated, living tertiary-synthesis workflow that can continuously surveil, harmonize, and visualize SRs/MAs across psychiatric topics, addressing fragmentation and timeliness beyond topic-specific umbrella reviews.

As the number and complexity of SRs/MAs continues to grow, the ability to perform reliable, interpretable, and up-to-date tertiary synthesis will become increasingly central to evidence-based psychiatry. TU® is not the final word in this evolution, but it is an operational and scalable starting point. It proposes a model in which semi-automation, methodological transparency, and human oversight are not opposing forces, but mutually reinforcing components. In doing so, it offers not just a tool for synthesis, but a path forward for the science of summarizing science.