Abstract
Though the term “treatment resistance” (TR) is increasingly applied in psychiatry, consensus regarding definition, criteria, and utility of the concept are missing, and treatments beyond pharmacological are seldom considered in diagnosis or staging methods. Resultingly, calculations of the incidence and prevalence of TR and meta-analysis of interventional strategies are not possible. Though treatment failures (TF) have always been noted, only recently has the focus shifted from the efficacy of treatments to the recalcitrance of a presumed disorder. Response, remission, and partial response must be clarified when considering TF, which should not be applied with diagnoses not currently treatable to remission.
Clinicians must not assume the TR concept matches a concrete reality; there are myriad causes of TF, each unlikely to be the same as many other cases. Copying the phrase “TR” from other professionals propagates a destructive meme that damages critical thinking. TF offers valuable data that must be appreciated and applied, altering our diagnostic and therapeutic hypotheses for each case. TF is an experience that must lead to greater knowledge and creative solutions, not TR, a dead-end pseudo diagnosis of questionable validity and no utility. Rather than oversimplifying our tasks, we must embrace inherent complexity and ambiguity in order to find answers in the granularity of each clinical situation.
Failures of response and remission are merely phases of treatment, and we must remain hopeful and optimistic, fully intending to persist until clinical outcomes improve – this is our responsibility to each therapeutic alliance. We make a conceptual error when we confuse TF with TR; reclassifying a diagnosis as TR represents a failure of clinical reasoning and judgment. As valid treatment options remain for 2/3 of cases described as TR, we must cease using this label and improve our problem-solving skills in order to discover and provide better clinical outcomes.
Keywords
Treatment resistance, Treatment failure, Treatment resistant depression, Treatment resistant schizophrenia, Clinical reasoning, Clinical judgement, Diagnosis, Misdiagnosis, Meme
Abbreviations
TR: Treatment Resistance; TF: Treatment Failure; TRD: Treatment Resistant Depression; TRS: Treatment Resistant Schizophrenia; OCD: Obsessive-Compulsive Disorder
Commentary
The percentage of psychiatric journal articles referring to “treatment resistance” (TR) has increased 75% in last two decades [1]. Our field, though, has never coalesced on clear, consistent criteria or a consensus definition for this term. Part of the utility of diagnoses is accurate communication with each other, yet TR is used to refer to a variety of treatment failures (TF) with few common features. The number, type, and quality of treatments attempted are not standardized for the proper application of TR within most diagnoses, particularly depression, schizophrenia, OCD, and PTSD [2-8]. Most authors have either ignored, or found difficulty incorporating, variables other than previous pharmacological treatments, such as psychotherapy [9,10]. As a result, the calculations of the incidence and prevalence of the expression TR, as well as meta-analysis of interventional strategies, are stymied by these inconsistent definitions, further damaging research into the phrase [1,11]. In one recent expert consensus survey, for example, 92% of psychiatrists reported they considered treatment resistant depression a useful concept, and 85% reported they used it in clinical practice, though 100% of them still yearned for an operational definition of the term [12]. How did we allow this chimeral pseudo diagnosis to become so entrenched?
References to failed treatment attempts appeared during the mid-twentieth century to psychotherapeutic, symptomatic, occupational, somatic, and pharmacological interventions [13-16]. Historically, the reports of inadequate efficacy had originally been directed towards treatments that appeared insufficient in some situations, but not to the condition itself. There was no intent to define a new disorder. Much effort originally centered on finding the right treatments for the right patient, activity that we should continue today [17-19]. TR schizophrenia (TRS), initially labeled therapy resistant or treatment refractory, has been discussed in the literature for more than 50 years [20,21]. TR (or refractory) depression (TRD) as a specific term first appeared in the 1970s [3,22], mostly, again, referring to “resistance” to a particular treatment, such as tricyclic antidepressants, for example [23]. During the 1990s, however, TRD became both an entity and a consistent term in the literature, though always with inconsistent criteria [24-26].
The many conceptions and definitions of TF include the National Institute of Mental Health contract study Sequenced Treatment Alternatives to Relieve Depression (STAR*D) [27], which used failure to achieve remission as the definition for TF. Remission refers to complete symptom resolution, with full functioning restored. Recall that by response, we mean a partial reduction in symptoms as measured by peer-reviewed clinical scales [28,29], although there is still no fully agreed-upon threshold. There are, of course, patients who have what may be termed a partial response, with some improvement, yet do not meet the criteria for response and require additional treatment [30].
Since the STAR*D study, it is more common to see TRD “diagnosed” following the failure of two attempts at pharmacological treatment. Occasionally, however, only one trial is considered adequate, such as with the pivotal study and indication for transcranial magnetic stimulation [12,31,32]. Again, psychotherapeutic treatments are seldom included in definitions or staging criteria, which themselves have also not been adequately assessed for predictive value, largely because of their heterogeneity [2,10,33,34]. Also inherent in any discussion of TF and the concept of TR is the necessity of restricting their application, withholding their use in conditions we do not currently expect to treat successfully. Cases of schizophrenia and obsessive-compulsive disorder (OCD) are particularly difficult to label as TR because we do not currently expect any treatment to result in a full remission [6,35,36]. It appears useful to restrict cases of distinct TF (and any consideration of TR) to exclude irreversible medical conditions for which there is currently no fully effective or sustainable treatment for any patient with the diagnosis.
How the focus on “resistance” changed from inadequate or suboptimal treatment to a recalcitrant disorder is not clear. We have always observed through outcome studies that not everyone responds to or remits from treatments that are commonly considered effective. Perhaps the word “resistant” itself distorted the context, implying the failure of a condition to accept the treatment, rather than that the treatment itself evidenced less broad efficacy than we had euphemistically expected. This represents a prime example of reification—believing that our concept of TR matches a concrete reality. And concepts generated by some, especially those in authority, influence the conclusions of others [37]. With TR, we have created a meme in the original, evolutionary sense—a system of behavior that is passed from person to person by imitation [38-40]. What is clear is that this change has damaged our critical thinking about treatment.
When we apply treatments that are not effective, this is useful data. We can use this information to alter our working model of the problem (i.e., diagnosis) and creatively consider new, more specific treatments for a new paradigm. We find success in our efforts to help our patients when we question our initial assessments repeatedly, particularly when guided by such new information from TF. When a treatment fails, we must examine the nature of our assault on the clinical challenge, as well as the target. Systematic errors, though always to be expected, are nevertheless too often ignored, leading to additional misleading predictions [41].
A recent survey found that 46.9% of first psychiatric diagnoses are revised to alternative diagnoses within the next 10 years; cases involving psychosis, single episodes of major depression, and substance abuse have the greatest diagnostic variability [42]. By generating alternative hypotheses and changing direction as soon as the data indicate [43,44], we can update our conceptualizations of the treatment scenario (our model) so that our problem-solving will be more exact and successful. When we justify our failure by labeling the problem TR, though, we not only absolve ourselves of responsibility for finding the solution, but also ignore the valuable information contained in the TF. At the very least, TF should constantly redirect us to consider and explore the individual characteristics of a case and the circumstances around TF, rather than accept a poorly characterized and operationally nonexistent entity as the explanation. Like “physician burnout,” TF/TR is an experience, not a diagnosis [45].
Practicing psychiatry is hard work. It is full of complexity and ambiguity. Our innate tendency is to oversimply our tasks to employ the least cognitive effort [46,47], but this deprives us of the actual information we need to solve patients’ problems – the needles in the haystacks. Writing off a problem as unsolvable, or TR, means we no longer have responsibility for finding treatment solutions, because, after all, they are not possible in that paradigm. When we attach this air of pessimism to a case, our patients perceive it. When we label them as TR, they feel hopeless. This therapeutic nihilistic can lead to malignant psychodynamics that limit clinical success as additional treatment attempts proceed [48,49]. Hope is, after all, essential for recovery [50,51].
Rather than identify TR as an endpoint diagnosis, or as an entity at all, we may better serve our patients by considering failures of response or remission as phases of treatment that contain important information, still expecting and looking forward to eventual remission. The treatment attempts themselves, but neither the patient nor the diagnosis, may be labeled TF, and a case may be described as pending remission, or even difficult to treat [52], rather than TR, because it is likely that many treatment options remain. In fact, second opinions have identified valid additional treatment options in over two thirds of cases labeled TR [53].
Broad literature review shows more success in identifying individual explanations for TF than actually identifying a “diagnosis” of TR. Some authors have explored the possibility that iatrogenic alterations from therapies may contribute to treatment failure, and the question of in vivo tachyphylaxis remains unclear. Declining response rates with subsequent pharmaceutical trials may be more a factor of lower placebo response rates and less spontaneous improvement than treatment resistance to a specific therapy [2,54].
There is no association with TRD and specific drugs or molecules, nor with augmentation or switching strategies. The number of previous treatment trials is predictive in some reviews but not prognostic in others. Data supporting associations with personality disorders are weak or nonconfirmatory [55-60]. There is some evidence that genetic and epigenetic factors play a role in TRD and psychosis (including TRS). Some genetic polymorphisms are linked to a wide definition of treatment resistance [58], and evidence exists of rare metabolic disorders that infrequently preclude response to standard treatments [61-63]. Literature review discouragingly reveals, however, that most articles continue to try (and retry) newer treatments and technologies to treat “TR,” rather than reexamine the underlying concept of it. There is a small, but growing awareness that we have yet to form a useful conception of TR [52,64], that due to heterogeneous biotypes different patients with the same diagnosis require different treatments [65], that inadequate treatment attempts still persist [66], and that our descriptions of TR are sorely inadequate for scientific study or clinical use [67].
TF has myriad causes, ranging from misdiagnosis and failure to appreciate comorbidities to poor understanding of and miscommunication with our patients, almost always fueled by provider overconfidence [68,69]. Approaching practice as a generalist, searching for and addressing all of a patient’s medical problems, often in consultation with primary care providers or specialists, not only helps avoid psychiatric TF, but also may lead to quicker and more complete responses to treatment for all conditions present [70,71]. We may not always be interested in taking sole responsibility for treating a patient’s nonpsychiatric comorbidities, but we must make every effort to help diagnose all of these conditions. Common underappreciated causes of TF include mild to moderate head trauma, endocrine disorders, lifestyle factors, infectious diseases, ruptures of the therapeutic alliance, poor provider communication skills, complications of polypharmacy and bioavailability, respiratory disorders, and gastrointestinal system disorders, to name a few.
We make a conceptual error when we confuse treatment failure with treatment resistance. The relabeling of a diagnosis during any treatment attempt as TR represents a failure of clinical reasoning and judgment. Our greatest value to our patients is our problem-solving skill, not just our knowledge and experience. Even long clinical experience, without reflection or feedback, can reinforce conceptual error and lower clinical performance [72,73]. The cause of each TF is unlikely to be the same as in many other cases, so it is essential that we see the forest and the trees, broadly examining our problem-solving methods, and obtaining regular feedback from peers, staff, and patients’ words and behaviors in order to correct our mistakes. Then we can successfully address suboptimal outcomes, TF, and this epidemic of misconceptualization that is inaccurately referred to as “treatment resistance.”
References
2. Anderson IM. We all know what we mean by treatment-resistant depression - don't we? Br J Psychiatry. 2018 May;212(5):259-261.
3. Malhi GS, Byrow Y. Is treatment-resistant depression a useful concept? Evid Based Ment Health. 2016 Feb;19(1):1-3.
4. Malhi GS, Das P, Mannie Z, Irwin L. Treatment-resistant depression: problematic illness or a problem in our approach? Br J Psychiatry. 2019 Jan;214(1):1-3.
5. McAllister-Williams RH, Christmas DMB, Cleare AJ, Currie A, Gledhill J, Insole L, et al. Multiple-therapy-resistant major depressive disorder: a clinically important concept. Br J Psychiatry. 2018 May;212(5):274-8.
6. Sheitman BB, Lieberman JA. The natural history and pathophysiology of treatment resistant schizophrenia. J Psychiatr Res. 1998 May-Aug;32(3-4):143-50.
7. Sippel LM, Holtzheimer PE, Friedman MJ, Schnurr PP. Defining Treatment-Resistant Posttraumatic Stress Disorder: A Framework for Future Research. Biol Psychiatry. 2018 Sep 1;84(5):e37-e41.
8. Voineskos D, Daskalakis ZJ, Blumberger DM. Management of Treatment-Resistant Depression: Challenges and Strategies. Neuropsychiatr Dis Treat. 2020 Jan 21;16:221-34.
9. Bennabi D, Aouizerate B, El-Hage W, Doumy O, Moliere F, Courtet P, et al. Risk factors for treatment resistance in unipolar depression: a systematic review. J Affect Disord. 2015 Jan 15;171:137-41.
10. Markowitz JC, Wright JH, Peeters F, Thase ME, Kocsis JH, Sudak DM. The Neglected Role of Psychotherapy for Treatment-Resistant Depression. Am J Psychiatry. 2022 Feb;179(2):90-3.
11. Zhdanava M, Pilon D, Ghelerter I, Chow W, Joshi K, Lefebvre P, et al. The Prevalence and National Burden of Treatment-Resistant Depression and Major Depressive Disorder in the United States. J Clin Psychiatry. 2021 Mar 16;82(2):20m13699.
12. Rybak YE, Lai KSP, Ramasubbu R, Vila-Rodriguez F, Blumberger DM, Chan P, et al. Treatment-resistant major depressive disorder: Canadian expert consensus on definition and assessment. Depress Anxiety. 2021 Apr;38(4):456-67.
13. Ball JR, Kiloh LG. A controlled trial of imipramine in treatment of depressive states. Br Med J. 1959 Nov 21;2(5159):1052-5.
14. Sargant W. Drugs in the Treatment of Depression. Br Med J. 1961 Jan 28;1(5221):225-7.
15. Watts CA. Endogenous depression in general practice. Br Med J. 1947 Jan 4;1(4487):11-4.
16. Young JP, Lader MH, Hughes WC. Controlled trial of trimipramine, monoamine oxidase inhibitors, and combined treatment in depressed outpatients. Br Med J. 1979 Nov 24;2(6201):1315-7.
17. Akiskal HS. A proposed clinical approach to chronic and "resistant" depressions: evaluation and treatment. J Clin Psychiatry. 1985 Oct;46(10 Pt 2):32-7.
18. Pare CM, Mack JW. Differentiation of two genetically specific types of depression by the response to antidepressant drugs. J Med Genet. 1971 Sep;8(3):306-9.
19. Remick RA. Antidepressants: which one? Can Fam Physician. 1986 Mar;32:587-92.
20. Itil TM, Keskiner A, Fink M. Therapeutic studies in "therapy resistant" schizophrenic patients. Compr Psychiatry. 1966 Dec;7(6):488-93.
21. Quintero J, del Cura EB, López-Ibor MI, López-Ibor JJ. The evolution of the concept of treatment-resistant schizophrenia. Spanish Journal of Psychiatry. 2011 Jul 1;39(4):236-50.
22. Cowen PJ. Depression resistant to tricyclic antidepressants. BMJ. 1988 Aug 13;297(6646):435-6.
23. Extein I, Spiegel D. Treatment of tricyclic-resistant depression. (No Title). 1989.
24. Gitlin MJ. Treatment-resistant depression. West J Med. 1991 Nov;155(5):521.
25. Nierenberg AA, Amsterdam JD. Treatment-resistant depression: definition and treatment approaches. J Clin Psychiatry. 1990 Jun;51 Suppl:39-47; discussion 48-50.
26. Nierenberg AA, White K. What next? A review of pharmacologic strategies for treatment resistant depression. Psychopharmacol Bull. 1990;26(4):429-60.
27. Howland RH. Sequenced Treatment Alternatives to Relieve Depression (STAR*D). Part 1: study design. J Psychosoc Nurs Ment Health Serv. 2008 Sep;46(9):21
28. Marshall M, Lockwood A, Bradley C, Adams C, Joy C, Fenton M. Unpublished rating scales: a major source of bias in randomised controlled trials of treatments for schizophrenia. Br J Psychiatry. 2000 Mar;176:249-52.
29. Swan K, Speyer R, Scharitzer M, Farneti D, Brown T, Woisard V, et al. Measuring what matters in healthcare: a practical guide to psychometric principles and instrument development. Front Psychol. 2023 Sep 18;14:1225850.
30. Nierenberg AA, DeCecco LM. Definitions of antidepressant treatment response, remission, nonresponse, partial response, and other relevant outcomes: a focus on treatment-resistant depression. J Clin Psychiatry. 2001;62 Suppl 16:5-9.
31. O'Reardon JP, Solvason HB, Janicak PG, Sampson S, Isenberg KE, Nahas Z, et al. Efficacy and safety of transcranial magnetic stimulation in the acute treatment of major depression: a multisite randomized controlled trial. Biol Psychiatry. 2007 Dec 1;62(11):1208-16.
32. Rush AJ, Trivedi MH, Wisniewski SR, Nierenberg AA, Stewart JW, Warden D, et al. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. Am J Psychiatry. 2006 Nov;163(11):1905-17.
33. Conway CR, George MS, Sackeim HA. Toward an Evidence-Based, Operational Definition of Treatment-Resistant Depression: When Enough Is Enough. JAMA Psychiatry. 2017 Jan 1;74(1):9-10.
34. Ruhé HG, van Rooijen G, Spijker J, Peeters FP, Schene AH. Staging methods for treatment resistant depression. A systematic review. J Affect Disord. 2012 Mar;137(1-3):35-45.
35. Sharma E, Math SB. Course and outcome of obsessive-compulsive disorder. Indian J Psychiatry. 2019 Jan;61(Suppl 1):S43-S50.
36. Iasevoli F, Avagliano C, D'Ambrosio L, Barone A, Ciccarelli M, De Simone G, et al. Dopamine Dynamics and Neurobiology of Non-Response to Antipsychotics, Relevance for Treatment Resistant Schizophrenia: A Systematic Review and Critical Appraisal. Biomedicines. 2023 Mar 14;11(3):895.
37. Putman III HP. Encountering Treatment Resistance: Solutions Through Reconceptualization. American Psychiatric Pub; 2024 Apr 15.
38. Dawkins R. The selfish gene. Repr. with corr. Oxford: Oxford Univ. Pr; 1981.
39. Duch W. Memetics and neural models of conspiracy theories. Patterns (N Y). 2021 Sep 25;2(11):100353.
40. Markov AV, Markov MA. Runaway brain-culture coevolution as a reason for larger brains: Exploring the "cultural drive" hypothesis by computer modeling. Ecol Evol. 2020 May 20;10(12):6059-77.
41. Box GE. Empirical model building and response surfaces. John Wiley & Sons google schola. 1987;3:27-37.
42. Høj Jørgensen TS, Osler M, Jorgensen MB, Jorgensen A. Mapping diagnostic trajectories from the first hospital diagnosis of a psychiatric disorder: a Danish nationwide cohort study using sequence analysis. Lancet Psychiatry. 2023 Jan;10(1):12-20.
43. Johnson DK. Inference to the Best Explanation and Avoiding Diagnostic Error. InEthics and Error in Medicine 2019 Oct 17 (pp. 243-261). Routledge.
44. Feng J, Gossmann A, Sahiner B, Pirracchio R. Bayesian logistic regression for online recalibration and revision of risk prediction models with performance guarantees. J Am Med Inform Assoc. 2022 Apr 13;29(5):841-52.
45. Summers RF. The Elephant in the Room: What Burnout Is and What It Is Not. Am J Psychiatry. 2020 Oct 1;177(10):898-9.
46. Croskerry P. Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Adv Health Sci Educ Theory Pract. 2009 Sep;14 Suppl 1:27-35.
47. Linders GM, Louwerse MM. Zipf's law revisited: Spoken dialog, linguistic units, parameters, and the principle of least effort. Psychon Bull Rev. 2023 Feb;30(1):77-101.
48. Shepherd DJ, Insole LM, McAllister-Williams RH, Ferrier IN. Are specialised affective disorder services useful?. Psychiatric Bulletin. 2009 Feb;33(2):41-3.
49. Cosgrove L, D'Ambrozio G, Herrawi F, Freeman M, Shaughnessy A. Why psychiatry needs an honest dose of gentle medicine. Front Psychiatry. 2023 Apr 21;14:1167910.
50. Langlands RL, Jorm AF, Kelly CM, Kitchener BA. First aid for depression: a Delphi consensus study with consumers, carers and clinicians. J Affect Disord. 2008 Jan;105(1-3):157-65.
51. Sari SP, Agustin M, Wijayanti DY, Sarjana W, Afrikhah U, Choe K. Mediating Effect of Hope on the Relationship Between Depression and Recovery in Persons With Schizophrenia. Front Psychiatry. 2021 Feb 9;12:627588.
52. Demyttenaere K. What is treatment resistance in psychiatry? A "difficult to treat" concept. World Psychiatry. 2019 Oct;18(3):354-5.
53. Nirodi P, Mitchell AJ, Mindham RH. Survey of expert second opinions in a tertiary psychiatric out-patient clinic in the Yorkshire region between 1988 and 2000. Psychiatric Bulletin. 2003 Nov;27(11):416-20.
54. Faltinsen E, Todorovac A, Staxen Bruun L, Hróbjartsson A, Gluud C, Kongerslev MT, et al. Control interventions in randomised trials among people with mental health disorders. Cochrane Database Syst Rev. 2022 Apr 4;4(4):MR000050.
55. De Carlo M. Conceptualization, in Scientific Inquiry in Social Work. Pressbooks, 2018, Updated March 2019. Available at https://scientificinquiryinsocialwork. pressbooks.com/chapter/9-2-conceptualization.
56. Papakostas GI, Petersen TJ, Farabaugh AH, Murakami JL, Pava JA, Alpert JE, et al. Psychiatric comorbidity as a predictor of clinical response to nortriptyline in treatment-resistant major depressive disorder. J Clin Psychiatry. 2003 Nov;64(11):1357-61.
57. Petersen T, Hughes M, Papakostas GI, Kant A, Fava M, Rosenbaum JF, et al. Treatment-resistant depression and Axis II comorbidity. Psychother Psychosom. 2002 Sep-Oct;71(5):269-74.
58. Smith-Apeldoorn SY, Veraart JK, Schoevers RA. Definition and epidemiology of treatment resistance in psychiatry. Treatment resistance in psychiatry: risk factors, biology, and management. 2019:3-24.
59. Souery D, Oswald P, Massat I, Bailer U, Bollen J, Demyttenaere K, et al; Group for the Study of Resistant Depression. Clinical factors associated with treatment resistance in major depressive disorder: results from a European multicenter study. J Clin Psychiatry. 2007 Jul;68(7):1062-70.
60. McIntyre RS, Alsuwaidan M, Baune BT, Berk M, Demyttenaere K, Goldberg JF, et al. Treatment-resistant depression: definition, prevalence, detection, management, and investigational interventions. World Psychiatry. 2023 Oct;22(3):394-412.
61. McClain LL, Shaw P, Sabol R, Chedia AM, Segretti AM, Rengasamy M, et al. Rare variants and biological pathways identified in treatment-refractory depression. J Neurosci Res. 2020 Jul;98(7):1322-34.
62. Mocking RJT, Naviaux JC, Li K, Wang L, Monk JM, Bright AT, et al. Correction to: Metabolic features of recurrent major depressive disorder in remission, and the risk of future recurrence. Transl Psychiatry. 2021 Feb 8;11(1):115.
63. Pan LA, Martin P, Zimmer T, Segreti AM, Kassiff S, McKain BW, et al. Neurometabolic Disorders: Potentially Treatable Abnormalities in Patients With Treatment-Refractory Depression and Suicidal Behavior. Am J Psychiatry. 2017 Jan 1;174(1):42-50.
64. Zanardi R, Carminati M, Attanasio F, Fazio V, Maccario M, Colombo C. How different definition criteria may predict clinical outcome in treatment resistant depression: Results from a prospective real-world study. Psychiatry Res. 2024 Apr;334:115818.
65. Nasrallah HA. Treatment resistance is a myth. Curr Psychiatr (2021). 2021 Mar 1;20:14-6.
66. von Knorring J, Baryshnikov I, Jylhä P, Talaslahti T, Heikkinen M, Isometsä E. Prospective study of antidepressant treatment of psychiatric patients with depressive disorders: treatment adequacy and outcomes. BMC Psychiatry. 2023 Nov 28;23(1):888.
67. Weissman MM. Does treatment-resistant depression need psychotherapy? World Psychiatry. 2023 Oct;22(3):417-8.
68. Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med. 2008 May;121(5 Suppl):S2-23.
69. Norman E. Why Metacognition Is Not Always Helpful. Front Psychol. 2020 Jul 2;11:1537.
70. Degruy FV 3rd. A note on the partnership between psychiatry and primary care. Am J Psychiatry. 2006 Sep;163(9):1487-9.
71. DeGruy FV III. Psychiatry's fit in the world of modern primary care. Honorary Fellowship Lecture presented at the American College of Psychiatrists An- nual Meeting, Tucson, AZ, February 25, 2023.
72. Tay SW, Ryan P, Ryan CA. Systems 1 and 2 thinking processes and cognitive reflection testing in medical students. Canadian medical education journal. 2016 Oct;7(2):e97.
73. Lim JY, Ong SYK, Ng CYH, Chan KLE, Wu SYEA, So WZ, et al. A systematic scoping review of reflective writing in medical education. BMC Med Educ. 2023 Jan 9;23(1):12.