In ongoing efforts to bring innovation to healthcare, technology and patient self-management are at an important intersection. Our study, "A Mobile App for Chronic Disease Self-Management for Individuals with Low Health Literacy: A Multisite Randomized Controlled Clinical Trial” [1], serves as a potentially useful contribution to this intersection by addressing the challenges faced by older adults with chronic conditions and low health literacy in managing chronic health conditions. This commentary explores the importance of the study's findings, participants' perceptions of the app, and acknowledges potential criticisms regarding the absence of significant control-experimental group differences. Additionally, it advocates for a broader interpretation of study outcomes beyond traditional null hypothesis statistical testing given the potential usefulness of our findings.

In order for readers to better understand comments on the study, a synopsis of the study is included in this section. The complete report is available open access on the site of the publisher.

Synopsis: The study titled "A Mobile App for Chronic Disease Self-Management for Individuals with Low Health Literacy: A Multisite Randomized Controlled Clinical Trial" aimed to evaluate the effectiveness of a mobile app designed to improve chronic disease self-management in patients aged 40 years and older with low health literacy. The participants were randomly assigned to use an app with information tailored to one of three reading levels: 3^rd grade, 6^th grade, and 8^th grade. The investigators hypothesized that the 8^th grade group would serve as a control condition, and that participants in the 3^rd and 6^th grade groups would show greater treatment effects than those in the control.

While 334 participants completed the study’s baseline assessments, only 315 individuals completed at least one intervention session. Participants were 40 years of age or older, had at least one chronic health condition for which they were treated, and were racially and ethnically diverse. The study assessed four primary outcomes: patient activation, chronic disease self-efficacy, health-related quality of life (HRQOL), and self-reported medication adherence.

Analyses showed that individuals in all three groups showed improvements in activation, self-efficacy, and HRQOL, but not in medication adherence. No between-group differences were statistically significant. Given the failure to find differences between the control and experimental groups, interpretation of the study’s findings must be limited to the observation that participants showed improvements in key outcomes, but that it is not clear why. Given positive feedback from participants in follow-up interviews, these findings highlight the potential of mobile health interventions to support chronic disease self-management in diverse populations, regardless of specific reading level adaptations. Future research could further explore the nuances of these findings and the potential for digital health tools to bridge gaps in health literacy and self-management capabilities.

The core of any healthcare innovation lies in its real-world impact on its users. In this study, feedback from participants was overwhelmingly positive, highlighting the app's practical benefits in managing chronic diseases. The participants reported enhanced engagement with their health conditions, increased self-efficacy, and a greater sense of being an active participant in their interactions with providers.

In exit interviews, participants commented that the information they learned from the app helped them to feel more in control of their health. They stated that they had learned new ways to manage mood and stress, and that it made an important difference in their daily lives. Participants’ comments like these underscore the app's potential to fill gaps in traditional healthcare delivery, particularly for individuals who may struggle to understand their health conditions and implement complex medical advice.

Further, the way the app was designed to include individually tailored information about specific health issues and to use multimedia elements was particularly well-received. Participants appreciated the app's ease of use and the relevance of the information provided as evidenced not only in their comments but also ratings of the app [2]. This feedback aligns with the study's aim to make health information they want and need available to patients when they need it and in a form they can use. This strategy with individuals with low health literacy ensured that the app’s content was accessible and actionable.

In their comments, participants also highlighted the app's role in facilitating better communication with healthcare providers. Many reported feeling more confident during medical appointments, arriving at them armed with specific questions and a better understanding of their conditions. This was a welcome change to these individuals with low health literacy, who often face barriers in navigating the healthcare system.

Despite the positive feedback from participants, it is important to acknowledge potential criticisms of the study. We had hypothesized that information provided at an 8^th grade level (often regarded as too difficult by experts) would have less impact on participants. We compared results of persons who received information at 3^rd and 6^th grade levels, but no statistically significant differences emerged. For some, this meant that the study was worthless, and that our efforts to present results that showed that all groups appeared to benefit from their participation were worthless. This raises a critical discussion about the role of statistical significance in evaluating the efficacy of healthcare interventions. Indeed, it is often forgotten that the originator of one of the most influential standards in all of research, Karl Fisher, never meant that a result should only be evaluated on the basis of the admittedly arbitrary standard of p <0.05 [3].

The study's primary outcomes—patient activation, chronic disease self-efficacy, and health-related quality of life—showed overall improvements across time in all groups, and the lack of control may lead some to question the study's impact. This reflects a broader issue in clinical research, where the emphasis on p-values and statistical significance often overshadows meaningful clinical outcomes.

It is important to note that while the statistical analysis did not reveal significant differences between the control and experimental groups, all groups showed positive changes in the key outcomes that could be interpreted as effect sizes. Providing effect sizes allows a comparison of our study to those of others – in this case, our results were similar to those of other studies that had statistically significant control-experimental group differences. In fact, the effect sizes obtained in this study were similar and even larger than those found in other studies that involved direct face-to-face interventions, as detailed in the discussion section of the original report [1]. This suggests that the app had a beneficial effect on participants, even if we cannot attribute them only to the effects of the intervention. Participants’ comments and ratings, however, suggest that they found the app useful and that it gave them tools to help them better manage their health. The lack of between-group differences might be the results of several factors, including the nature of the intervention, the multimedia format that facilitated understanding even in those with low health literacy, or the amount of personal contact with study personnel. We would suggest that the findings should be used to further investigate their mechanism, rather than discarding them as useless because of a lack of between-group statistical significance.

Qualitative feedback from participants provided valuable insights into the app's effectiveness and areas for improvement. Many participants shared personal stories of how the app helped them manage their health, highlighting its real-world impact. For example, some participants reported that the app helped them to better understand their medications and adhere to their treatment plans. Others noted that the stress management techniques provided by the app helped them cope with the emotional challenges of living with a chronic condition. This qualitative feedback underscores the importance of incorporating patient perspectives into the evaluation of healthcare interventions. While quantitative measures provide valuable data on outcomes, qualitative feedback offers a deeper understanding of how interventions work in practice and their impact on patients' lives. It is essential, however, to note that participants may respond positively because of social desirability, so that positive comments should not be taken at face value with additional support from other data sources.

The lack of statistically significant differences does not diminish the real-world impact observed in the study as indicated by our participants’ comments and ratings. In clinical practice, the practical benefits experienced by patients can be as important, if not more so, than statistically significant outcomes. The improvements in participants' activation, self-efficacy, and quality of life highlight the app's potential to make a tangible difference in the daily management of their health.
For instance, patient activation is a crucial factor in self-management. Activated patients are more likely to engage in health-promoting behaviors, adhere to treatment plans, and experience better health outcomes [5,6]. The study demonstrated a significant positive improvement over time in participants’ level of patient activation, suggesting that the app successfully engaged participants in their healthcare. Similarly, increases in self-efficacy for managing their chronic conditions were observed, indicating that participants felt more capable of managing their health conditions. Self-efficacy is a well-documented predictor of health behavior change [7], and its enhancement through the app suggests that participants were better equipped to handle their chronic conditions. Health-related quality of life (HRQOL) also showed improvement, which was another key measure of the intervention's impact. HRQOL encompasses various aspects of a patient's well-being, including physical, mental, and social health. Enhancing HRQOL is particularly significant for individuals with chronic conditions, as it reflects their overall ability to live well despite their health challenges.

A notable aspect of this research was its emphasis on individuals with low health literacy, a demographic that frequently encounters challenges in accessing healthcare. People with low health literacy are vulnerable to health consequences and inequities in healthcare. When enrolling individuals with low health literacy in this study we observed that a significant number had limited educational backgrounds, reduced financial means and belonged to historically minoritized communities. 
The app’s design was specifically crafted to tackle these obstacles by presenting information at various reading levels and using multimedia to aid comprehension. By customizing the content to suit individuals with low health literacy the app played a role in equipping these patients with the resources for effective health management. Emphasizing health literacy is crucial especially as there is a growing awareness of health inequalities and the necessity for interventions that target determinants of health. With its user accessible features, the app has the potential to diminish health disparities and enhance outcomes for marginalized communities. Leveraging technology interventions could prove instrumental in addressing healthcare inequities [8,9].

Potential criticisms of this study shed light on an issue in research practices of relying too heavily on statistical significance as the main indicator of an intervention’s effectiveness. While p values and confidence intervals play an important role in data interpretation, they should not be the sole basis for assessing a study’s worth. It is equally important to consider overall significance, for example, how the findings translate into tangible benefits for patients. The positive changes experienced by participants in this study, such as feeling more empowered about their health and adopting management strategies, represent outcomes with potential long-term benefits.

The healthcare community should move towards a more comprehensive approach in evaluating interventions, recognizing their potential impact as well as the communities affected by them. This more comprehensive approach would include considering qualitative feedback from participants, considering their satisfaction, and the practical implications of the findings as well as assessing the significance of changes in outcome measures. This is not to suggest abandoning testing, but rather to consider other aspects of an intervention study in addition.

Such an approach would provide a more comprehensive understanding of an intervention's impact and promote innovations that genuinely improve patient care. Further, the emphasis on statistical significance often ignores the complexity of healthcare interventions. Real-world effectiveness can be influenced by numerous factors that are difficult to capture in a randomized controlled trial. For instance, individual differences in health literacy, socio-economic status, and access to healthcare services can all affect the outcomes of an intervention.

An important aspect of our original evaluation plan was to evaluate important mediators and moderators of study outcomes. At this point in assessing the intervention’s effect, we want to address two key findings: (1) there were no differences between control and experimental groups, but (2) participants seem to have changed positively on a number of outcome measures, including activation, self-efficacy, and quality of life [4] and sleep [10]. We want to recognize the positive changes that occurred among participants over the course of the study and evaluate what factors may have mediated or moderated the changes. A mechanism we proposed in the initial development of the app was based on the potential effect of tailoring information to each person’s needs, and we plan to evaluate whether the extent to which success in tailoring was a factor in positive change. Elsewhere, we integrated our hypotheses about the mechanisms by which health literacy interventions have a positive effect on health with the Theory of Planned Behavior [11], hypothesizing that helping patients develop knowledge and skills related to disease management might enhance their perceived behavioral control and help them develop more positive attitudes toward self-care. We plan to explore those as possible mechanisms by which the app might have had an effect on participants. These relations can be evaluated in structural equation models, allowing for statistical testing of these mechanisms. In conducting these analyses, it will be important to include sensitivity analysis procedures in order to test not only for the existence of effects but to evaluate their strength and the conditions under which they occur.

The findings of this study give us several avenues for future development of the app. First, we may explore expanding the app to include additional modules and resources (such as self-monitoring capability) that could enhance its utility. Modules on nutrition, physical activity, and mental health could provide a more comprehensive tool for chronic disease self-management. Second, integrating the app with other healthcare technologies, such as electronic health records (EHRs) and telehealth platforms, could enhance its availability. If the app were integrated into patient portals, it could be readily available to patients; providers could even “prescribe” a module to help ensure the patient completes it. This sort of integration would allow providers to monitor patients' progress and give them personalized feedback and support. Third, exploring the use of artificial intelligence (AI) and machine learning could further enhance the app's ability to tailor information and recommendations to individual patients. By analyzing user data, AI algorithms could provide more precise and personalized guidance, improving the app's effectiveness. Finally, expanding the app's reach through partnerships with healthcare organizations and community groups could help to ensure that more patients benefit from this innovative tool. Given the nature of our participants with respect to age, education, health literacy, further development must include evaluation of the app in other populations to better establish the generalizability of our findings.

The findings of this study have several implications for future research and practice. First, it underscores the need for more inclusive and comprehensive measures of intervention effectiveness. Future studies should incorporate both quantitative and qualitative methods to capture the full spectrum of patient experiences and outcomes. Second, there is a need for more research on the specific components of the app that were most effective. Understanding which features were most beneficial can help refine and improve the app, making it even more effective for future users. We are currently studying how to better understand what mechanisms may have influenced the positive changes we observed in the study. The approach of modeling change processes rather than relying on traditional hypothesis testing may yield important information about why change occurs [12]. Third, future research should explore the long-term impact of the app on patient outcomes. While this study showed positive trends over the course of the intervention, it would be valuable to understand how these changes persist over time and whether they translate into sustained improvements in health and quality of life. Last, the study highlights the importance of patient-centered design in developing healthcare interventions. Engaging patients in the design and testing process ensures that the tools developed are relevant, accessible, and effective for the target population.

The study on which this commentary focuses offers valuable insights into the potential of digital health interventions in addressing issues of health equity. Despite the potential criticisms due to the lack of statistically significant control-experimental group differences, the positive feedback from participants underscores the app's practical benefits. As the healthcare landscape continues to evolve, it is imperative to balance statistical rigor with clinical relevance, especially when there is substantial evidence of an intervention’s impact. Emphasizing real-world outcomes and patient experiences will pave the way for innovations that truly enhance healthcare delivery and patient well-being. The findings of this study serve as a reminder that meaningful improvements in patients' lives can and should be the goal of healthcare interventions.

In conclusion, while statistical significance is an important aspect of evaluating healthcare interventions, it should not overshadow the practical and meaningful benefits experienced by patients. This study demonstrated that a mobile app for chronic disease self-management may positively impact patient activation, self-efficacy, and quality of life, even in the absence of significant between-group differences. Moving forward, the healthcare community should adopt a more integrated approach to evaluating interventions, considering both quantitative and qualitative outcomes to fully understand their impact. By focusing on the real-world benefits and patient experiences, we can develop more effective and patient-centered healthcare solutions. This approach will not only improve the quality of care but also empower patients to take an active role in managing their health, leading to better health outcomes and a higher quality of life.

The study described in this commentary was funded by the U.S. National Institute on Minority Health and Health Disparities (grant number R01MD010368), Dr. Ownby, principal investigator.