In 2019, an estimated 2 million babies were stillborn [1] and 2.4 million died within 28 days of life (neonatal deaths) [2]. About 80% of these stillbirth and neonatal deaths occurred in low- and lower-middle- income countries (LMICs) with sub-Saharan Africa region contributing to more than 50% of these deaths [1-3]. A child born in sub-Saharan Africa has an eleven times greater chance of dying in the first month of life than a child born in a high-income nation [2]. Malawi like other sub Saharan African countries (Kenya, Zambia, Zimbabwe, Congo and Lesotho) contributes to high global neonatal mortality with current stillbirth rate at 16.3 per 1000 births and neonatal mortality at 27 per 1000 live births [4,5].

Despite an increase in the number of births assisted by skilled attendants in LMICs, stillbirths remain common, and new-borns are still dying from preventable causes due to poor quality of care [6-8]. Each year, one million stillbirths and new-born deaths could be prevented if services were of high quality [8]. LMICs need to invest in healthcare system strengthening to provide high quality new-born health services.

Stillbirth and neonatal death audit is a widely recommended intervention to improve quality of care and reduce stillbirths and neonatal mortality thereby helping to attain Sustainable Development Goal 3.2 [9]. This approach is also in line with Every Newborn Action Plan strategic objective 2 and 5 to improve the quality of maternal and newborn care and count every newborn through measurement, program tracking and accountability to generate data for decision making and action [10]. Audits empower staff to learn from mistakes and initiate significant changes in the care of patients or the healthcare system more generally [11]. Stillbirths and neonatal death audit helps identify gaps and implementation of ways to improve the quality of new-born care [12]. However, the audit and feedback cycle need to link to actions at the point of care; audit alone does not necessarily reduce deaths [11]. Effective audit requires a functional system with constant monitoring and evaluation and with the feedback loop in place as per the audit cycle (Supplementary Figure 1) [13]. In 2016, WHO developed stillbirth and neonatal death audit guidelines to assist facilities in implementing quality audits [9].

Despite the adaptation of WHO audit guidelines at national level and audit and feedback being widely used, its effectiveness is variable, with some studies reporting positive effects on mortality [8,10-13] while others reporting no effect [14]. Furthermore, in some Sub-Saharan African countries, conducting stillbirth and neonatal death audits, it is unclear whether the data collected are linked to health outcomes considering the high number of deaths in facilities [15]. Studies have reported challenges in implementing audits linked to national engagement, organizational support, formulating appropriate recommendations and implementing changes [15-17]. In Malawi, stillbirths and neonatal reviews/audits are not as well established as maternal audits [18].

We assessed the quality of facility-based stillbirth and neonatal death audit implementation in seven districts in Malawi based on WHO audit guidelines and provided recommendations on how hospitals in this and similar settings can support staff to improve the audit process.

Study design and setting

This descriptive mixed-method study was part of a quality improvement project conducted between August 2019 and November 2020 evaluating processes and outcomes of stillbirth and neonatal death audit and the context in which audits are conducted in public hospitals in the southern region of Malawi. The WHO Making Every Baby Count: Audit and Review of stillbirth and Neonatal deaths guidelines and toolkit comprises of 6 modules. The data collection tools were developed basing on this WHO audit guidelines. We used WHO audit cycle 6 steps to design audit meeting observation checklist, minimum perinatal and neonatal indicators checklist while audit document and audit forms review were based on WHO audit tool kits. The assessment also incorporated aspects of creating enabling environment for audit as per WHO guidelines.

Population and sampling

The study population comprised health workers working in maternity and neonatal (nursery) wards, in-charges of wards, focal persons responsible for neonatal care and members of stillbirths and neonatal death audit committees. Seven hospitals that implemented audits were selected and their neonatal mortality rates fall within the lowest, medium and highest categories. The seven hospitals included one central hospital (tertiary level; hospital 1) and six district hospitals (secondary level; hospitals 2 to 7).

Measurements and data collection

We reviewed pregnancy and neonatal outcome data (morbidity and mortality) and all audited deaths from August 2019-November 2020 in each hospital. We reviewed audit documents and observed 12 neonatal death audit meetings. Thirty-five health workers in maternity and neonatal wards, available on the interview day, were conveniently selected for a semi-structured questionnaire survey about audit knowledge, practice and impact.

Institutional deliveries, stillbirths, and neonatal outcome data review

A standardized form, adapted from WHO guidelines for stillbirths and neonatal deaths [9], was used to extract monthly data on deliveries, stillbirths and neonatal admissions and deaths from the hospital Health Management Information System department and nursery ward registers (Supplementary Table 1). Data were collected for 15 months retrospectively from August 2019 to January 2020 and then prospectively and monthly from February 2020 to October 2020.

Document review

Based on WHO audit guidelines, we reviewed neonatal death audit national guidelines and a classification list of causes of deaths and modifiable factors for neonatal deaths. In each hospital, data collection templates, reporting templates, and follow-up records of action plans were checked and reviewed for their availability, use and type of information captured.

Quantitative and qualitative data processing and analysis

All completed audit forms from August 2019-November 2020 were reviewed using a newborn-perinatal audit form for stillbirths (Supplementary Form 1) and neonatal death audit form (Supplementary Form 2). Scanned copies with identifiable information redacted were saved in a password-protected computer. Information extracted included the frequency of audit meetings, proportion of deaths audited, the proportion of forms with complete admission and patient information, the proportion of common causes of deaths audited, summary of the proportion of deaths that could have been prevented, proportion with modifiable factors and proposed solutions that may prevent deaths, quality of action plans, quality of completed audit forms. Modifiable factors and proposed solutions were grouped into health provider, administrative and caregiver or patient factors as per WHO guidelines. The quality of the action plan was assessed if they were specific, measurable, achievable, realistic, and timebound (SMART) as per WHO guidelines and grading of these parameters was guided by a predefined template (Supplementary Table 2) adapted from Kimambo et al. [19] which are consistent with WHO SMART description in the guide. Each component of the plan was scored for appropriateness, with scores 1, 2 and 3 representing unsatisfactory, good, and very good. The quality of completed audit forms was assessed using a predefined template adapted from the UK Data Management Association working group [20] (Supplementary Table 3). Assessed parameters included completeness (all parts of form filled), accuracy (correct information), consistency (agreement of information within the form), and validity (representing what it aims to measure). A score of ‘excellent (100%)’ was assigned to complete, accurate, consistent, and valid forms, ‘good (75%) ‘if less than 5 items were missing and ‘unsatisfactory (50%)’ if more than 5 or more items were missing.

Observation of audit meetings

Observation of audit meetings was conducted in all hospitals between March 2020 and November 2020 using a pretested observation checklist (Supplementary Checklist 1). The observations checklist included general meeting organization and the six WHO mortality audit cycle parameters: identifying deaths, collecting information, analyzing information, recommending solutions, implementing solutions, evaluating, and refining processes (Supplementary Figure 1) [9].

Staff interviews

A pretested, semi-structured questionnaire adapted from Nyamtema et al. [17] was completed during interviews with nurses and clinicians from the maternity and neonatal wards who were audit committee members to assess their perceptions of audit quality (Supplementary File 1). The questionnaire included their level of knowledge, views, perceived impact of audit and suggestions for improving the stillbirth and neonatal death audit process. Convenience sampling was used to select at least five staff from each hospital on duty on the interview day. The number of 5 was chosen as a minimum number of staff per shift was between 4 and 6.

Data management and analysis

Pre-coded quantitative data were entered into a Microsoft access database and cleaned and backed-up daily. Data was analyzed using SPSS 26.0 software. Quantitative data was described using frequencies and proportions. The qualitative information from the checklist and forms were analyzed by thematic analysis using predefined themes in the survey guide in the category of knowledge and impact of audit (Supplementary File 1).

Ethics approval and consent to participate

Approval was obtained from the College of Medicine (P.11/19/2869) and the Liverpool School of Tropical Medicine (19-076) ethics committees. All hospitals gave permission to conduct the study. All healthcare workers who took part in the study signed an ethics-approved informed consent form.

Hospital births, stillbirths, and neonatal outcomes during the study period

Of 55,685 births that occurred in the seven hospitals from 1^st August 2019 to 30^th November 2020, there were 1318 (2.4%) stillbirths. The overall stillbirth rate was 23.7 per 1000 births (95% CI:22.4-24.9) ranging from 20.0 to 26.5 per 1000 births for individual hospitals (Table 1). About 54% of all stillbirths were intrapartum. Amongst 13,113 neonatal admissions, 1732 (13.2%; 95% CI: 12.6-13.8) died. Wide variations were observed in neonatal deaths, with hospitals 4 and 7 having the highest proportion of deaths (18.7 % and 18.0% respectively), while hospital 5 had the lowest (8.3%). Birth asphyxia was the leading cause of deaths overall (45.0%) and in the individual facilities, seconded by prematurity (18.8%). Nearly all neonatal deaths (1636; 94.4%) were early, occurring within seven days of life (Table 1).

Document review

Neonatal admission forms, audit data collection, reporting templates and a classification list of causes of deaths and modifiable factors for neonatal death audit were available in all hospitals. The modifiable factors list did not include any factors from maternity wards (antenatal, labor and postnatal wards) or other departments (Supplementary File 2). Four hospitals (3, 5-7) did not have any form for stillbirth audit while the other three hospitals (1,2 and 4) had one, although this was not standardized across these hospitals. The following documents were not available for use in any of the hospitals: national audit guidelines, recommendation follow-up records of action plans template or forms detailing whether proposed solutions had been implemented. The reporting template in use did not have a section to report whether a proposed solution was implemented or not.

Death audit results

Conduct of audit and characteristics of audited stillbirths and neonatal deaths                         

Only 0.6% (n=8) of all stillbirths were audited with only two hospitals (hospitals 1 and 2) undertaking stillbirth audit (Table 1). Six of audited stillbirths were born with a normal birth weight of more than 2500g. Five of audited stillbirths were intrapartum (fresh stillbirths), and 3 were antepartum stillbirths (macerated stillbirths). All audited stillbirths occurred due to intrapartum related events that resulted in perinatal asphyxia. Only 25.3% (n=438) of all neonatal deaths were audited with hospital 2 accounting for most of all audited deaths (50.9%), representing 76.1% of its facility neonatal deaths (Table 1). More than 80% of all neonatal deaths audited were early neonatal deaths. Almost two-thirds of audited neonatal deaths occurred during night shifts.

Completeness of information for audited stillbirths and neonatal deaths

Twenty-nine percent and 20% of admission and critical care pathway forms of audited neonatal deaths were incomplete, respectively. Feeding charts were not used in almost half of deaths audited, and there was no information available on their usage in 34.2% of deaths audited, and no maternal file (labor graph) was attached in 74.9% of audited deaths (Supplementary Table 4). We did not assess these parameters on audited stillbirth as they were not part of stillbirth audit form.

Causes of stillbirths and neonatal deaths

Almost all eight stillbirths audited occurred due to intrapartum related events that resulted in perinatal asphyxia. The intrapartum related events included prolonged second stage of labor (n=3), ruptured uterus (n=2), cord accident (n=2) and severe pre-eclampsia (n=1). Supplementary Table 5 shows the aggregated contribution of conditions to the cause of neonatal death audited. Birth asphyxia, prematurity, and respiratory distress syndrome were the leading cause.

The proportion of neonatal deaths that could have been avoided

Overall, about one third of neonatal deaths audited (36.1%; n=158) were considered preventable, 44.7% (n=196) were considered unpreventable, while the team was unsure in the case of 14.8% (n=16) deaths. No information was included for 4.3% (n=19) deaths (Figure 1). More neonatal deaths were assessed to be likely to be preventable in hospitals 3, 4, 5, and 7 (87.5%, 54.1%, and 80.6%, respectively) than in hospitals 1 and 2 (26.0% and 26.5%; Figure 1).

Modifiable factors and proposed solutions

Stillbirth audits generated 31 modifiable factors of which 93.5% were health provider related factors (Supplementary Table 6). The overall mean number of modifiable factors identified per audited neonatal death was 2.1 ranging from 1.7 (hospital 2) to 4.7 (hospital 4) (Supplementary Table 7). Most factors were health provider related with an overall mean of 1.9 modifiable factors per audited death ranging from 1.3 (hospital 6) to 4 (hospital 4). These health provider related factors included inadequate monitoring of sick neonates, documentation, clinical review, management or treatment, feeding, and investigation (Supplementary Table 7). The overall mean number of identified administrative and caregiver factors per audited death were 0.1 each. The proposed actions for audited neonatal deaths strongly emphasized improving health provider factors at nursery wards, with more than 70% of solutions targeting the management of sick neonates in the nursery ward. The overall mean number of solutions per neonatal death was 2.3 ranging from 1.3 (hospital 4) to 3.4 (hospital 3) (Supplementary Table 7).

Quality of reviews

The overall average score for all neonatal death action plans (n=996) was 2.5 (SD=0.6) indicating the need for improvement (Table 2). Overall, performance was high (very good) in identifying modifiable factors (mean=2.9; SD=0.4) and assigning responsible persons (2.6; 0.7; Table 2). Proposed solution and feasibility of timeframe components scored below standard (2.4; 0.8 and 2.3; SD=0.7 respectively) (Table 2). There was consistency across hospitals in these quality scores. Stillbirth audit only generated modifiable factors and solutions but not full action plan.

Quality of completed neonatal audit forms

Review of all neonatal audit forms (n=438) revealed that accuracy and consistency were high (Table 3). However, only about 1 in 5 forms scored excellent for completeness (mean score of 77.1%; SD=15.0) and only about half of forms were scored excellent for validity (86.6%; 12.8) with marked variability in these two factors across hospitals.

Frequency of audit meetings

There was marked variation in the frequency of audit meetings with weekly or monthly meetings in hospitals 1 and 2 but only occasional and irregular meetings in other hospitals (Supplementary Table 8). Overall meeting frequency increased from July to November 2020 (Supplementary Table 8) as hospitals 1, 2, and 3 were supported by external partners who provided a lunch allowance of approximately $5 per individual and refreshments during audit meetings. Hospital 1 had no external partner in the district from September 2019 to June 2020 and the central management team supported audits from March 2020 to June 2020; the external partners came in July 2020 and funded audits quarterly. Hospital 2 had an external partner in the district who funded audit meetings from January 2019 to November 2019, and again from May 2020. The other districts conducted audits when district management funded the activities (refreshments or lunch allowance), although this support was inconsistent.

Observation of neonatal death audit meetings

Audit organization: Between March and November 2020, 55 audit meetings were conducted in all hospitals, with most meetings done in hospitals 2 (n=29;52.7%) and hospital 1 (n=15; 27.3%; Supplementary Table 8). Twelve meetings were observed, three meetings each in hospitals 1, 2, and 5, two meetings in hospital 2 and one meeting in hospital 4. No meetings were observed in hospitals 6 and 7 as research staff were not informed of when meetings took place. We did not observe any stillbirth audits.

For 12 observed neonatal audit meetings, all hospitals had a neonatal focal person who coordinated neonatal death audit meetings. Nurses from nursery wards and pediatric clinicians mostly attended the meetings (Figure 2). None of the meetings started with reviewing previous audit summaries or minutes or checking whether recommendations had been implemented. Meetings had a welcoming atmosphere where a chairperson encouraged a ‘no blame or shame’ approach except in 4 meetings (2 from hospital 1 and one each from hospitals 2 and 3) where staff were identified by tracing or interpreting their signatures. Four meetings (2 from hospital 3 and 1 each from hospitals 4 and 2) included teaching sessions.

WHO mortality cycle parameters: All hospitals used audit forms to document causes of death, modifiable factors, and action plans. Neonatal deaths were mainly identified from patient files from nursey wards but not source documents from other wards. In one hospital nurses collected demographic and clinical data soon after the death but other hospitals filled this information during death audit meetings. No hospital collected a minimum set of neonatal indicators or used them during analysis. During audit meetings, the patient file was read a loud and background and clinical information documented. Clinical information was verified if already entered. The members discussed causes of death using a national checklist of causes and decided on modifiable factors and actions. None of the meetings started by reviewing ward statistics, previous audit summaries or minutes or checking whether previous recommendations had been implemented. Meetings had a welcoming atmosphere where a chairperson encouraged a ‘no blame or shame’ approach except in 4 meetings (2 from hospital 1 and one each from hospitals 2 and 3) where staff were identified by tracing or interpreting their signatures. Four meetings (2 from hospital 3 and 1 each from hospitals 4 and 2) included teaching sessions.

Only hospital 3 approached the system as a whole when analyzing modifiable factors. The other hospitals focused on identifying factors that could have been prevented in the nursery wards but not from labor, antenatal, and postnatal wards. For example, death of a baby that was severely asphyxiated when admitted to the nursery ward was deemed to be unavoidable as no more could have been done at the nursery ward; however, avoidable factors in the labor and delivery ward were not considered. As a result, recommended actions mainly targeted nursery wards as guided by the provided modifiable factors list. No written feedback or documentation system for tracking and following up on recommendations was observed in any of the hospitals.

Health worker perceptions about the neonatal death audit process: Almost two thirds of staff interviewed were nurses and one third were doctors or clinical officers working either maternity, nursery, paediatric wards or administrative roles. Only one participant was from the administration department.

The level of awareness of neonatal death audit was high among staff in all seven hospitals: 97.1% mentioned at least one reason why such audits were established, 85.7% knew of the presence of audit committees, and 80.0% knew the objective or vision of audit committees (Supplementary Table 9). However, only 67.0% mentioned that the objectives or vision had been shared with the rest of the other ward staff and departmental representatives. Nearly all the participants knew the permanent members of the maternal or neonatal death audit committee citing 122 nurses and 35 clinicians working in maternity, nursery, and pediatric wards.

All hospital staff believed that death audits could improve neonatal health services provided everywhere and at their hospitals (Supplementary Table 9). Nearly all staff knew and remembered at least one recommendation made during maternal or neonatal death audit at their hospital and mentioned at least one action implemented in their ward due to audit. However, 54.3% offered at least one suggestion to improve the audit process and care delivery (Supplementary Table 9).

Although systematic reviews have shown that well-conducted audits and feedback can improve care and professional practice [21,22], we found significant deficiencies in the implementation and quality of stillbirth and neonatal death audits assessed according to WHO guidelines in hospitals in southern Malawi. Audits appeared to be conducted to fulfil requirements for reviewing deaths rather than as internal mechanisms for improving practice. This section discusses the study findings against general literature and WHO audit guidelines, focusing on creating enabling environment and six steps of audit cycle.

Creating enabling environment for audit

A positive enabling environment at national, regional and facility level makes easier to move through phases of mortality audit process [9]. The WHO guidelines stress the importance of the availability of clear national policy and guidelines. Unfortunately, no national guidelines for stillbirth and neonatal death audit were found in all hospitals to guide health workers on audit process. Only national standardized audit tools for neonatal death audit were available and none for stillbirth audit. The list of neonatal death modifiable factors provided by national level focused on nursery ward related factors rather than other departments like labor and delivery where most stillbirth and newborn death causes originates.

Furthermore, the WHO guideline stresses the need for leadership, task-oriented minutes, staff stability, communication, and the existence of guidelines and protocols [9] all of which were deficient in this study. The guideline further reports experience from maternal death reviews in Senegal and Mali that a bundle of approaches is needed to translate recommendations into action [9,23,24]. The bundle included involving leaders, quality improvement committee involvement and strengthening the capacity of health professionals. Leaders are required at both national and facility levels to create an enabling environment as they act as change agents. In our study, although there was good awareness and knowledge among staff about audit and its impact on care, the majority of staff mentioned nurses working in maternity, nursery, and pediatric ward as frequent members attending audits and senior facility management staff rarely participated in audit meetings consistent with the findings of other studies [15,17,25]. The guideline further recommends facility-based mortality audits to include representatives of various departments and community liaison officers of which it was deficient in this study.

Another way to create an enabling environment to effect change is to have individual members of staff who are accountable with appropriate follow-ups [7]. We found that the frequency of audit meetings was dependent on the support of partners who provided monitory incentives attached and low when such support was not available. These findings agree with a study done in Uganda, which had difficulties bringing staff to audit meetings during lunch breaks if no lunch or snacks were provided [26].

From audit observations, four meetings had no welcoming atmosphere and failed to follow a no blame, and no shame approach in contrast to studies done in Solomon Island and Tanzania, where the meetings had a welcoming atmosphere [16,27]. Staff who fear blame, judgement or negative consequences may be reluctant to attend mortality audit meetings or suppress information about events, which might affect the effectiveness of the process [28]. The WHO guideline stresses that the committee and facility leadership should nurture a conducive environment of no blame for successful audit process.

Six WHO audit cycle

Identify stillbirth and neonatal deaths: To ensure that all birth and death outcomes are recorded and accounted for, the WHO audit and review of stillbirth and neonatal death guidelines emphasizes collection of a minimum set of perinatal and neonatal indicators on each birth and death through hospital HMIS register or electronic system [9]. The WHO guidelines advise audit committees to make good use of these outcome data during the audit process and our study confirmed that these indicators were collected in the hospital HMIS system. The guidelines recommend analyzing the trend on perinatal and neonatal outcomes and that facility administrators or local policymakers identify particular indicators to focus on, collect more information or follow up after implementing audit recommendation [9]. However, our findings revealed that perinatal and neonatal outcome data were not reviewed during audit meetings.

Collecting information: After identifying the deaths, the team decides on deaths that they need to collect more information. Our findings of incomplete documentation regarding maternal information and clinical details agrees with other studies that identified missing information as a barrier to mortality audit [17,29,30]. Whilst acknowledging the challenge of locating medical records in low resource settings, WHO recommends extracting relevant medical information using standardized form as soon after the birth and death occur. This occurred in only one hospital in our study where relevant demographic variables and medical history were collected prior to audit meetings.

There was significant variation between hospitals in audit frequency and the number of stillbirth and neonatal deaths audited. We observed a low proportion of audited stillbirths and variation in the proportion of neonatal deaths audited across hospitals. Only one hospital audited more than half of its neonatal deaths. Given the high numbers of stillbirths and neonatal deaths in this setting, the low audit rate may not be sufficient to identify gaps in care. After implementing regular audits and auditing 75% of all neonatal deaths, a study in Moldova noted a decrease of 1.5 deaths per 1000 births (95% CI 0.6% to 2.4%; p=0.0015) [31]. However, in Uganda, auditing only 34% of all perinatal deaths and implementing local solutions was associated with a reduction of 4.9 deaths per 1000 births [32]. In addition, a second study in Uganda which audited only 20% of perinatal deaths due to a high volume of deaths, reported a statistically significant decrease in neonatal mortality although no effect on perinatal mortality or stillbirth rates [33].

The WHO guideline is not explicit on what proportion of deaths should be sampled for auditing; rather this depends on staffing and workload at the facility and considers the length of review meetings [9]. Where the burden of stillbirths and neonatal deaths is high and it is infeasible to review all deaths, WHO recommends either selecting a subset of cases for detailed review or limiting review to cases that are most likely to be preventable such as stillbirths, early neonatal death or neonatal death among near-term babies [9]. This appears relevant to our setting in that 50% of all stillbirths were intrapartum and 94.4% of all deaths and 87.9% of all audited deaths were early neonatal deaths. Our findings suggest a need for developing a sampling framework to guide the proportion of types of deaths reviewed to improve the feasibility and quality of audit and facilitate international comparisons.

Analyzing information: We found that birth asphyxia was the leading cause of neonatal deaths in all hospitals, followed by prematurity. Almost all neonatal deaths (94.4%) and audited neonatal deaths (87.2%) occurred within seven days of life indicating the need to focus interventions during pregnancy, labor and delivery. However, the identified modifiable factors and proposed solutions focused on caring for neonates in the nursery ward rather than the labor and delivery ward, which would likely miss root causes. A focus on nursey ward factors was also noted during audit meeting observations and was evident from the document review. These findings are contrary to WHO guideline which emphasize conducting a root-cause analysis to identify underlying causes and remaining open to all possible underlying problems and factors [9]. We also noted that the focus on nursery ward factors affected the audit staff’s assessment of the proportion of deaths that could have been prevented, with only one-third of deaths considered to have been avoidable. The hospitals with the lowest proportion of preventable deaths (hospitals 1 and 2) did not approach the system as a whole but mainly focused on nursery ward factors. The marked variation in the proportion of preventable deaths between hospitals is consistent with studies of audits in Tanzania, France, Solomon Island, and Uganda with proportions ranging from 20- 80% [16,19,32,34,35].

The hospitals in our study followed the WHO guidance to identify modifiable factors at the family/patient, administrator and health provider levels [9]. The most frequently identified modifiable factors were health provider-related, which agrees with other studies' findings [16,32,34,36].

Although WHO recommends the use of a globally recognised approach to the classification of stillbirths and neonatal death causes such as the International Classification of Disease 10 to share common language and allow comparisons across settings [9], this was not done in the hospitals in this study.

Recommending solutions: WHO recommends formulating solutions in action plans that are SMART [7]. In this study, the proposed solutions and the feasibility of timeframes to implement solutions were limited. This agrees with a study done in Tanzania which found that most action plans were unsatisfactory [19].

Implementing solutions: Implementing actions to prevent deaths is the reason for conducting audit. Though staff reported witnessing improvements in care resulting from audit, this was difficult to verify objectively as there was no evidence of implementation of audit recommended solutions in any of the facilities. The neonatal death audit data collection and reporting tools had no section to report or follow up recommendations made during previous audit meetings. The WHO guidelines for the action plan section includes a follow-up section, where audit recommendations are assessed as completed or not at the next meeting. This was lacking in the hospital audit forms which likely affected implementation of audit findings by staff. Similar challenges have been reported in a systematic review focusing on LMICs [37].

In contrast to WHO recommendations, review of previous minutes and follow-up records to see if previous recommendations had been implemented during audit meetings were not observed in any of the hospitals. This is likely due to the limitation of the audit form and reporting template. National level staff need to consider providing a reporting template that includes these elements to motivate staff to implement the solutions proposed during the audit process

Evaluating and refining: Furthermore, WHO stresses the importance of evaluating and refining the components of the audit cycle to identify what worked and what did not [9] but this was lacking in our study. Documenting change over time and having a system to provide real-time feedback linked to data showing long term trends motivates staff [9]. WHO further recommends that audit committee members assess and reflect on progress at each implementation stage from creating awareness of audit to integration in the routine practice [7].

The strengths of this study are that it used a mixed-method approach to provide a comprehensive and detailed assessment of the gaps in the quality of stillbirth and neonatal death audit in an LMIC. We consider that this generated reliable evidence that can directly inform areas for improvement and provide credible recommendations for practice, policy and research in this and similar settings. The study was multisite, including seven hospitals in 7 districts of Malawi, thus increasing its applicability. Additionally, we assessed the quality of stillbirth and neonatal death audits using internationally recognized WHO guidelines to allow comparison with other studies and similar settings.

Limitations result from the inclusion of one central hospital and six district hospitals in one region of Malawi. Hence, our findings might not reflect audit processes in the other hospitals in Malawi and other LMICs. We used convenience sampling for staff questionnaires that included staff from different cadres; however, our findings may not be generalizable to all staff. We could not collect complete patient information for all audits due to missing records in some hospitals. Despite this, we benefited from triangulation across different data collection methods to assess the quality of the audit.

The practical implication of our study is for national policymakers to ensure that developed stillbirth and neonatal death audit guidelines are used in hospitals and that the national or partner data collection or reporting templates are consistent with the audit cycle parameters in the WHO audit and review of stillbirth and neonatal death guidelines. Furthermore, to guide implementers and allow international comparisons a sampling framework needs to be developed to guide the proportion and types of deaths to be reviewed in settings where number of deaths are high. Quality audits need to be conducted regularly at the facility level, adhering to and completing the WHO death audit cycle [9]. In addition, implementers need to be trained on all steps and supported by the management. Finally, few studies have reported the outcome of stillbirths and neonatal death audit on newborn outcomes [11,33]. We propose that a multi-country trial is required to evaluate the effectiveness of audits on stillbirth rates and neonatal mortality.

The quality of stillbirth and neonatal death audit was poor due to challenges the hospitals faced in creating enabling environment for audit and completing WHO audit cycle steps. This information is valuable for implementers, policymakers and researchers to improve the stillbirth and neonatal death audit process and, through this, the quality of care within maternity, postnatal, neonatal and nursery hospital wards. The implementation of regular audit cycles of a consistently high standard has the potential to reduce stillbirths and neonatal mortality.

The authors would like to thank staff and management team members from all seven participating hospitals who provided this study information. I would also like to thank Ismaela Abubakar from the Clinical Trial Unit at Liverpool School of Tropical Medicine, who helped develop forms and tables in Microsoft Access. I would also like to thank Lophina Sitima Phiri, who helped evaluate action plans and the quality of the data collection tool, Dr Joana Raven and Dr Florence Mgawadere for guidance on the concept development and reviewing the manuscript. Many thanks to Professor David Lissauer and Maternal Health group and Mtundu khongono from Malawi-Liverpool Wellcome Trust, Clinical Research Programme for support during field work.

Not applicable.

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request. Most data generated or analyzed during this study are included in this published article [and its supplementary information files].

The authors declare that they have no conflict of interest.

This work was supported by the 'Commonwealth Scholarship Commission' (MWCSC, 2018, 802). This paper is part of the first author PhD studies.

MJG conceived the idea of the study, collected, analyzed data and drafted the manuscript. All authors (MJG, ND, MA, and SA) were instrumental in the study's development, in reviewing data collected and analyzed, drafts of the paper and in approval of the final manuscript.

Supplementary Checklist 1. Observation checklist for audit process and cycle

Supplementary Figure 1. Death Audit Cycle

Supplementary Form 1. Newborn-perinatal death audit form used in Malawi

Supplementary Form 2. Neonatal death audit form

Supplementary Table 1. Health Facility Surveillance form

Supplementary Table 2. Assessment criteria for quality of action plan

Supplementary Table 3. Criteria for grading quality of audit form

Supplementary Table 4. Characteristics of audited neonatal deaths and completeness of admission information

Supplementary Table 5. Presumed causes of neonatal deaths

Supplementary Table 6. Avoidable factors and proposed solutions identified during stillbirth audit

Supplementary Table 7. Modifiable factors and proposed action; data are number (%)

Supplementary Table 8. Frequency of neonatal death audit meetings by facility

Supplementary Table 9. Health works perceptions about neonatal death audit process