Country Learning on Maintaining Quality Essential Health Services (EHS) during COVID-19 in Timor-Leste: A mixed methods qualitative analysis
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (PeerRef)
Abstract
Objective
This research study examines the enabling factors, strengths, and challenges experienced by the Timor-Leste health system as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.
Design
A mixed methods qualitative analysis
Setting
National, municipal, facility levels in Baucau, Dili and Ermera Municipalities in TLS
Participants
Key informant interviews (n=40) and focus group discussions (n=6) working to maintain quality EHS in TLS.
Results
A reduction in people accessing general health services was observed in 2020, reportedly due to fears of contracting COVID-19 in healthcare settings, limited resources (eg. human resources, personal protective equipment, clinical facilities, etc) and closure of health services. However, improvements in maternal child health services simultaneously improved in the areas of skilled birth attendants, prenatal coverage, and vitamin A distribution, for example. Five themes emerged as enabling factors for maintaining quality EHS including 1) high level strategy for maintaining quality EHS, 2) implementation of quality activities across the three levels of the health system, 3) measurement for quality and factors affecting service utilization 4) the positive impact of quality improvement leadership in health facilities during COVID-19, and 5) learning from each other for maintaining quality EHS now and for the future. Other countries may benefit from the challenges, strengths and enablers found on planning for quality.
Conclusion
The maintenance of quality essential health services (EHS) is critical to mitigate adverse health effects from the COVID-19 pandemic. When quality health services are delivered prior to and maintained during public health emergencies, they build trust within the health system and promote healthcare seeking behavior. Planning for quality as part of emergency preparedness can facilitate a high standard of care by ensuring health services continue to provide a safe environment, reduce harm, improve clinical care, and engage patients, facilities, and communities.
DATA SHARING
All data is kept with MBK and GR and is available upon request. The dataset analysis is available from the corresponding author upon reasonable request.
QUALITATIVE CHECKLIST
The Standards for Reporting Qualitative Research (SRQR) checklist was used for this original research.
STRENGTHS AND LIMITATIONS OF THIS STUDY
-
The qualitative data gave detailed insights to the operationalization of key strategic COVID-19 emergency documents and the national quality implementation strategy.
-
Data collection was performed in three out of thirteen municipalities, including the largest metropolitan city of Dili.
-
The qualitative research was conducted in the participants native language (Tetum).
-
Not all pre-identified national level KII participants were available to provide feedback.
Article activity feed
-
Author response
To Mary Anne Mercer
Introduction: The aims of the study are to “…learn from the efforts…to maintain quality EHS” (p 1) as well as to “understand the variance in service utilization, including the best practices in improving maternal mortality and SBA.” That leads the reader to expect proposed answers to these questions in the conclusion, which didn’t happen.
Thank you for your feedback on this important point. The data shows that there were improvements in maternal and child health, but not in general primary health services. So, the overall aim is to understand the preparedness activities or lack there of took place in TLS as a comparison to the efforts taken to maintain maternal/child health service. The objective has been reworded: ‘This case study examines the enabling factors, strengths, challenges and lessons …
Author response
To Mary Anne Mercer
Introduction: The aims of the study are to “…learn from the efforts…to maintain quality EHS” (p 1) as well as to “understand the variance in service utilization, including the best practices in improving maternal mortality and SBA.” That leads the reader to expect proposed answers to these questions in the conclusion, which didn’t happen.
Thank you for your feedback on this important point. The data shows that there were improvements in maternal and child health, but not in general primary health services. So, the overall aim is to understand the preparedness activities or lack there of took place in TLS as a comparison to the efforts taken to maintain maternal/child health service. The objective has been reworded: ‘This case study examines the enabling factors, strengths, challenges and lessons learned from Timor-Leste (TLS) as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.’
Methods: Extensive discussion of methods leaves a few important gaps, such as ages/genders of the few community informants.
The table of documents reviewed could be in an appendix. Thank you for your feedback on this. This was an area that the authors had discussed extensively, and we had originally included both gender and age, but had decided to remove this for the protection of interviewees upon several of their request.
Results: A lot of useful results are presented, but some areas need more explanation. For example, we learn about the “Twinning Partnership for Improvement” (TPI) that seems to have a substantial effort on efforts to continue or improve quality. That entity was not well described and could potentially add a lot to an understanding of what happened to services during the pandemic. How important was the role of other partners? Were they a factor explaining why maternal care services performed better than general outpatient services?
Thank you for this feedback. We have clarified the usefulness of the TPI in the discussion section. It now reads: ‘To maintain quality EHS, planning for quality must be proactively considered across all levels in service delivery packages, municipal health offices and health facilities (44). While the national level sets the priorities, municipal health offices and facilities need to participate in strategic quality planning for implementation to minimize the interoperability gap (47). As such, the authors recommend for countries to consider aligning quality planning and learning systems throughout the health system to be better prepared to maintain quality EHS (44, 47). For example, by involving all three levels of the health system, the activities taken by the maternal and child health program during COVID-19 may have contributed towards minimizing the interoperability gap. The program leaders took an integrated approach to quality by developing national guidance, disseminating, and orientating that guidance to the municipal and facility levels, while also promoting healthy behaviors in the community. Another example is the TPI, which planned quality activities through a learning system between the national, municipal and facility levels and then later shared their lessons learned in implementing IPC improvements during COVID-19 to other municipalities (33, 48). This approach had impact on a referral hospital 18 months later, which helped reduce neonatal sepsis.’ Table 4 shows the ongoing support which was provided by partners, which is can have great impact if done in alignment with the national strategic plans We have also clarified this in the discussion: ‘To better prepare health systems to maintain quality EHS during a public health emergency, the analysis highlighted several key recommendations from TLS that may be of benefit to other countries. First, developing national level strategy and/or policy for quality will set the foundation for the national quality priorities (46). From here, national leaders can inform stakeholders, national and international agencies, and all health systems levels of the quality priorities. For example, table 4 shows that several health partners in TLS had ongoing activities that are supporting quality service delivery before and during COVID-19. Ensuring alignment of these partners to the national quality priorities can greatly contribute towards maintaining quality EHS during any future health emergency.’
The large Table 4 is useful but lacks some important information. Given the decrease in infacility maternal mortality cases – what are the data on overall maternal mortality? Health facilities have some information on home deliveries, which are an important risk for mortality. Were no infant mortality data available?
Thank you for this important feedback. Table 4 is ‘intended to show a mapping between WHO Primary health care measurement framework and indicators (34), the Donabedian Framework (35) and available data in TLS, which was collected from partners, MOH programs, and health facilities. The WHO Primary health care (PHC) framework highlights that health worker density and distribution, and existing policies for example, are health service determinants which reflect the capacity of PHC services. The framework also includes health service delivery indicators (eg. processes and outputs) that may impact health system objectives (outcomes). While the WHO PHC framework monitors progress and performance in PHC, the Donabedian Framework accounts for quality at all levels of care, including secondary and tertiary. Efforts to maintain quality in TLS are highlighted in this table and show quality-focused data before and during COVID-19 that was supported by MOH programs, CQAH and partners. For example, as part of the national direction on quality, TLS had given particular attention to water sanitation and hygiene (WASH) and IPC practices prior to COVID-19, which could have had some impact on maintaining quality EHS. Table 4 also helped direct the authors to formulate the KII questionnaires to better understand what contributed to improved maternal and child health services and outcomes verses the decrease in outpatient, emergency department, and primary care service visits.’ All available data on maternal and child health was included in the table, as it relates to the maintenance of essential health services
Another important issue is identifying what happened, if anything, to health staff availability during the pandemic. In many countries, health worker numbers were substantially decreased because of COVID illness. Those data would be very useful for Table 4. Also, at the bottom of p 7, an FGD participant mentions “worried…about being sent to a COVID-19 treatment center” as a reason for not using health services. That sounds like a fear of being required to go somewhere for treatment? Would be good to clarify.
Thank you for this useful comment and we agree with the reviewer. However, up to date health worker data was not available during COVID-19. We have listed health worker distribution and density as a health system determinate in table 4, which serves as an indication for how well the health system is equipped to support through health workers. We have revised table 4 to make the objective of table 4 stronger and more apparent.
Conclusion: This section is particularly unhelpful in summarizing the results of the case study. It’s essentially a review of what the country should do in the future, rather than what was learned--what the study found had happened during the pandemic. Summarizing a few key findings would be helpful, in areas such as the role of partners, the factors that led to even better maternal care use during the pandemic, or other factors that helped or hindered service provision during the pandemic at the three levels studied. Then the conclusion as to how to use that information for the future would be more meaningful.
Thank you. We have revised the conclusion and it now reads: ‘Maintaining quality EHS is a continuous process that must be considered in health emergency preparedness planning. By setting national level priorities in quality, stakeholders, national and international agencies, and all health facilities can align quality service delivery activities, particularly when resources are strained. Integrating quality planning into preparedness plans can bring awareness and accountability to health systems leaders to ensure that quality EHS will be maintained, while also empowering facility staff to continue advocating for quality during a public health emergency.
This study is a snapshot of quality EHS in TLS during the COVID-19 pandemic and the analysis identified approaches and recommendations that may be of benefit to other countries. Even though more knowledge is needed on how to operationalize plans for maintaining quality EHS, this learning can support countries towards better quality planning for public health emergencies.’
To Supriya Mathew
The paper is well written and discusses an important topic that is also of future use. The study documents the experiences of the Timor-Leste health system as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.
Sentence –“ no patients were involved in the recruitment…” page 6 is confusing.
Thank you for this feedback and we agree. We have since deleted and provided an updated statement. “No patients were involved in the recruitment to conduct the study or directly involved in the design of this study. The study will be made available to all participants and shared directly with MOH, municipal health departments and health facilities involved.”
Page 12 -the paper mentions that there was a decrease in service utilization- were there any quotes from the community/patient category that could be added below it?
Thank you for this comment. We have included the following: Table 4 also revealed a large decrease in OPD and emergency room visits (HNGV only). To understand the decrease in service utilization by community members, KIIs revealed that patients were not visiting emergency departments and OPD largely related to the “Lack of trust [in the health system].” [Municipal level, KII participant], “Increased fear [of the hospital].” [Facility level, FGD participant],
When probed further about their ‘lack of trust’ or ‘increased fear’, participants described family and community members, not going to the hospital because they were:
“Worried about being sent to a COVID-19 treatment centre [if becoming COVID-19 positive]” [Facility level, FGD participant].
“Essential Health Services did not work properly during COVID-19 because all staff focused on COVID-19. Chronic patients did not take their medications and they died as a result. Taxi and bus services were not running so patients could not afford to get to the hospitals for treatment.” [Facility level, KII participant]
Page 13 suddenly mentions one family member, is the quote from a community member? Were there any differences between rural and urban quality of care?
Thank you for this reflection. The quote is from a community member. We have revised to clarify: One family member said:
‘The IPC used to be great and implemented in all health facilities, but the consciousness is decreased, we can see limit of PPE and there is no social distance among health workers, and patients with health workers. This makes us not want to come’ [Facility level, KII participant]
Another family member stated:
“[My family member] has been here since morning at 10 am, however [they] did not get clinical care until 08.00 pm. After that [they] got clinical care by putting infusion.” [Facility level, KII participant]
We did not call out urban verses rural in health facilities as the capitol of Dili is mainly the urban populations and the other municipalities are largely rural.
Table 1: year of publication has been mentioned, could it be categorised into pre during and post COVID publications
Thank you for your feedback. This is a fine point. Will consider marking with an asterisk those documents that were post COVID 19.
Table 4: • Data has been provided only from 2019 onwards; is data available for at least 3 years before the COVID period?
This has been considered. We have since updated the table to make the objective of the table more clear.
May be add one more column to indicate insufficient data for comparison between years?
Thank you for the suggestion. The point of the table is not really to compare data as such, although it has been helpful to understand what services were ongoing during covid 19 verses which were not. A main objective of the table is understand what data is available to support ongoing quality activities. We have clarified the objectives of the table: ‘Developed during the document review, table 4 shows a mapping between WHO Primary health care measurement framework and indicators (34), the Donabedian Framework (35) and available data in TLS, which was collected from partners, MOH programs, and health facilities. The WHO Primary health care (PHC) framework highlights that health worker density and distribution, and existing policies for example, are health service determinants which reflect the capacity of PHC services. The framework also includes health service delivery indicators (eg. processes and outputs) that may impact health system objectives (outcomes) (see Figure 1).
While the WHO PHC framework monitors progress and performance in PHC, the Donabedian Framework accounts for quality at all levels of care, including secondary and tertiary. Efforts to maintain quality in TLS are highlighted in this table and show quality-focused data before and during COVID-19 that was supported by MOH programs, CQAH and partners. For example, as part of the national direction on quality, TLS had given particular attention to water sanitation and hygiene (WASH) and IPC practices prior to COVID-19, which could have had some impact on maintaining quality EHS. Table 4 also helped direct the authors to formulate the KII questionnaires to better understand what contributed to improved maternal and child health services and outcomes verses the decrease in outpatient, emergency department, and primary care service visits.’
The paper mentions that the patient satisfaction improved as per patient satisfaction survey. How were the surveys administered (e.g. online)? Were the sample sizes across each years comparable?
The sample sizes are comparable, and the surveys are administered in written format at the health facility.
-
Peer review report
Title: Country Learning on Maintaining Quality Essential Services during CVOID-19 in Timor Leste: A mixed methods qualitative study
version: 2
Referee: Mary Anne Mercer
Institution: University of Washington
email: mamercer@uw.edu
General assessment
A very extensive, complex review of the TLS health system performance during the COVID-19 pandemic.
Introduction: The aims of the study are to “…learn from the efforts…to maintain quality EHS” (p 1) as well as to “understand the variance in service utilization, including the best practices in improving maternal mortality and SBA.” That leads the reader to expect proposed answers to these questions in the conclusion, which didn’t happen.
Methods: Extensive discussion of methods leaves a few important gaps, such as ages/genders of the few community informants. The table of …
Peer review report
Title: Country Learning on Maintaining Quality Essential Services during CVOID-19 in Timor Leste: A mixed methods qualitative study
version: 2
Referee: Mary Anne Mercer
Institution: University of Washington
email: mamercer@uw.edu
General assessment
A very extensive, complex review of the TLS health system performance during the COVID-19 pandemic.
Introduction: The aims of the study are to “…learn from the efforts…to maintain quality EHS” (p 1) as well as to “understand the variance in service utilization, including the best practices in improving maternal mortality and SBA.” That leads the reader to expect proposed answers to these questions in the conclusion, which didn’t happen.
Methods: Extensive discussion of methods leaves a few important gaps, such as ages/genders of the few community informants. The table of documents reviewed could be in an appendix.
Results: A lot of useful results are presented, but some areas need more explanation. For example, we learn about the “Twinning Partnership for Improvement” (TPI) that seems to have a substantial effort on efforts to continue or improve quality. That entity was not well described and could potentially add a lot to an understanding of what happened to services during the pandemic. How important was the role of other partners? Were they a factor explaining why maternal care services performed better than general outpatient services?
The large Table 4 is useful but lacks some important information. Given the decrease in in-facility maternal mortality cases – what are the data on overall maternal mortality? Health facilities have some information on home deliveries, which are an important risk for mortality. Were no infant mortality data available?
Another important issue is identifying what happened, if anything, to health staff availability during the pandemic. In many countries, health worker numbers were substantially decreased because of COVID illness. Those data would be very useful for Table 4. Also, at the bottom of p 7, an FGD participant mentions “worried…about being sent to a COVID-19 treatment center” as a reason for not using health services. That sounds like a fear of being required to go somewhere for treatment? Would be good to clarify.
Conclusion: This section is particularly unhelpful in summarizing the results of the case study. It’s essentially a review of what the country should do in the future, rather than what was learned--what the study found had happened during the pandemic. Summarizing a few key findings would be helpful, in areas such as the role of partners, the factors that led to even better maternal care use during the pandemic, or other factors that helped or hindered service provision during the pandemic at the three levels studied. Then the conclusion as to how to use that information for the future would be more meaningful.
Essential revisions that are required to verify the manuscript
Professional proofreading would correct a number of typos.
Important error: on page 10, Table 4: I assume that for the Objectives/Indicators row, the proper indicator six rows down should be Postnatal or Postpartum Care coverage (rather than antenatal care, which has already been presented…the 1 week or 1-6 week category makes no sense for antenatal care).
Other suggestions to improve the manuscript
This document is far too detailed for publication in a journal; it would need substantial editing if it were meant for that purpose. However, assuming that it is meant primarily for national use, it does discuss a wide range of issues and practices that were meant to affect the quality and utilization of care during COVID and with revisions would be well worth publishing.
Decision
Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.
-
Peer review report
Title: Country Learning on Maintaining Quality Essential Services during CVOID-19 in Timor Leste: A mixed methods qualitative study
version: 2
Referee: Supriya Mathew
Institution: Menzies School of Health Research
email: supriya.mathew@menzies.edu.au
ORCID iD: 0000-0002-8078-3708
General assessment
The paper is well written and discusses an important topic that is also of future use. The study documents the experiences of the Timor-Leste health system as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.
Sentence –“ no patients were involved in the recruitment…” page 6 is confusing
Page 12 -the paper mentions that there was a decrease in service utilisation- were there any quotes from the community/patient category that could be added below it?
Page 13 suddenly mentions one …
Peer review report
Title: Country Learning on Maintaining Quality Essential Services during CVOID-19 in Timor Leste: A mixed methods qualitative study
version: 2
Referee: Supriya Mathew
Institution: Menzies School of Health Research
email: supriya.mathew@menzies.edu.au
ORCID iD: 0000-0002-8078-3708
General assessment
The paper is well written and discusses an important topic that is also of future use. The study documents the experiences of the Timor-Leste health system as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.
Sentence –“ no patients were involved in the recruitment…” page 6 is confusing
Page 12 -the paper mentions that there was a decrease in service utilisation- were there any quotes from the community/patient category that could be added below it?
Page 13 suddenly mentions one family member, is the quote from a community member?
Were there any differences between rural and urban quality of care?
Table 1 : year of publication has been mentioned, could it be categorised into pre during and post COVID publications
Table 4:
• Data has been provided only from 2019 onwards; is data available for at least 3 years before the COVID period?
• May be add one more column to indicate insufficient data for comparison between years?
• The paper mentions that the patient satisfaction improved as per patient satisfaction survey. How were the surveys administered (e.g. online)? Were the sample sizes across each years comparable?
Essential revisions that are required to verify the manuscript
Nil
Other suggestions to improve the manuscript
The paper uses too many acronyms which makes it hard to read the paper. It would be good to expand acronyms at least at the beginning of each major section. The paper could be improved by a final proofreading as there are a few typos/grammatical errors in the paper. The reference section needs to be proofread as well.
Examples of typos in the paper:
• Use of past tense: Page 3: ‘there was a decrease in maternal mortality
• Page 8 category of first couple of quotes not mentioned
• Conclusion subtitle spelling is wrong
• Quotes could be edited for readability. For example,
• Page 8 last quote “To guarantee ..” unclear
• Page 9 first quote could be edited slightly for readability
• References 34 author format, ref 18 is the title right?
Decision
Verified: The content is academically sound, only minor amendments (if any) are suggested.
-
Peer review report
Title: Country Learning on Maintaining Quality Essential Health Services (EHS) during COVID-19 in Timor-Leste: A mixed methods qualitative analysis
version: 2
Referee: Alvaro Alonso-Garbayo
Institution: Independent consultant
email: alvaro_alonso@yahoo.com
ORCID iD: https://orcid.org/0000-0002-4120-
I am currently under a consultancy contract with WHO which represent a CoI as per PeerRef policy. Therefore no decision on the preprint is provided.
General assessment
Methods:
Lit. review: suggest explaining how data obtained from each document identified was extracted, gathered and analysed
Suggest including a detailed description of the sample (e.g. demographic characteristics) including gender
Mistakes in literature gathered/analysed (Table 1):
National Health Sector Strategic Plan II 2011-2030 doesn’t match with …
Peer review report
Title: Country Learning on Maintaining Quality Essential Health Services (EHS) during COVID-19 in Timor-Leste: A mixed methods qualitative analysis
version: 2
Referee: Alvaro Alonso-Garbayo
Institution: Independent consultant
email: alvaro_alonso@yahoo.com
ORCID iD: https://orcid.org/0000-0002-4120-
I am currently under a consultancy contract with WHO which represent a CoI as per PeerRef policy. Therefore no decision on the preprint is provided.
General assessment
Methods:
Lit. review: suggest explaining how data obtained from each document identified was extracted, gathered and analysed
Suggest including a detailed description of the sample (e.g. demographic characteristics) including gender
Mistakes in literature gathered/analysed (Table 1):
National Health Sector Strategic Plan II 2011-2030 doesn’t match with reference 7 but ref. 9
Name of document: Ref. 18 “Ministry of Financial Annual Report” (?)
The only document classified as “municipal level” (e.g. “Exceptional and temporary measures of health surveillance in response to C-19 pandemic”) is not municipal but a national law and it was not published in 2020 but in 2021
It is surprising that given the indicators used to measure quality among which there are reproductive health ones, UNFPA has not been identified as a main source of information for relevant literature and/or included relevant UNFPA staff as a key informant for interviews
The section “Patient and Public Involvement” doesn’t make clear the link between the literature review and the decision to involve users (which should have been made anyway independently of the results of the literature review). The statements “No patients were involved in the recruitment to conduct the study (?) and patients were not involved in the design of this study (?)” are not clear (review/revise). The last phrase seems to be copy and paste from the study protocol (future tense?). What was the purpose of sharing “the study” with participants? Did they give any feedback? If so, what can be concluded from it?
Table 4 would be more informative (e.g. covering pre-pandemic and pandemic periods) if more data from 2021 would have been included
Table 4 requires full revision: some examples of mistakes/issues
How data about density of [key] health workers from only one year (2019) can inform efficiency? How density of medical doctors (only data from 2020) provides information about health worker distribution? How that links with efficiency?
Physical infrastructure: % of population -or facilities?- with basic hand washing facilities
Safety: how a reference dated in 2018 (i.e. Ref. 36) can provide information about 2019, 2020 or 2021?
Maternal mortality decrease (from 20 to 16 per 100,000 live births – measuring unit not included in the table) reported only from facilities doesn’t match with the previous statement in the introduction reporting decrease in [overall?] maternal mortality. This is particularly important in Timor-Leste where delivery at home assisted by grandmothers is culturally well rooted and still relatively common practice (particularly in rural/remote areas where access to health services may be difficult). This most probably have pushed pregnant women to choose this option during the pandemic which most probably, and despite the increase in institutional delivery reported (not sure but think that home delivery in TL is considered skilled birth if attended by a skilled professional), has increased the overall MMR.
Discussion:
- While the methods section reports that the selection of participants for KIIs looked for informants with different backgrounds (e.g. rural/urban), discussion about the different perspectives provided by different participants is missing. Most importantly there is no mention to gender perspectives as seems that this characteristic was not sought during recruitment.
Overall:
- The manuscript would benefit of a final proof reading in English to solve orthographic and grammatical issues
-
-