Exploring the feasibility of establishing a core set of sexual, reproductive, maternal, newborn, child, and adolescent health indicators in humanitarian settings: results from a multi-methods assessment in Jordan

Abstract

Background

Reliable and rigorously collected sexual, reproductive, maternal, newborn, child, and adolescent health (SRMNCAH) data in humanitarian settings is often sparse and variable in quality across different humanitarian settings. To address this gap in data quality, the World Health Organization (WHO) developed a core set of indicators for monitoring and evaluating SRMNCAH services and outcomes in humanitarian settings, and assessed their feasibility in the field in Jordan, in addition to three other countries; with the goal of aggregating information from global consultations and field-level assessments to reach consensus on a set of core SRMNCAH indicators for services and outcome evaluation in humanitarian settings among WHO global partners.

Methods

The feasibility assessment in Jordan focused on the following constructs: relevance/usefulness, feasibility of measurement, systems and resources, and ethical issues. The multi-methods assessment included five components; a desk review, key informant interviews, focus group discussions, facility assessments, and observational sessions.

Results

Findings suggest that there is widespread support among regional, national, as well as global stakeholders for developing a core list of SRMNCAH indicators for monitoring and evaluation of services and outcomes in humanitarian settings in Jordan. There are numerous resources and data collection systems that could be leveraged, built upon, and improved to ensure the feasibility of collecting this proposed set of indicators. However, the data collection load requested from donors, the national government, international and UN agencies, coordination/cluster systems must be better harmonized, standardized, and less burdensome.

Conclusions

Despite stakeholder support in developing a core set of indicators, this would only be useful if it has the buy-in from the international community. Greater harmonization and coordination, alongside increased resource allocation, would improve data collection efforts and allow stakeholders to meet indicators’ reporting requirements.

Plain English summary

In humanitarian settings, data collection is often unreliable and not standardized – especially with regards to sexual, reproductive, maternal, newborn, child, and adolescent health (SRMNCAH). In order to address this gap in data quality, the World Health Organization proposed a list of core indicators to four countries, including Jordan. The goal was to use the findings from the Jordanian context to contribute to the determination of a core set of indicators, and accumulate a list of necessary policy, funding, infrastructure, capacity and training that will be required. This study had two components: a feasibility assessment and a multi-methods assessment. The feasibility assessment in Jordan focused on the relevance of the proposed indicators, the feasibility of measurement, the resources in place, and ethical issues. The multi-methods assessment included a desk review, interviews with key informants and focus groups, and facility assessments (including observational sessions). The findings showed that partners in Jordan supported developing a standard set of SRMCAH indicators that can be used among all humanitarian partners in the country. There are a number of systems in place that could be used or strengthened to improve data collection. However, the findings also showed that there must be better collaboration and coordination between the different partners in Jordan, as well as increased resources, to alleviate burden on front-line staff, starting with standardizing and strengthening Jordan’s health information system (HIS). In conclusion, though there is a desire to harmonize indicators, more input and resources are needed from the international community to aid in the standardization of data collection in order to meet local reporting requirements and ease burden on local staff.

Background

Syrian crisis in Jordan

As the Syrian crisis continues, Jordan hosts more than 650,000 registered Syrian refugees in both urban and camp settings [1], and another estimated 1.2 million unregistered refugees in urban and rural areas across the country [2]. The majority of Syrian refugees in Jordan (79%) live outside refugee camps [1] in urban and rural areas throughout the country, and Syrians who reside in these communities have less access to, or have more difficulty navigating, local health services [3]. As of 2021, approximately fifty percent of refugees in Jordan are women and girls. Of this population, 160,000 are women of reproductive age [2], an estimated 16,000 of whom are pregnant at any given time [4]. More than half of these pregnant women are under the age of 25 [5]. This has caused considerable strain on Jordan’s already weak existing infrastructure and consequently imposed many implications on sexual, reproductive, maternal, newborn, child, and adolescent health (SRMNCAH) services, access, and outcomes [2, 3, 6, 7].

SRMNCAH data collection and indicator reporting

Humanitarian emergencies are frequently characterized by the collapse and fragmentation of basic health services [8, 9]. In addition to the strain caused by humanitarian emergencies on host countries’ health systems and services, the large influx of refugees into Jordan has placed an unsustainable strain on the economy [10, 11], infrastructure, and security systems [12]; consequently, organizations are unable to meet the simultaneous demands of both refugees and its residents [9]. For better decision-making, coordination and response in crisis settings such as Jordan, multi-lateral and bilateral humanitarian agencies and their implementing partners need access to appropriate information [13,14,15]. Over the last few decades, researchers have extensively reported that humanitarian emergencies could generally lead to either a shortage or an overload of information—and both situations impair the provision of effective humanitarian assistance [15, 16].

Timely and rigorous collection, aggregation, and use of SRMNCAH data for services and outcomes evaluation in humanitarian settings is an important component of accountability and transparency within the aid community, and most importantly, accountability for the populations that the programs and developments are catered to [17]. Collecting complete, reliable, and quality data is also key to increase aid efficiently and effectively for populations affected by emergency and humanitarian crises [16, 18]. Transparency, accountability and measurement of the impact of humanitarian interventions are not only important for humanitarian organisations but are often requirements set by donors [15, 19]. Rigorous data collection and analysis would contribute to measurable assessments of the objectives, goals, and purpose of humanitarian initiatives in terms of process, impact, and intended outcomes [20, 21].

SRMNCAH data collection and indicator reporting in Jordan

In the Jordanian context, multiple concerted efforts have been made to address the gaps in availability of SRMNCAH data. The most recent of such efforts is led by the Higher Population Council National Strategy for Reproductive Health and Family Planning (NSRHFP) by creating guidelines to support and engage key stakeholders in the coordination of all sexual and reproductive health (SRH) efforts in the country, including the data needed to meet the agreed upon national goals and global standards of care [22]. However, with a three-tiered health care system, limited budgets, and varying policies and regulations, the development of national priorities and guidelines for improving SRMNCAH indicators’ reporting for its population—and for refugee populations more specifically – has proven to be challenging [23].

Aims and objectives

In light of the above, WHO, in close coordination with local, regional, and global partners, agreed to test the feasibility of a candidate set of SRMNCAH indicators for humanitarian settings at the field level. This was done in four countries experiencing different types of humanitarian crises, including Jordan, to determine the feasibility, relevance, and acceptability of these indicators. In the article, we discuss the results of this assessment in Jordan. The assessment took place in both camp and urban settings in the capital city of Amman as well as in Irbid and Mafraq (Al-Za’atari), and Karak and Aqaba in the North and South regions of Jordan, respectively.

By assessing feasibility, we aimed to explore the potential impact of the intended data collection and analysis, whether or not national and non-governmental monitoring and evaluation systems have the needed resources to collect SRMNCAH indicators, and the ability of the system to adhere to ethical practice and safeguard clients’ confidentiality and privacy. The results of Jordan’s country level assessment will be eventually aggregated with the results from other field-level assessments in order to reach a global consensus on a minimum set of core SRMNCAH indicators for services and outcomes evaluation in humanitarian settings among donor agencies, UN agencies, and international NGOs working in humanitarian settings. Specifically, this paper focuses on providing an overview of the assessment results in Jordan to determine the feasibility, relevance, and acceptability of the M and E framework, based on the perception of the multiple stakeholders of this assessment.

Methods

Study design

This multi-methods assessment consisted of five main components: (1) a desk review of published articles and reports as well as internal documents (in English and Arabic); (2) key informant interviews with representatives from government entities, international non-governmental organizations, and community-based organizations; (3) facility assessments at primary, secondary, and tertiary facilities that provide services to refugees in Jordan; (4) observation sessions focused on the logistical, ethical, and privacy practices associated with data collection and storage at select facilities; and (5) focus group discussions (FGD) with frontline workers at primary, secondary, and tertiary health centers (see Fig. 1).

Fig. 1

Data collection for Phase II. NGO Non-governmental organization; CBO Community based organization

The assessment centered on seeking and understanding different stakeholders’ perceptions and attitudes towards: SRMNCAH issues in Jordan, SRMNCAH service provision in Jordan for refugee populations, current reporting practices on SRMNCAH indicators, and the feasibility of reporting on the candidate set of core SRMNCAH indicators; and also the necessary buy-in needed from the sector to successfully nationally scale up, endorse and report against these indicators.¹

Desk review

The project was initiated with a comprehensive review of peer-reviewed literature, existing published and unpublished data, including institutional and donor reports that focused on SRMNCAH indicators’ reporting and analysis in Jordan; coupled with an in-depth examination of the national SRMNCAH indicators’ list that organizations are required to report against. This desk review also helped informed the selection of the target populations for each of the KIIs and FGDs.

Field-level assessment (See Fig. 1).

Key Informant Interviews (KIIs)

We compiled a list of key agencies working and providing SRMNCAH services to Syrian refugees across Jordan; as the majority of offices for these agencies were headquartered in Amman, we interviewed KIs individually or in small groups in Amman, Jordan. We conducted 10 KIIs with 27 representatives from different entities including government entities, and national and international non-governmental organizations between January and February 2020. Using a semi-structured interview guide developed specifically for the overarching study, we focused on KIs’ perceptions and attitudes towards: SRMNCAH issues in Jordan, SRMNCAH service provision in Jordan for refugee populations, current reporting practices on SRMNCAH indicators, and the feasibility of reporting on the candidate set of core SRHMNAH indicators. We also explored stakeholders’ perceptions and attitudes of current challenges in documenting and resources needed to successfully report against these indicators. We further explored the necessary buy-in needed among donor, governmental, and non-governmental agencies to enable the success of this effort.

Facility assessments

A list of facilities that provided RH services to Syrian refugees in Al-Za’atari camp, Irbid, and Karak was compiled by level (primary, secondary, and tertiary facilities). Each facility was informed, and WHO and United Nations Population Fund (UNFPA) country offices in Jordan facilitated the needed authorization prior to the evaluation. A total of five facility assessments were conducted. These assessments aimed to determine the nature and extent of SRMNCAH services offered, the ways in which patient information was collected, logged, stored, and safeguarded, and the types of human and technological resources used in data capture. In conjunction with the facility assessments, observational sessions were also carried out in all five facilities. These observational sessions aimed to assess existing resources currently being employed to collect data and additional resources needed to collect additional needed data for the core set of SRMNCAH indicators.

Focus group discussions

The facilities identified as the largest subsidized providers for SRMNCAH services for Syrian refugees in their respective governorates were selected for the FGDs. Five FGDs were conducted with 27 frontline workers from the same five agency health care clinics. Participants provided verbal consent at the beginning of each FGD, which lasted an average of 90 min and took place in Arabic. With consent, we audio-recorded all five FGDs, debriefed as a team after each discussion, and wrote analytic memos to capture group dynamics and identify early themes.

Analytic approach

An iterative, multi-phased approach was employed to analyse the data [24, 25]. All KIIs and FGDs were analysed for content and themes, using both inductive and deductive techniques, which were then combined with results from the facility assessments, validation discussions with KIs, and feedback from the WHO led to the final recommendations. The analysis focused on the four core elements: (1) feasibility of collecting the proposed core set of SRMNCAH indicators, (2) relevance and usefulness of SRMNCAH data management mechanisms; (3) availability of existing resources and systems for national and humanitarian SRMNCAH data collection; and (4) ethical considerations of collecting and storing data.

Research ethics

The Research Project Review Panel (RP2) of WHO’s Department of Sexual and Reproductive Health reviewed and approved this study. We also obtained authorization from the WHO’s country office in Jordan and the Jordanian Ministry of Health (MoH). The Social Sciences and Humanities Research Ethics Board of the University of Ottawa provided ethical approval (Protocol number: S-08-18-1029).

Findings

The results of this assessment focused on the four core elements highlighted in the study objectives: (1) feasibility of collecting the proposed core set of SRMNCAH indicators, (2) relevance and usefulness of SRMNCAH data management mechanisms; (3) availability of existing resources and systems for national and humanitarian SRMNCAH data collection; and (4) ethical considerations. We first start by outlining the feasibility of the collecting the proposed SRMNCAH indicators, then we move to describing the current and potential advantages as well as challenges with SRMNCAH data capturing, followed by outlining the available data collections systems for the proposed indicators by the different humanitarian agencies. The findings section will conclude with discussing the different enforced measures in Jordan to protect data privacy and confidentiality among the different implementing agencies.

Feasibility

The findings of this assessment indicated that 48% of the proposed indicators were considered relevant and feasible to collect (see Table 1); many of the STI and RTI (100%), newborn (81%), contraception (75%), maternal (53%), and abortion (40%) indicators were perceived to be relevant and feasible (see Tables 1 and 2). Of these indicators recommended for inclusion; 45% of indicators, reportedly, are currently being collected and 14% of those that are not currently being collected were perceived to be able to be collected given resources and trainings (see Table 2). The findings also indicated that 48% of the proposed indicators were deemed unfeasible or irrelevant; especially indicators in child (30%) and adolescent health (17%), HIV, and prevention from mother to child (PMTC) (Table 2).

Table 1 Summary and percentage of the indicators by domain that were perceived relevant and to the Jordanian context, by number and percentage respectively

Table 2 Summary findings of the feasibility of collecting the following proposed SRMNCAH indicators in the humanitarian context of Jordan

The findings from this assessment revealed that Jordan has multiple existing data collection information systems. These systems often compete with each other, which results in gaps in data collection and place undue burden on frontline staff. Lack of internal capacity and staffing impede the ability of organizations to develop and implement comprehensive computerized data capturing and reporting systems across health facilities. Frontline workers are required to report to multiple systems utilizing various templates and requiring different information. With this pre-existing arduous process in place, frontline workers expressed reticence at collecting a list of this length and perceived that the standardization of indicators is necessary to achieve feasibility of their data collection. While many perceived that some of the new proposed indicators in the framework could be easily integrated into existing systems for collection; the feasibility of collecting an indicator did not correlate with the relevance of the indicator, as participants highlighted that such data collection necessitates the availability of the needed resources to ensure the accuracy of the collected data. In Table 2, we provide an overview of the included and excluded list of indicators, the reported percentage of agencies who are currently collecting these indicators; sites of data collection; perceived facilitators and barriers for routine data collection; and any necessary modifications and resources necessary for routine data collection.

The results showed that three of the four proposed indicators on contraceptives were feasible to collect (indicators 1.1–3) given some wording modifications. The indicator connected to post abortion care (PAC) (indicator 1.4) was suggested for exclusion due to the legal status of abortion care in Jordan. It was indicated that policy and legislative changes are needed in parallel with training on the legal status and new IAFM guidelines along with VCAT sessions for frontline workers to make abortion indicators feasible for data collection in the Jordanian context.

The maternal health indicators were perceived as fairly feasible to collect, with the exception of indicators on service availability in the maternal health sector (indicators 3.9-12). Participants perceived that these indicators are not feasible to collect because of low burden of disease and sporadic service provision (indicators 3.13-16). Most of the proposed newborn health indicators relied on facility-based information and therefore are either being collected or were perceived to be easily feasible to be collected (indicators 4.1–8). Participants, however, expressed concerns surrounding the way that neonatal deaths and stillbirth are defined, recorded, and audited, particularly for cases outside of camps (indicators 4.1-2). On the other hand, participants expressed concerns over the feasibility of collecting a number of indicators related to child (indicators 5.2-6) and adolescent health (indicators 6.2-6), because these proposed indicators are population-level indicators and participants were concerned with impracticalities associated with measuring the needed denominators and/or the low burden of disease (indicators 5.7–9) in the local context.

Our findings suggested that significant gaps exist around data collection for sexual and gender-based violence (SGBV)-related indicators (indicators 7.2-7). Deep concerns were expressed that public reporting of SGBV information could result in a de-prioritization of efforts to expand clinical management of rape and GBV services. While none of the agencies that participated in this project are currently reporting any data on pregnancy outcomes for rape survivors or related abortion requests (indicator 7.5), participants felt this information could be collected if appropriate resources, ownership, supports and training were adequately set in place. Finally, participants raised concerns about the inclusion of any of the HIV/AIDS-related indicators (indicators 8.1-3 and 9.1-9.4) due to major stigma and discrimination from service providers, and the legislative barriers to providing care to refugees in Jordan who are screened positively with HIV/AIDs; in parallel with the low prevalence of disease in Jordan. We provide additional information about the specific indicators proposed for each topic in Table 2 and provide a detailed narrative for each domain of indicators in WHO’s Jordan country-level report (available upon request).

Relevance and usefulness of humanitarian SRMNCAH data management mechanisms

Perceived advantages with current and proposed SRMNCAH indicator reporting

All the different participants in this assessment agreed that accurate and reliable SRMNCAH data provide the opportunity for implementing evidence-based programming, defining priorities, ensuring accountability among implementors. Hence, the majority of the proposed indicators were perceived to be essential in order to improve the health outcomes for both refugee and vulnerable communities in Jordan. In addition to the benefits of meeting the needs of refugee and vulnerable populations, our KIs noted that the collection and reporting of SRMNCAH indicators allows organizations to monitor and evaluate whether SRMNCAH services provided meet global and regional standards of care, population health goals, and donor reporting requirements. As noted by one KI, “We can’t capture everything, we have to be strategic with which indicators we select. We prioritize of course the indicators that contribute to our country program documents, the ones that align with the Sustainable Development Goals and of course to meet our reporting requirements [to donors].”

However, stakeholders stressed the need for a core list of SRMNCAH indicators and encouraged that this same list to be used across all the different stakeholders simultaneously i.e. donors, the MoH, UN agencies, and international NGOs. In general, the stakeholders in all agencies supported the contours of the proposed core set of SRMNCAH indicators and noted that many of the specific indicators are aligned with many of their current and ongoing data collection practices and priorities. Stakeholders noted that across the board, that indicators derived from facility-level data were more relevant, useful, and practical to collect compared to those indicators that required information at the population level. As noted by an FGD participant, “The facility level indicators have both the potential to be more precise and really useful at the programmatic level in a way that population-level indicators tend to be harder to collect and are not necessarily accurate.” Stakeholders also repeatedly asserted that having a standardized set for SRMNCAH indicators is important for advocacy-related activities, including establishing or amending policies to help increase access to services for refugee populations and supporting requests for increased funding.

Perceived disadvantages with current SRMNCAH indicator reporting

Our findings also indicated several disadvantages that could be associated with collecting some of the proposed SRMNCAH indicators. As noted by many of our stakeholders and FGD participants, a multitude of barriers contribute to the significant under reporting for SGBV cases in Jordan. For instance, many of the stakeholders felt that publishing the low numbers of rape and other reported SGBV-related cases would undermine the prioritization of those services, as this would be interpreted as lack of demand for this service. As one KI explained, “Recording the data [SGBV cases] is okay but presenting it as an indicator that may be published to external agencies? These small numbers can be misleading. They will argue you don’t have a decent issue.” Other indicators equally perceived as controversial were some of those related to the adolescent health, child health and HIV/AIDs.

Perceived gaps in the proposed SRMNCAH indicators

Indicators that should be removed from the core set of SRMNCAH indicators

Participants identified a number of the proposed indicators to either be not relevant or useful in the Jordanian context and therefore recommended their removal. The rationale for suggesting their exclusion, as per the KIs, revolved around one or more of the following: (1) the indicator relates to a condition for which there is low burden of disease in the Jordanian context; (2) national regulations and protocols restrict collecting information on the subject; (3) services related to the indicator are not (routinely) available; and/or (4) the indicator would not have any practical or actionable applications. Stakeholders were uniform in their belief that population-level indicators were not practical and could only be collected in conjunction with larger nationally representative surveys that take place every 5–10 years. More information on these specific indicators suggested for removal along with the reasons for removal can be found above in Table 3.

Table 3 List of indicators that should be excluded according to our stakeholders, with primary rationale

All Jordanian stakeholders raised concerns about the inclusion of HIV/AIDS-related indicators. Our KIs explained that due to national regulations and policies, all HIV/AIDS related cases must be referred to the MoH and foreigners who test positive for HIV are subject to immediate deportation—including refugees. Other indicators were deemed irrelevant because services are not routinely provided in Jordan’s context, and policies and regulations hindering adequate capturing of data. These included indicators related to: comprehensive abortion care (CAC), syphilis screening, kangaroo mother care (KMC), and the LIVES² intervention in cases of intimate partner violence. Indicators related to adolescent suicide rates and sexual violence against children were generally perceived as controversial for the participants of this study. KIs expressed that the indicator on suicide rate wouldn’t be actionable and presents as an outlier from the core list of SRMNCAH indicators. The findings also suggest that indicators with population-level denominators should be removed and even if it could be collected, the data wouldn’t inform programming due to, as it stands in Jordan, available population level data are unreliable.

Additional Indicators that should be added to the core set of SRMNCAH indicators

Stakeholders in Jordan also proposed some additional indicators for inclusion to the core SRMNCAH list. Those suggested indicators were mostly focused on adolescent sexual and reproductive health, supply-chains, commodities/stock outs, and coordination. Related to the issue of child/early marriage, both KIs and FGD participants felt that the proposed maternal death indicator needs to be further disaggregated by age, as the risk of maternal morbidity and mortality differed for those 15 years of age and under compared to those aged 16–18. On Fig. 2, we provide a list of additional topics that the Jordanian stakeholders perceived should be included on any core SRMNCAH indicator list.

Fig. 2

Topics that should be included in the core list of SRMNCAH indicators

Existing systems and resources for collecting SRMNCAH indicators

The findings from this assessment indicated that several robust health information systems are developed, implemented, and utilized by both international and national partner organizations in Jordan. In the last few years, there has been a significant national investment in the amelioration of national reporting systems for a subset of the proposed SRMNCAH domains, including systems for contraception, maternal health, and SGBV. Access to comprehensive, user-friendly, computerized systems, necessary staff, and capacity varied across agencies. As a result, agencies often had to input and analyze their data manually, impacting accurate, standardized, and timely data reporting. Although, a number of organizations have built their own dedicated HIS internally, not all agencies have access to the same level of funding and/or the same human and technological capacities, meaning that some agencies are working with less contextually developed systems. We provide a list of the available data collection resources and systems reported by our stakeholders in Table 4.

Table 4 Existing monitoring and evaluations systems reported by our stakeholders, by domain of indicators

Ethical considerations

The findings from the KIIs, FGDs and facility assessments suggested that there are inconsistent and fragmented efforts for data protection and confidentiality surrounding SRMNCAH data and research across the different humanitarian agencies in Jordan. Health care facilities have basic protocols in place for data logging and protection, especially surrounding GBV data and access to the gender-based violence information management system (GBVMIS). However, the availability of data privacy and confidentiality protocols was directly correlated with the resources allocated for data collection and protection. Many agencies were able to run health care clinics and allocated funding to specialized Monitoring, Evaluation, Accountability and Learning (MEAL) departments and staff, institutional protocols, and a comprehensive HIS; yet, the majority of the health facilities we visited (n = 4) utilized Microsoft Excel for delivering and storing data. This was done mainly on an institution’s computer with a fixed password, an e-mail, a USB flash memory, or even tally-sheet printouts or the use of mobile phones. Some of the facilities visited kept a manual patient logbook could be accessed by ‘any’ staff member (n = 2), while the remaining facilities (n = 3) had specific (often the head nurse or midwife) on shift who had access to the master patient logbook.

All of our stakeholders discussed concerns with confidentiality and the national mandatory reporting requirements for GBV and early marriage. Participants explained that humanitarian organizations working in Jordan are mandated to abide by the country’s laws and regulations. This further exacerbates the underreporting of SGBV and early marriage cases for refugees residing in Jordan [26]. Similar challenges were noted surrounding the reporting of HIV and PMTCT-related indicators. Given that refugees face deportation if they are diagnosed with HIV [27], stakeholders had deep concerns about confidentiality, and most do not keep internal records. Noting that all refugees testing positive for HIV can only receive treatment through the MOH mandate and dedicated facility. As a participant explained,

We inform them [the HIV positive refugee] that we will have what we call a gentleman’s agreement with the MOH. When it comes to that refugee, if he or she found to be positive, we inform them that we will find a durable solution for the case, and that’s resettlement. So, they wait for us, until we get the settlement³ for the case.

Discussion

The findings from this multi-methods feasibility assessment provide a comprehensive overview of the feasibility of collecting a core set of SRMNCAH indicators in the humanitarian context of Jordan for improved humanitarian response. The outlined recommendations are steppingstones for SRMNCAH service and outcomes monitoring and evaluation in humanitarian settings. The recommendations also aid communities, donors, and humanitarian actors in creating an enabling environment for quality SRMNCAH data collection and evidence-based decision making. In Fig. 3, we outline the policy, strategy, capacity, training, HR, and communication factors that are supported by the literature that need to be implemented and addressed at the global and national, programmatic and facility levels to increase the feasibility of current indicator reporting practices and the quality of SRMNCAH data reporting [12, 14, 28,29,30,31].

Fig. 3

Challenges and solutions for timely, reliable, quality SRMNCAH indicator reporting at the facility, programmatic, and national and global levels in Jordan, reported by the study participants

With a three-tiered health care system,⁴ limited budgets and varying policies and regulations in Jordan [32], the development of national priorities and guidelines for improving SRMNCAH indicators’ reporting for its population—and for refugee populations more specifically—has proven to be challenging. The literature indicates that despite the surfeit of data requested by the overarching humanitarian structure, researchers have found systematic reporting gaps for different SRMNCAH programming and evidence for impact on SRMNCAH and outcomes [15, 33, 34]. In 2014, Obrégon and colleagues concluded that there were major gaps in basic information needed to assess the quality and context of the data and reports they reviewed were lacking, irrespective of state of emergency or region [33]. A noted challenge to this by the research respondents, is that the requested data collection from donors, the national government, international and United Nations (UN) agencies, coordination/cluster systems is highly variable and burdensome. A mixed-methods assessment of routine health information systems and data conducted in Addis Ababa found that the extensive presence of parallel reporting and unstandardized routine data collection practices resulted in over and under reporting of health indicators and had negative implications on service provider’s perceptions toward routine health data collection practice; impacting data completeness and quality [35]. Jordanian stakeholders were clear that this core set of indicators would only be useful with buy-in from the global community that results in harmonizing and coordinating data collection efforts and relevant indicators’ reporting requirements. The literature outlines the incompatibility between de-centralized humanitarian organizational structure (i.e., allowing for a high degree of independence for sub and field offices, with minimal oversight by the headquarter (HQ), including HQ’s HIS infrastructure [16]. The availability of a robust computerized system will therefore directly depend on the budget and prioritization of monitoring systems [36], combined with a focus om preventing standardized horizontal communication and encouraging coordination efforts between and within different agencies [16]. Therefore, the sustainability of these data collection systems is directly dependent on the agencies’ and donors’ willingness to provide regular and continuous multi-year funding for national and international organizations, which currently is often crippled.

Stakeholders were clear that collecting the SRMNCAH indicators that required a population-level denominator may not be often feasible, particularly in urban environments. More than 70% of all Syrian refugees in Jordan reside outside of camps [1]. Although, these populations are concentrated in some areas, this population was only fully counted after completion of the most recent Jordan Population and Family and Health Survey (JPFHS), the results of which were published in 2018 [37]. Participants noted that the Demographic and Health Survey Program cycle is pre-determined. It requires significant resources and should continue to be collected and coordinated at the national level, which could impact the data quality of those indicators that rely on population level denominators. Study participants repeatedly emphasized their preference for a core set of facility-based indicators, as they perceived them to be more programmatically relevant and actionable. Guha-Sapir and Scales (2020) found that the use of facility-based data, preferably in the form of patient records, improved data analysis and quality, and strengthens the estimation of mortality and morbidity in emergency settings [38]. The study findings also suggest that facility-based indicators could be used as proxies to reflect recognized services delivery standards of SRMNCAH care in humanitarian settings, as most are pertinent to the Minimum Initial Services Package (MISP). Hence, for those indicators, even if they were not currently being collected, KIIs and FGD participants could generally envision how to incorporate this information into the facility-level data collection systems.

Our findings also indicated that there are socio-political as well as cultural barriers that could impede the collection of certain SRMNCAH indicators, specifically those surrounding sexual and gender-based violence (SGBV), abortion, and HIV/AIDs domains of indicators. A study exploring barriers of reporting of sexual violence in Al-Za’atari camp found that Syrian women did not report these cases, due to shame, limited trust in helpers and insufficient legal protections [26]. The literature shows that mobilizing innovative and validated methods such as “the Neighbourhood method” for estimating prevalence rates with an accuracy could be a suitable alternative for programmatic purposes in conflict-affected and post-conflict settings [39]. Adopting novel approaches for estimating abortion incidence could also help mitigate the socio-cultural barriers to access, disclosure and reporting [40]. Further, although Jordan does not currently have a dedicated product for emergency contraception, other modalities that can be used post-coitally including the use of the copper IUD and Yuzpe method are available [41]. Policies and legislative changes surrounding lack of approvals for a dedicated progestin-only EC should be considered, as this sends a signal that the medication is not safe or effective.

Stakeholders raised concerns about the inclusion of all the HIV/AIDS-related indicators due to major stigma and discrimination from service providers, as national reporting and treatment protocols preclude organizations from collecting data on HIV and prevention of mother-to-child transmission (PMTCT) [27, 42]. According to the Jordanian legislative system, medical professionals and health facilities are mandated to report an individual’s HIV status to the government [27]; foreign nationals, including refugees, who are found to be HIV-positive, are deported regardless of the consequences to their health and safety, and are not permitted to return to Jordan [42]. This highlights the need to reflect on the ethical implications; while considering the inclusion of these indicators. As, requiring reporting of HIV data without stringent protocols for data protection will only serve to increase stigma, which has been shown to delay enrolment in care for people living with HIV [27].

Given the need for consistency and accountability for monitoring and evaluation of SRMNACH services and outcomes in humanitarian settings, WHO, in collaboration with partner agencies, has supported the development of a ‘Data and Accountability Roadmap for Improving Data, Monitoring and Accountability for SRH in Crises’ [30]. This Roadmap responded to the need for collecting, aggregating, making accessible, using clear and consistent data; aligning with the SDGs, the ‘Every Women Every Child Strategy’, and the commitments made at the 2017 Family Planning Summit to address data gaps.

The findings from our assessment also indicated the need for resources, training, and capacity building to enhance the feasibility of accurate and ethical reporting on the proposed list of SRMNCAH indicators. An array of trainings accompanied with a tool kit guide on data collection for each of the SRMNCAH indicators’ domains is necessary before the rollout of the proposed indicators. The Jordanian context requires a centralized Health Information System that is harmonized and adequately coordinated with global and national reporting requirements to prevent overburdening staff at the data-collection level [29]. The system should be developed and tailored for use in the local context [10] as in the past, many data collection systems that were specifically designed for humanitarian actors were actually utilized in national systems with field workers. There should be a sustainable funded effort at the facility-level to update data collection systems and adapt medical records and logbooks, in order to create standardized data collection fields for the information derived from patient medical records. There must be capacity building at government-run organizations and local clinics to clean, correct, and annotate data.

Strengths and limitations

The strategies recommended by Guba [43] were used for evaluating the trustworthiness of the data in this assessment. Credibility, confirmability, and transferability were used to ensure the trustworthiness of the data. Prolonged engagement, triangulation, member checks, peer-debriefing with the study team in both Jordan and Canada were used to ensure the credibility of the data [44]. The mixed-methods design of this study enabled the integration of data from multiple sources. Qualitative data was used to explore and explain the quantitative findings, with the findings from the facility assessments also validating (or in some cases dispelling) key themes identified in analyses of FGDs and KIIs. The facility assessments provided important findings relating to current management SRMNCAH monitoring and evaluation systems as well as the availability and distribution of specific resources. Despite these strengths, however, our study was limited by challenges with documentation, for example, despite extensive efforts, it was not always possible to locate and access SRMNCAH monitoring and evaluation records. The positionalities of the research team members undoubtedly influenced the participant-researcher interaction as well as our interpretation of data collected. Through memoing and regular debriefings, we attempted to reflect on and understand these dynamics, thereby enhancing the credibility and trustworthiness of the findings.

Conclusion

The findings from our multi-methods feasibility assessment suggest that, overall, there is widespread support for developing a core and concise list of SRMNCAH indicators among humanitarian stakeholders in Jordan. Representatives from a variety of institutions noted numerous already existent resources and systems that could be leveraged, built upon, and improved to ensure the feasibility of collecting this core set of indicators for monitoring and SRMNCAH services and outcomes in humanitarian settings in Jordan. However, an important challenge to this was noted by the research respondents, which was that the requested data collection from donors, the national government, international and UN agencies, coordination/cluster systems is highly variable and burdensome. Stakeholders appreciated that a core set of SRMNCAH indicators would need to be globally relevant and feasible. However, they also clarified that need of this global set to be adequately aligned with national priorities in order to solicit a greater buy-in. The recent implementation of the Maternal Mortality Surveillance and Response System (MMSRS) instilled confidence among our participants that it would be possible to develop national, harmonized systems for reporting and aggregating information, provided that sufficient resources in both personnel and information technology is made available to build capacity and infrastructure.

Stakeholders were also clear that collecting indicators that required a population-level denominator in most instances is not feasible, particularly in urban environments. Health workers repeatedly emphasized their preference for a core set of facility-based indicators, as they perceived them to be more programmatically relevant and actionable. In addition to facility-based indicators, participants discussed the need for certain policy and regulatory changes in Jordan must be made before adopting the proposed indicators for programmatic relevance and actionability.

Finally, the Data and Accountability Roadmap for Improving Data, Monitoring and Accountability for SRH in Crises’ [30] combined with data literacy and indicator management awareness, digital aid frameworks and amendments of laws and regulations should be prioritized at the national and programmatic level to create an enabling environment to reduce duplicity, and improve communication channels and overarching SRMNCAH data quality [45].

Finally, it worth to reiterate that this assessment in Jordan is part of an extensive collaborative global effort that was culminated in June 2021, which lead to the finalization of the proposed M and E framework for SRMNCAH services and outcomes in humanitarian settings. During this global endorsement, and in line with the findings of this assessment, it was emphasized that it is crucial to consider the feasibility (logistical, socio-cultural and political), usefulness, ethical implications, and opportunity costs during the measurement and data collection of these SRMNCAH indicators, especially those indicators that are most sensitive.