Introduction

Communicable diseases are one of the primary concerns in humanitarian emergencies and disasters (1-20). Humanitarian emergencies include disasters associated with natural hazards such as earthquakes, floods and tsunamis, as well as man-made disasters such as famine, conflict and complex emergencies. These emergencies usually require a large-scale international response and affect large proportions of a community, country or region. The importance and overall risk of communicable diseases and communicable disease outbreaks differs between different disaster types. It is particularly low in geo-disasters such as earthquakes or volcanic eruptions (21), higher for flooding (14-20), and much worse again in refugee crises (2, 4-8, 10-12, 22) or complex humanitarian emergencies (1, 23).

While the problem of a potentially increased risk of communicable diseases in humanitarian emergencies is well documented, information on specific risk factors and the levels at which these risk factors become critical is lacking. Yet, the identification of risk factors and their interaction is crucial for risk management. Knowing the overall risk profiles can help identify those sites where proactive interventions may reduce the impact of communicable diseases. Key risk factors for communicable diseases identified in the academic literature can be broadly grouped into categories such as Water, Sanitation and Hygiene (WASH), health and public health system, environment, humanitarian response, infrastructure, insecurity, living conditions, nutrition, mass population displacement and economy (23). Within those broader categories, individual risk factors are defined more specifically, although the categories themselves serve as general risk factors as well (1, 2, 23-33). While similar groups of risk factors have been identified as significant for all emergency types, their weights can differ depending on the individual setting, as does the overall risk of a communicable disease outbreak. For example, as Floret et al. (21) noted, the risk of communicable disease outbreak is almost negligible in geo-disasters that do not trigger a secondary disaster such as a displacement crisis. For each site, it is also important to know which risk factors are of the most pressing concern to allocate resources correctly and prioritise interventions.

In this paper, we summarise the results from three stages of structured online expert consultations we performed to determine the 20 most critical risk factors (across all types of humanitarian emergencies), the thresholds for those factors that could be assessed by a quantitative indicator, and their weights in different types of emergencies. These data were later used in the development of a rapid risk assessment tool to be used by non-experts to assess needs and priorities in humanitarian emergencies. The factors selected to be the 20 most critical were included in the tool and the thresholds and weights for each factor were used as the basis for a risk score for each factor and a combined overall risk score. The risk factors identified, their weights and thresholds, and especially the rapid risk assessment tool do not substitute detailed needs assessment and are designed to rapidly assess communicable disease outbreak risk and, as such, are not a suitable basis for humanitarian programming.

Methods

We conducted a three-stage structured expert elicitation.

Recruitment and participants

Participants who self-identified as having experience in health protection and/or humanitarian assistance were invited to take part. Participants were recruited by email through dedicated listservs that cover areas such as health protection, public health intelligence, humanitarian assistance and disaster studies as well as through the personal and professional contacts of the research team. Participants were then guided to an online questionnaire.

Recruitment included personalised emails to 16 individuals we knew professionally and via dedicated relevant listservs. Recipients were encouraged to share with interested colleagues. Most of the targeted individual recipients had recent field experience supporting response to humanitarian disasters. Table 1 lists the affiliations of targeted individuals and the specific list serves; most affiliations were with public health agencies, charitable aid organisations and/or research institutions. Many targeted respondents had multiple relevant affiliations. To help assure confidentiality we did not ask during the survey for identifying information such as current employer, job title or years of experience. The specific Email listservs we used and characteristics of the individuals we personally asked to fill in the survey are listed in Table 1.

Table 1

Email list servers (n=11), with affiliations and characteristics of targeted individuals (n=16)

Public Health Agencies:
Philippine Ministry of Health, Public Health England, World Health Organisation, Unicef, UNESCO, UNRWA
NGO's involved in humanitarian response:
Global Student Embassy, Médecins Sans Frontières, Mercy Corps Indonesia
Universities or Research Institutions:
Adnan Menderes Üniversitesi, Institute of Tropical Medicine in Antwerp, Northumbria University, Tufts University, University of East Anglia, Würzburg University
Job titles of targeted individuals:
Associate Professor, Consultant for WHO, Consultant in Global Disaster Risk Reduction, Director of Health programme, Director of Operations Research, Geostatistical Modeller, Operations Researcher, Professor, Research Fellow, Researcher, Senior Fellow, WASH cluster coordinator, Water Coordinator, Water Hygiene and Sanitation Officer
Email List servers:
German Disaster Research Listserv, JISCMAIL Health Geography Listserv, Healthcare Information for All listserv, JISCMAIL Public Health Listserv, JISCMAIL Medical Sociology Listserv, JISCMAIL Disaster Research Listserv, JISCMAIL Disaster Research Listserv, JISCMAIL Global Health Listserv, JISCMAIL Public Health Listserv, Humanitarian Listserv, Society of Apothecaries, Healthcare Information for All listserv

Questionnaires are included in the supplemental files. Participants could fill out one or more of the three stages of online questionnaires. Participation in a previous questionnaire was not required to take part in the second and/or third stages. The first questionnaire asked participants to identify the 20 most critical risk factors from a list compiled based on the wider literature and a recent literature review by the research team (23). The first questionnaire also asked participants to assign weights (on a scale from 0-5) to each risk factor to allow the calculation of a weighted average for each factor. The weighted average was calculated from the mean score of level of importance (on a scale from 0-5) times the number of participants selecting this weight for this factor. Weighted averages were calculated in case the initial mechanism for selection of the 20 most critical factors based on how many participants considered them to be in the top-20 proved to be inconclusive. In the second questionnaire, participants were invited to assign yellow (urgent, action required) and red (critical, action required immediately) thresholds for all quantifiable risk factors.

The third and final questionnaire sought to identify the respective weights (on a scale from 1-5) of the 20 most critical risk factors in nine different types of emergencies, as broadly described by Spens and Kovács (34). The types of crises were: famine (F), complex emergency (CHE), conflict (C), refugee and IDP camp (RC), flooding (FL), geo-disaster (GD), protracted crisis (PC), tropical storm (TC) and tsunami (T). Complex emergencies describe situations in which widespread internal or external conflict has led to a complete breakdown of authority and widespread damage to society. They are defined by requiring a multi-facetted, multi-agency international response (23, 35). Conflicts include inter- and intra-state warfare, civil war and insurgency. Geo-disasters include earthquake, landslides, volcanic eruptions and other disasters caused by geological hazards. Flooding refers to fresh water flooding. Tropic storms include Hurricanes, Typhoons, Cyclones and similar hydro-meteorological hazards. This list of types of emergencies was not meant to be complete or to comprise mutually exclusive types of crises. Displacement crises are usually an additional humanitarian emergency secondary to conflicts, complex emergencies, or disasters associated with a natural hazard. However, we believe the risks for communicable disease outbreaks differ significantly enough for these to form distinct categories.

Analysis

Answers were collected online and analysed in Microsoft Excel. Weighted averages, median and mean scores were calculated where appropriate. Additionally, correlations were done in SPSS version 23 using Pearson correlation.

Results

Responses

The first questionnaire was completed by 21 participants; the second questionnaire was completed by 24 and the last questionnaire by 25 persons. We only stored, recorded and analysed completed questionnaires and not those left half-completed in order to comply with the possibility for participants to withdraw consent to partake until the end of the survey. Given that the surveys were advertised widely, this represents a relatively small proportion of possible respondents. However, it is not possible to characterise the actual response rate.

Risk Factors

The first questionnaire sought to identify the 20 most critical risk factors, irrespective of the emergency type and their relative importance. The 20 risk factors chosen by the most respondents (see column ‘Selected (n)’ in Table 2) were input to the Stage 2 and 3 surveys. 19/20 of these also had the overall highest weighted average scores (see Table 3).

Table 2

List of the selected 20 most critical risk factors irrespective of emergency type and setting. Participants (n=21) were asked to select 20 factors out of the given 59 options.

Risk factor Selected, n (%) Included in stage 2-3 surveys
No access to clean water 19 (90.48) Yes
Lack of functioning toilets 19 (90.48) Yes
Exposure to disease vectors 17 (80.95) Yes
Lack of waste management 17 (80.95) Yes
Lack of health facilities 16 (76.19) Yes
Lack of health professionals (doctors, nurses, community health workers) 16 (76.19) Yes
Insufficient vaccination coverage 15 (71.43) Yes
Poor health status of the population 15 (71.43) Yes
Extreme poverty 15(71.43) Yes
Overcrowding 14 (66.67) Yes
Lack of medicines 12 (57.14) Yes
Insufficient nutrient intake 11 (52.38) Yes
Lack of health education 11 (52.38) Yes
Inadequate distance between housing etc. and human waste disposal 11 (52.38) Yes
Ongoing conflict 11 (52.38) Yes
Population displacement 11 (52.38) Yes
Lack of organisational and political will to address public health problems 11 (52.38) Yes
Flooding (waste water) 10 (47.62) Yes
Breakdown of government services 10 (47.62) Yes
Reluctance to follow recommended procedures to limit disease spread 10(47.62) Yes
Lack of disease surveillance 9 (42.86) No
Inadequate shelter 9 (42.86) No
No soap 8 (38.10) No
Local endemicity of key communicable diseases 8 (38.10) No
Lack of trust in health care provided 7 (33.33) No
Flooding (fresh water) 7 (33.33) No
Environmental vulnerability 7 (33.33) No
Local endemicity of disease vectors 7 (33.33) No
Inequalities 7 (33.33) No
Political instability 7 (33.33) No
Lack of electricity 6 (28.57) No
Illiteracy (among target recipients of aid) 6 (28.57) No
Unsafe burial rites 5 (23.81) No
Breakdown of authority 5 (23.81) No
Displacement into camp(s) 5 (23.81) No
Low levels of education (among target population) 5 (23.81) No
Indoor fires/air pollution 4 (19.05) No
Sexual and Gender-based Violence 4 (19.05) No
Increased contact with domestic animals 3 (14.29) No
Flooding (sea water) 3 (14.29) No
Very high temperatures 3 (14.29) No
Lack of belief in germ model — preference for other explanations of diseases 3 (14.29) No
Ethnic rivalry 2 (9.52) No
Seismic risk (dry mass displacement) 2 (9.52) No
Landslide risk (wet mass displacement) 2 (9.52) No
High precipitation 2 (9.52) No
Very low temperatures 2 (9.52) No
Violence 2 (9.52) No
Increased contact with wildlife 1 (4.76) No
Temporary housing (not tents) 1 (4.76) No
Drought 1 (4.76) No
Dust storms 1 (4.76) No
De-forestation 1 (4.76) No
Economic stagnation 1 (4.76) No
Competition for resources 1 (4.76) No
Arms proliferation 1 (4.76) No
Lack of fuel for cooking or heating 1 (4.76) No
Housing in tents 0 (0) No
Volcanic risk 0 (0) No

Table 3

Weighted averages of the importance of the risk factors in humanitarian emergencies and disasters, irrespective of emergency type and setting. 0= Not selected/not important; 1= A little important; 2= Important; 3= Quite important; 4= Very important; 5= Extremely important. Green indicates those factors included in stages 2 and 3 while the factors marked in red were discarded after stage 1.

Risk factor 0 1 2 3 4 5 Weighted Average Included
No access to clean water 2 0 0 0 3 15 4.35 Yes
Lack of functioning toilets 2 0 2 1 8 7 3.7 Yes
Lack of health facilities 5 0 1 0 7 7 3.25 Yes
Lack of health professionals (doctors, nurses, community health workers) 5 0 1 2 3 9 3.25 Yes
Extreme poverty 5 0 1 3 4 7 3.1 Yes
Insufficient vaccination coverage 6 0 1 3 3 7 2.9 Yes
Exposure to disease vectors 4 0 4 3 4 5 2.9 Yes
Lack of waste management 4 0 1 6 7 2 2.9 Yes
Poor health status of the population0 6 0 0 4 8 2 2.7 Yes
Lack of medicines 9 0 0 2 4 5 2.35 Yes
Overcrowding 7 0 2 4 7 0 2.2 Yes
Ongoing conflict 10 0 0 3 3 4 2.05 Yes
Lack of organisational or political will to address public health problems 9 0 2 3 2 4 2.05 Yes
Insufficient nutrient intake 9 0 2 2 5 2 2 Yes
Inadequate distance between housing, etc. and human waste disposal 9 0 1 3 7 0 1.95 Yes
Flooding (waste-water) 11 0 0 1 5 3 1.9 Yes
Lack of health education 9 0 1 6 3 1 1.85 Yes
Population displacement 10 0 2 0 7 1 1.85 Yes
Breakdown of government services 10 1 2 2 2 3 1.7 Yes
Inadequate shelter 11 0 2 3 1 3 1.6 No
Inequalities 13 0 0 0 5 2 1.5 No
No soap 13 0 1 0 3 3 1.45 No
Lack of disease surveillance 12 0 1 3 3 1 1.4 No
Reluctance to follow recommended procedures to limit disease spread 11 0 2 4 3 0 1.4 Yes
Political instability 13 0 1 1 3 2 1.35 No
Local endemicity of key communicable diseases 13 0 1 2 3 1 1.25 No
Flooding (fresh water) 13 0 1 2 4 0 1.2 No
Local endemicity of disease vectors 14 0 1 1 1 3 1.2 No
Environmental vulnerability 13 0 2 2 2 1 1.15 No
Lack of electricity 14 0 1 2 2 1 1.05 No
Breakdown of authority 15 0 0 1 2 2 1.05 No
Lack of trust in health care provided 14 0 0 4 2 0 1 No
Illiteracy (among target recipients of aid) 14 0 1 3 1 1 1 No
Displacement into camp 5 0 1 1 2 1 0.9 No
Low levels of education (among target persons) 15 0 0 3 1 1 0.9 No
Sexual and Gender-based Violence 16 0 0 1 2 1 0.8 No
Indoor fires/indoor air pollution 16 0 0 2 1 1 0.75 No
Increased contact with domestic animals 17 0 0 1 2 0 0.55 No
Unsafe burial rites 16 0 2 1 1 0 0.55 No
Ethnic rivalry 18 0 0 0 1 1 0.45 No
Flooding (salt-water) 17 0 1 1 1 0 0.45 No
Very high temperatures 17 0 0 3 0 0 0.45 No
Lack of belief in germ model — preference for other explanations for disease causes 17 0 1 1 1 0 0.45 No
Violence 18 0 0 1 0 1 0.4 No
Seismic risk (dry mass displacement) 18 0 1 0 0 1 0.35 No
Very low temperatures 18 0 0 1 1 0 0.35 No
Increased contact with wildlife 19 0 0 0 0 1 0.25 No
Landslide risk (wet mass displacement) 18 0 1 1 0 0 0.25 No
High precipitation 18 0 1 1 0 0 0.25 No
Drought 19 0 0 0 0 1 0.25 No
Economic stagnation 19 0 0 0 0 1 0.25 No
Arms proliferation 19 0 0 0 0 1 0.25 No
Dust storms 19 0 0 0 1 0 0.2 No
De-forestation 19 0 0 0 1 0 0.2 No
Lack of fuel for cooking or heating 19 0 0 0 1 0 0.2 No
Temporary housing (not tents) 19 0 1 0 0 0 0.1 No
Competition for resources 19 0 1 0 0 0 0.1 No
Housing in tents 20 0 0 0 0 0 0 No
Volcanic risk 20 0 0 0 0 0 0 No

Thresholds

Table 4 shows the expert-identified yellow and red thresholds for the nine quantifiable risk factors. A yellow threshold indicated a situation of concern that should be addressed as soon as possible while a red threshold indicated a highly critical situation that needs to be a top priority. These thresholds are described individually below.

Access to clean water was measured in litre per person per day. The median red threshold was 2 (mean 5.25, SD 5.01) litres and the median yellow threshold 6.5 (mean 10.5, SD 8.92) litres.

The available number of hospital beds per 10,000 persons was used as a proxy indicator for the risk factor health care facilities. The median red threshold was 5 beds (mean 18.77, SD 27.28) per 10,000 persons and the median yellow threshold was 20 beds (mean 45, SD 54.70) per 10,000 persons.

The median red threshold for functioning toilets was 4 (mean 4.92, SD 4.95) toilets per 100 persons and the median yellow threshold was 9 (mean 10.86, SSD 11.74) toilets per 100 persons.

The number of health professionals per 10000 was measured in three categories. The median red threshold for doctors per 10000 persons was 1.5 (mean 19.21, SD 35.24) and the median yellow threshold was 5 (mean 27.31, SD 55.91) doctors per 10000 persons. The median red threshold for nurses was 6 (mean 96.79, SD 256.24) per 10000 persons and the median yellow threshold 10 (mean 63, SD 111.29) nurses per 10000 persons. The median red threshold for community health care workers was 8.5 (mean 15.86, SD 26.18) per 10000 persons and the median yellow threshold was 20 (mean 42.46, SD 55.51) community health care workers per 10000 persons.

Vaccination coverage was measured for the following four diseases: measles, meningococcal meningitis, polio and hepatitis B. The median red threshold for measles vaccination coverage was 75 % (mean 67.21, SD 23.46) and the median yellow threshold was 90 % (mean 81.92, SD 14.88). The median red threshold for meningococcal meningitis vaccination coverage was 72.5 % (mean 62.21, SD 23.92) with a median yellow threshold at 80 % (mean 73.08, SD 21.53). The median red threshold for polio vaccination coverage was 75 (mean 64.31, SD 25.89) percent with a median yellow threshold of 87.5 % (mean 83.33, SD 12.80). The median red threshold for Hepatitis B vaccination coverage was 50 % (mean 52.00, SD 23.90) with a median yellow threshold of 72.5 % (mean 70.83, SD 17.42).

Poverty was measured in percentage of the population living below 1 $ US per person per day. The median red threshold was 20 % (mean 29.07, SD 25.70) and the median yellow threshold was also 20 % (mean 28.27, SD 22.88)

Overcrowding was measured in the number of persons living per 100 square metres (m2). The median red threshold was 10 (mean 20.58, SD 22.28) persons per 100 m2 and the median yellow threshold was 5 (mean 13.09, SD 14.53) persons per 100 m2.

Nutrition was measured in kcal per adult per day. The median red threshold was 1000 (mean 1009.30, SD 742.52) and the median yellow threshold was 1750 (mean 1716.67, SD 692.62) kcal per adult per day. These figures – especially the seemingly ‘high’ figure for the yellow threshold must be understood in the context of the impact of mal- and undernutrition for the severity of communicable disease outbreaks through mechanisms such as increased susceptibility and greater shedding and transmission. Poor nutritional status is a common attribute of affected populations in many humanitarian emergencies and is known to exacerbate the size and severity of communicable disease outbreaks. (1, 24, 36-38).

The median red threshold for the distance between human waste disposal and housing was 20 metres (mean 71.00, SD 138.53) and the median yellow threshold was 50 metres (mean 79, SD 89.60).

Table 4

Summary of yellow and red thresholds for 9 quantifiable risk factors.

Risk Factor Threshold Min Max Median Mean SD n
Clean water in litre per person per day Yellow 0 30 6.5 10.5 8.92 16
  Red 0 15 2 5.25 5.01 20
Hospital beds per 10 000 persons Yellow 5 200 20 45 54.7 13
  Red 1 100 5 18.77 27.28 13
Functioning toilets per 100 persons Yellow 1 50 9 10.86 11.74 14
  Red 1 20 4 4.92 4.95 13
Doctors per 10 000 persons Yellow 1 200 5 27.31 55.97 13
  Red 0 100 1.5 19.21 35.24 14
Nurses per 10 000 persons Yellow 1 400 10 63 111.29 13
  Red 0 1000 6 96.79 256.24 14
CHW per 10 000 persons Yellow 1 200 20 42.46 55.51 13
  Red 0 100 8.5 15.86 26.18 14
Measles vaccination percentage Yellow 40 95 90 81.92 14.88 13
  Red 1 90 75 67.21 23.46 14
Meningitis vaccination percentage Yellow 10 90 80 73.08 21.53 13
  Red 1 85 72.5 62.21 23.92 14
Polio vaccination percentage Yellow 45 95 87.5 83.33 12.8 12
  Red 1 90 75 64.31 25.89 13
Hepatitis B vaccination percentage Yellow 20 90 72.5 70.83 17.42 12
  Red 1 90 50 52 23.9 13
Persons living under 1 $ US percentage Yellow 1 60 20 28.27 22.88 11
  Red 1 80 20 29.07 25.7 14
Persons per 100 square meters Yellow 1 50 5 13.09 14.53 11
  Red 1 75 10 20.58 22.28 12
Kcal per adult per day Yellow 800 3500 1750 1716.67 692.62 12
  Red 1 2500 1000 1009.3 742.52 13
Distance housing and human waste disposal (meters) Yellow 10 300 50 79 89.6 10
  Red 1 500 20 71 138.53 11

Weights in different emergency types

Weights for the different risk factors were similar for different types of emergencies, with only minor differences (see Figure 1 and Tables 5 and 6). On a scale from 1 (not important) to 5 (very important) all included risk factors score above 4 (both mean and median) when combining all emergencies. The only two risk factors with a median of 3 were ‘insufficient nutrient intake’ and ‘lack of health education’ in the context of a tropical storm. Mean values for all risk factors in all different emergency types (not combined) remained above 3.4, except for ‘lack of health education’ in the context of flooding (mean 3.29, SD 1.14, median 4) and ‘lack of health education’ in the context of a tropical storm (mean 3.22, SD 1.28, median 3). This suggests a reinforcement of the importance of these risk factors across different humanitarian emergency types.

There was considerable correlation between risk factors, demonstrating the highly interactive nature of risk and risk factors in humanitarian emergencies as well as the complexity of such situations (see Table 7).

Table 5

Median values for the weights of the selected risk factors in different types of emergencies

Risk Factor F CHE C F GD PC RC TS T
No access to clean water 5 5 5 5 5 5 5 5 5
Lack of functioning toilets 4 5 5 5 5 5 5 5 5
Exposure to disease vectors 4.5 5 4 5 5 5 5 5 5
Lack of waste management 4 4 4 4.5 4 5 5 4 4
Lack of health facilities 4.5 5 5 4.5 5 5 5 5 5
Lack of health workers 4 5 5 4 5 5 5 4 4
Insufficient vaccine coverage 4.5 4 4 4 4 4.5 5 4 4
Poor health status 5 5 4.5 4 4 5 4.5 4 4
Extreme poverty 4.5 4 4 4 4 5 4.5 4.5 4
Overcrowding 4 4 4 4 4 4 5 4 4
Lack of medicines 4 5 5 4 5 5 5 4 4
Insufficient nutrient intake 5 4 4 4 4 5 5 3 4
Lack of health education 4 4 4 4 4 4 4 3 4
Inadequate distance between housing and human waste disposal 4 4 4 4 4 5 5 4 4
Ongoing conflict 5 5 5 4 4 5 5 4 4
Population displacement 4.5 4 5 4.5 4 5 5 4.5 4
Lack of organisational and/or political will to address public health problems 5 5 5 5 4 5 5 4 5
Flooding (waste water) 4 4 4 5 4 4.5 4 5 5
Breakdown of government services 5 4 5 4.5 4 5 4.5 4 4
Reluctance to follow disease control procedures 4 4 4 4 4 4 4.5 4 4
Figure 1 

Distribution of mean weights in different emergency types

Table 6

Mean values for the weights for the risk factors in different emergency types (standard deviations in brackets).

  F CHE C FL GD PC RC TS T
No access to clean water 4.79 (0.41) 4.84 (0.46) 4.68 (0.55) 4.71 (0.54) 4.80 (0.40) 4.75 (0.43) 4.83 (0.47) 4.83 (0.37) 4.74 (0.44)
Lack of functioning toilets 3.96 (1.27) 4.52 (0.90) 4.20 (1.10) 4.38 (1.03) 4.36 (1.02) 4.58 (0.76) 4.67 (0.80) 4.38 (1.03) 4.39 (1.05)
Exposure to disease vectors 4.25 (0.83) 4.60 (0.57) 4.08 (0.89) 4.38 (0.90) 4.32 (0.84) 4.42 (0.76) 4.71 (0.54) 4.38 (1.07) 4.17 (1.20)
Lack of waste management 3.75 (1.33) 4.32 (0.79) 4.12 (1.07) 4.17 (1.07) 4.20 (0.80) 4.46 (0.82) 4.46 (0.87) 4.04 (1.10) 4.09 (0.93)
Lack of health facilities 4.08 (1.11) 4.48 (0.70) 4.50 (0.76) 4.21 (1.00) 4.42 (0.81) 4.54 (0.76) 4.54 (0.71) 4.21 (1.04) 4.30 (0.95)
Lack of health workers 4.13 (1.05) 4.52 (0.70) 4.42 (0.70) 4.13 (1.01) 4.40 (0.75) 4.54 (0.71) 4.38 (0.86) 4.17 (0.99) 4.26 (0.94)
Insufficient vaccine coverage 4.04 (1.24) 4.36 (0.69) 4.08 (0.95) 3.63 (1.15) 3.64 (1.05) 4.25 (0.92) 4.42 (0.86) 3.63 (1.18) 3.74 (1.03)
Poor health status 4.63 (0.56) 4.32 (0.93) 4.25 (0.88) 3.83 (1.25) 4.00 (0.98) 4.46 (0.76) 4.25 (0.92) 3.88 (1.05) 3.86 (1.22)
Extreme poverty 4.38 (0.70) 4.24 (0.81) 4.00 (1.04) 3.71 (1.21) 4.20 (0.89) 4.33 (0.90) 4.04 (1.21) 4.08 (1.15) 3.78 (1.21)
Overcrowding 3.96 (1.21) 4.20 (0.69) 3.79 (1.04) 3.92 (0.95) 3.96 (1.00) 4.25 (0.83) 4.38 (0.81) 4.00 (1.08) 3.91 (0.93)
Lack of medicines 3.88 (1.17) 4.24 (0.86) 4.38 (0.81) 4.00 (1.22) 4.25 (0.88) 4.50 (0.71) 4.42 (0.81) 4.17 (0.94) 3.96 (1.16)
Insufficient nutrient intake 4.71 (0.61) 4.08 (0.93) 4.13 (0.97) 3.78 (1.06) 3.72 (1.08) 4.42 (0.81) 4.38 (0.81) 3.46 (1.08) 3.70 (1.08)
Lack of health education 3.54 (1.22) 3.96 (0.82) 3.70 (1.20) 3.29 (1.14) 3.68 (1.05) 4.04 (0.84) 3.75 (1.09) 3.22 (1.28) 3.48 (1.02)
Inadequate distance between housing and human waste disposal 3.71 (1.24) 4.08 (0.93) 3.79 (1.26) 3.96 (1.14) 3.80 (0.94) 4.13 (1.09) 4.50 (0.87) 3.91 (1.10) 3.83 (1.01)
Ongoing conflict 4.04 (1.31) 4.32 (0.88) 4.67 (0.75) 3.63 (1.41) 3.72 (1.15) 4.29 (1.06) 4.33 (0.94) 3.58 (1.41) 3.65 (1.34)
Population displacement 4.13 (1.13) 4.00 (0.94) 4.46 (0.64) 4.21 (0.96) 4.12 (0.82) 4.33 (0.85) 4.29 (1.10) 4.21 (0.96) 4.04 (1.12)
Lack of organisational and/or political will to address public health problems 4.38 (0.99) 4.44 (0.70) 4.42 (0.76) 4.25 (1.01) 4.08 (0.84) 4.46 (0.71) 4.33 (0.99) 4.21 (0.91) 4.13 (1.15)
Flooding (waste water) 3.63 (1.41) 4.24 (0.76) 3.75 (1.20) 4.54 (0.82) 4.00 (1.06) 4.04 (1.10) 4.21 (0.91) 4.57 (0.71) 4.35 (0.91)
Breakdown of government services 4.29 (1.02) 4.24 (0.65) 4.54 (0.71) 4.25 (0.92) 4.20 (0.75) 4.46 (0.71) 4.13 (1.09) 4.25 (0.83) 4.09 (0.97)
Reluctance to follow disease control procedures 3.75 (1.23) 4.28 (0.78) 4.04 (0.93) 4.13 (0.93) 4.12 (0.86) 4.00 (1.04) 4.29 (0.84) 4.00 (1.04) 4.00 (0.98)

Table 7

Correlation between risk factors (all emergency types combined)

W = Water; T = Toilets; V = Vectors; WM = Waste Management; HF = Health Facilities; HC = Health Care Workers; VA = Vaccinations; HS = Health Status; P = Poverty; O = Overcrowding; M = Medicines; N = Nutrition; HE = Health Education; D = Distance between housing and human waste disposal; C = Conflict; DI = Displacement; W = Will to address problems; F = Flooding (waste water); B = Breakdown of government services; R = Reluctance to follow procedures; PC = Pearson Correlation
    W T V WM HF HC VA HS P O M N HE D C DI W F B R
W PC 1 .522** .350** .314** .378** .306** .301** .262** .280** .310** .354** .337** .204* .336** .309** .329** .368** .363** .243** .405**
T PC .522** 1 .463** .692** .486** .361** .357** .297** .293** .554** .482** .297** .388** .586** .400** .406** .427** .622** .367** .519**
V PC .350** .463** 1 .507** .547** .507** .584** .420** .441** .467** .509** .423** .337** .415** .469** .474** .432** .374** .401** .592**
WM PC .314** .692** .507** 1 .566** .447** .384** .313** .311** .539** .632** .260** .523** .623** .359** .445** .485** .495** .467** .490**
HF PC .378** .486** .547** .566** 1 .874** .540** .453** .397** .492** .796** .432** .485** .531** .545** .500** .562** .394** .522** .549**
HC PC .306** .361** .507** .447** .874** 1 .539** .508** .513** .484** .737** .452** .482** .467** .560** .531** .612** .289** .605** .525**
VA PC .301** .357** .584** .384** .540** .539** 1 .611** .570** .422** .547** .525** .555** .376** .565** .519** .503** .246** .423** .628**
HS PC .262** .297** .420** .313** .453** .508** .611** 1 .796** .544** .504** .744** .530** .301** .559** .476** .453** .193** .418** .441**
P PC .280** .293** .441** .311** .397** .513** .570** .796** 1 .644** .449** .633** .479** .312** .593** .592** .539** .244** .553** .478**
O PC .310** .554** .467** .539** .492** .484** .422** .544** .644** 1 .511** .503** .517** .426** .503** .524** .549** .368** .485** .475**
M PC .354** .482** .509** .632** .796** .737** .547** .504** .449** .511** 1 .485** .619** .584** .551** .542** .642** .450** .583** .589**
N PC .337** .297** .423** .260** .432** .452** .525** .744** .633** .503** .485** 1 .473** .399** .526** .388** .411** .192* .335** .408**
HE PC .204* .388** .337** .523** .485** .482** .555** .530** .479** .517** .619** .473** 1 .428** .484** .406** .463** .290** .389** .503**
D PC .336** .586** .415** .623** .531** .467** .376** .301** .312** .426** .584** .399** .428** 1 .438** .352** .370** .629** .365** .620**
C PC .309** .400** .469** .359** .545** .560** .565** .559** .593** .503** .551** .526** .484** .438** 1 .610** .572** .271** .509** .528**
DI PC .329** .406** .474** .445** .500** .531** .519** .476** .592** .524** .542** .388** .406** .352** .610** 1 .642** .417** .598** .531**
W PC .368** .427** .432** .485** .562** .612** .503** .453** .539** .549** .642** .411** .463** .370** .572** .642** 1 .368** .828** .558**
F PC .363** .622** .374** .495** .394** .289** .246** .193** .244** .368** .450** .192* .290** .629** .271** .417** .368** 1 .340** .547**
B PC .243** .367** .401** .467** .522** .605** .423** .418** .553** .485** .583** .335** .389** .365** .509** .598** .828** .340** 1 .464**
R PC .405** .519** .592** .490** .549** .525** .628** .441** .478** .475** .589** .408** .503** .620** .528** .531** .558** .547** .464** 1
* Correlation is significant at the 0.01 level (2-tailed), ** Correlation is significant at the 0.001 level (2-tailed)

Discussion

The results from the first questionnaire, regarding the selection of risk factors, confirm that, as suggested in the wider literature, WASH (39-42), health care (36, 43), nutrition (1, 36, 37) and emergency specific risk factors such as poverty (44-46), displacement and overcrowding (1, 24, 28, 47), and (ongoing) armed conflict or war (48) are among the primary factors influencing communicable disease outbreaks in humanitarian emergencies and disasters. These results are further confirmed by the outcomes of the third questionnaire which indicates the high importance of the selected risk factors across all types of humanitarian emergencies. While some of the risk factors identified in this research were – deliberately – broad, additional discussion with humanitarian aid providers (which were not strictly speaking part of this research) revealed some of the most common interpretations of these risk factors and showed that, while encompassing a range of issues, they were interpreted similarly by all people we spoke to. For example, ‘breakdown of government services’ was generally interpreted as encompassing wider infrastructure issues such as transportation and roads, telecommunications, safety and security, and sometimes education. Many of these have complex interaction pathways (23).

For some of the risk factors, responses included seemingly extreme values. Due to this we suggest, for any use of the data, to rely on median values rather than means to make sure that extremes have little effect. However, we are not confident enough that they are simply mistakes to omit them from the analysis. Extremes of 1 or 0 could also mean that the responder didn’t think this was a relevant factor. We cannot know why such a value was selected. If such values had been mentioned in interviews, it would have been highly interesting to know if this was a mistake or an intentional way to signify that a risk factor or threshold would – in the responder’s opinion – not have a significant effect on communicable disease outbreak risk.

While we focused on the 20 most critical risk factors, this does not mean that other factors are not important when assessing the risk of communicable disease outbreaks in such situations. However, our aim was to establish which factors need to be priority concerns. We were interested in identifying quantitative thresholds for the risk factors that could support quick assessment using minimal resources and man-power by not requiring professional judgements. The argument could be raised that thresholds for many of these factors can be as easily obtained from the Sphere standards (49). However, the thresholds listed in the Sphere standards have important limitations if used for the purpose of assessing the risk of communicable disease outbreaks in humanitarian emergencies. The Sphere standards were developed to assess the adequacy of overall humanitarian response and provide general minimum standards. Thus, the Sphere standards are neither intended as risk assessment nor are they specific to communicable diseases. Secondly, the Sphere standards have a normative component, as they indicate standards that should be reach based on ethical considerations rather than those that empirically relate to changes in the level of risk experience. While this makes the Sphere standards an unsuitable comparison, it might be interesting to see how this difference in approach shapes the suggested thresholds. Sphere standards indicate a minimum of 15 litres of water per person per day (49). Our survey found a yellow threshold for clean water availability at 6.5 litres per person per day. This difference is explained by the fact that the thresholds we sought to identify are only thresholds for increases in disease outbreak risk. A yellow threshold for clean water at 6.5 litres per person per day does not suggest that a person does not need more that 6.5 litres of water per day but rather that below that the risk for a communicable disease outbreak critically increases. Additionally, some of the risk factors and especially their measurements are simply proxies. This becomes clear when looking at vaccination coverage. The selected vaccines are not meant to be the main, the only, or even vaccination priorities at all in all emergencies but rather they are used as proxies to estimate the reach of vaccination programmes.

Keeping this in mind, the measures and risk factors identified are entirely unsuitable to base humanitarian programming upon. This should follow a suitable method for needs assessment – which obviously communicable disease outbreak risk assessment, which the factors suggested here are meant for, is not – and an estimation of minimum standards based on internationally accepted levels such as the Sphere standards.

In contrast, the thresholds identified by our surveys indicate precise and transferable tipping points for levels of risk. They are the first step towards developing a rapid risk assessment mechanism for communicable disease outbreaks in humanitarian emergencies that, rather than asking the person or persons completing it for qualitative and personal assessments of the severity without any indicators what this should be based on, uses pre-defined thresholds and risk levels against which a situation can be judged. Hence our thresholds are hopefully useful in real world risk assessment, because they identify specific risk thresholds using simple quantitative indicators.

Limitations

While we made every attempt to maximise participation, the main limitation of this work is the small number of respondents. However, it can be argued that the field of experts suitable for participation is not large. Our expert opinions are in line with assessments in scientific literature of the relative importance of different risk factors. Expert elicitations have their limits and are subject to biases (50, 51). Overconfidence in the results of expert elicitations should be avoided (51). Hence, we do not recommend accepting the results without further inquiry, even if they are mostly in line with the literature.

Additionally, the above-mentioned lack of specification and possibly blurred and broad definitions of some of the risk factors is a potential limitation. That would certainly be the case if the results from this research would be used uncritically to make decisions in the field, even if they were used just for risk assessment without further additional investigation. However, considering that we do not recommend using these results beyond the realm of risk assessment and that for risk assessment we considered this research to be a first stage within a larger research project, the results form a good starting point to understand expert opinion on some of the most critical risk factors for communicable disease outbreaks in humanitarian emergencies.

Conclusion

Communicable disease outbreaks remain a significant concern in the aftermath of emergencies and disasters, especially in low- and middle-income countries. Broadly, expert consensus seems to be that WASH, access to healthcare, nutrition and wider societal and emergency specific factors are among the most important indicators and risk factors for communicable disease outbreaks in such situations. These factors remain important across different types of humanitarian emergencies. Beyond establishing current expert opinion, this research also serves as a starting point to assess and improve risk assessment tools, methods and protocols for communicable disease risks in humanitarian emergencies and disasters. Current risk assessment tools, such as the WHO tool used in the context of the EWARN system (52, 53), also use individual risk factors. However, there is a strong need to make risk assessments clearer and more explicit by using, where possible, previously determined risk factor thresholds that can be assessed without expert knowledge in each domain. Ideally, this risk summary would be based on an independent needs assessment and require minimal additional primary data collection in the field. The expert consultation described in this article, combined with a systematic review performed in parallel (23) and additional research by the research team, seeks to be the basis for such a pragmatic, easy-to-use and novel risk assessment tool. No system captures the complexity and diversity of humanitarian emergency settings perfectly and even accepted international standard such as Sphere are under constant revision and do not cover all aspects of humanitarian response. However, such a risk assessment tool can be seen as an attempt to capture some of the main risk factors for communicable disease outbreaks in such settings, especially as it does not assume considerable expert knowledge from the person or persons using it, like the WHO’s risk assessment tool for communicable diseases in humanitarian emergencies does (52, 53).

Supplemental Information

Supplemental information as referenced in the text (PDF)

Ethical Approval

The research study has been approved under the regulations of the University of East Anglia’s Faculty of Health and Medicine Ethics Committee.

Competing Interest statement

The authors declare no competing interests.

Authors' Contributions

All researchers helped design the study and provided input for the surveys. CH designed the tool with support from JB. Analysis was done by CH under supervision from PH and with input from JB. All authors approved the final analysis and manuscript.

Funding

The research was in part funded by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response at King's College London in partnership with Public Health England (PHE), in collaboration with the University of East Anglia, Norwich Medical School. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health or Public Health England.

Acknowledgements

We thank all respondents for their input and their time. Additionally, we would like to thank all colleagues who have made comments on the initial list of risk factors that respondents could choose from in survey 1.

References

1. Connolly M, Gayer M, Ryan M, Salama P, Spiegel P, Heymann D. Communicable diseases in complex emergencies: impact and challenges. The Lancet. 2004;364(9449):974-1983. DOI: https://doi.org/10.1016/S0140-6736(04)17481-3

2. Goma Epidemiology Group. Public health impact of Rwandan refugee crisis: what happened in Goma, Zaire, in July, 1994? Lancet. 1995;345:339-44. DOI: https://doi.org/10.1016/S0140-6736(95)90338-0

3. Ahmad K. Measles epidemic sweeps through Afghanistan. Lancet. 2000;355:1439. DOI: https://doi.org/10.1016/S0140-6736(05)74645-6

4. Diaz T, R A. Infectious diseases in a Nicaraguan refugee camp in Costa Rica. Trop Doct. 1989;19:14-7. DOI: https://doi.org/10.1177/004947558901900105

5. Hatch D, Waldman R, Lungu G, Piri C. Epidemic cholera during refugee resettlement in Malawi. Int J Epidemiol. 1994;23:1292–9. DOI: https://doi.org/10.1093/ije/23.6.1292

6. Mercer A. Mortality and morbidity in refugee camps in eastern Sudan: 1985–90. Disasters. 1992;16:28-42.DOI: https://doi.org/10.1111/j.1467-7717.1992.tb00373.x

7. Rowland M, Munir A, Durrani N, Noyes H, Reyburn H. An outbreak of cutaneous leishmaniasis in an Afghan refugee settlement in north/west Pakistan. Trans R Soc Trop Med Hyg 1999;93:133-6.DOI: https://doi.org/10.1016/S0035-9203(99)90285-7

8. Rowland M, Nosten F. Malaria epidemiology and control in refugee camps and complex emergencies. Annals of Tropical Medicine and Parasitology. 2001;95(8):741-54. DOI: https://doi.org/10.1080/00034983.2001.11813694

9. Paquet C, Hanquet G. Control of infectious diseases in refugee and displaced populations in developing countries. Bulletin of the Institut Pasteur. 1998;96:3-14.DOI: https://doi.org/10.1016/S0020-2452(98)80024-9

10. Santaniello-Newton A, Hunter P. Management of an outbreak of meningococcal meningitis in a Sudanese refugee camp in Northern Uganda. Epidemiol Infect 2000;124:75-81. DOI: https://doi.org/10.1017/S0950268899003398

11. Talley L, Spiegel P, Girgis M. An investigation of increasing mortality among Congolese refugees in Lugufu camp, Tanzania, May–June 1999. J Refug Stud. 2001;14:412-27. DOI: https://doi.org/10.1093/jrs/14.4.412

12. Toole M, Waldman R. An analysis of mortality trends among refugee populations in Somalia, Sudan, and Thailand. Bull World Health Organ. 1988;66:237-47.

13. Watson J, Gayer M, Connolly M. Epidemics after Natural Disasters. Emerging Infectious Diseases. 2007;13(1):no pagination.

14. Ahern M, Kovats R, Wilkinson P, Few R, Matthies F. Global health impacts of floods: epidemiologic evidence. Epidemiological Review. 2005;27:36-46. DOI: https://doi.org/10.1093/epirev/mxi004

15. Few R, Ahern M, Matthies F, Kovats R. Floods, health and climate change: a strategic review. Norwich: Tyndall Centre for Climate Change, University of East Anglia; 2004.

16. Du W, FitzGerald G, Clark M, Hou X. Health impacts of floods. Prehospital and Disaster Medicine. 2010;25:265-72. DOI: https://doi.org/10.1017/S1049023X00008141

17. Ivers L, Ryan E. Infectious diseases of severe weather-related and flood-related natural disasters. Current Opinion in Infectious Diseases. 2006;19(5):408–14. DOI: https://doi.org/10.1097/01.qco.0000244044.85393.9e

18. Abaya S, Mandere N, Ewald G. Floods and health in Gambella region, Ethiopia: a qualitative assessment of the strengths and weaknesses of coping mechanisms. Global Health Action. 2009;2.

19. Alderman K, Turner L, Tong S. Floods and human health: A systematic review. Environment International. 2012;47:37-47. DOI: https://doi.org/10.1016/j.envint.2012.06.003

20. Carroll B, Balogh R, Morbey H, Araoz G. Health and social impacts of a flood disaster: responding to needs and implications for practice. Disasters. 2010;34:1045-63. DOI: https://doi.org/10.1111/j.1467-7717.2010.01182.x

21. Floret N, Viel J, Mauny F, Hoen B, Piarroux R. Negligible Risks for Epidemics after Geophysical Disasters. Emerging Infectious Diseases. 2006;12(4):543-8. DOI: https://doi.org/10.3201/eid1204.051569

22. Toole M. Mass population displacement. A global public health challenge. Infectious Disease Clinics of North America. 1995;9(2):353-66.

23. Hammer C, Brainard J, Hunter P. Risk factors and risk factor cascades for communicable disease outbreaks in complex humanitarian emergencies: A qualitative systematic review. BMJ Global Health. 2018;3(4):e000647. DOI: https://doi.org/10.1136/bmjgh-2017-000647

24. Brennan RJ, Nandy R. Complex humanitarian emergencies: A major global health challenge. Emergency Medicine. 2001;13(2):147-56. DOI: https://doi.org/10.1046/j.1442-2026.2001.00203.x

25. Coulombier D, Pinto A, Valenciano M. Epidemiological surveillance during humanitarian emergencies. Medecine Tropicale. 2002;62(4):391-5.

26. Cuadrado PC, González PA. Epidemic cholera in complex emergencies. Revista Espanola de Salud Publica. 2014;88(2):191-201.

27. Fisher ML, Brahmbhatt D, Powers R, Daily E. Health issues in humanitarian emergencies. International Disaster Nursing2010. p. 427-48. DOI: https://doi.org/10.1017/CBO9780511841415.028

28. Bompangue D, Giraudoux P, Piarroux M, Mutombo G, Shamavu R, Sudre B, et al. Cholera epidemics, war and disasters around Goma and Lake Kivu: an eight-year survey. PLoS neglected tropical diseases. 2009;3(5). DOI: https://doi.org/10.1371/journal.pntd.0000436

29. Stich A. Frequent infectious diseases in migrants. [German]. Internist. 2016;57(5):409-15. DOI: https://doi.org/10.1007/s00108-016-0057-3

30. McNichols A, Lennon D, Crampton B, WHowden-Chapman P. Overcrowding and infectious diseases--when will we learn the lessons of our past? New Zealand Medical Journal. 2000;113(1121):453-4.

31. Al-Khatib I, Ju'ba A, Kamal N, Hamed N, Hmeidan N, Massad S. Impact of housing conditions on the health of the people at al-Ama'ri refugee camp in the West Bank of Palestine. International Journal of Environmental Health Research. 2003;13(4):315-26. DOI: https://doi.org/10.1080/09603120310001616092

32. Kouadio I, Aljunid S, Kamigaki T, Hammad K, Oshitani H. Infectious diseases following natural disasters: prevention and control measures. Expert Review of Anti-infective Therapy. 2010;10(1):95-104. DOI: https://doi.org/10.1586/eri.11.155

33. Mipatrini D, Stefanelli P, Severoni S, Rezza G. Vaccinations in migrants and refugees: a challenge for European health systems. A systematic review of current scientific evidence. Pathogens and Global Health. 2017;111(2):59-68. DOI: https://doi.org/10.1080/20477724.2017.1281374

34. Spens K, Kovács G. Identifying challenges in humanitarian logistics. International Journal of Physical Distribution & Logistics Management. 2009;39(6):506-28. DOI: https://doi.org/10.1108/09600030910985848

35. Macias L. Complex Emergencies. Austin: Center for Climate Change and African Political Stability; 2013.

36. Banatvala N, Zwi A. Public health and humanitarian interventions: developing the evidence base. BMJ. 2000;321(7253):101-5. DOI: https://doi.org/10.1136/bmj.321.7253.101

37. Cuadrado P, González P. CÓLERAEPIDÉMICOENSITUACIONESDEEMERGENCIACOMPLEJA. Rev Esp Salud Pública. 2014;88:191-201.

38. Burkle Jr FM. Pediatric issues in complex emergencies. Ambulatory Child Health. 2001;7(2):119-26. DOI: https://doi.org/10.1046/j.1467-0658.2001.00122.x

39. Nicole W. The WASH Approach: Fighting Waterborne Disease in Emergency Situations. Environ Health Perspect. 2015. DOI: https://doi.org/10.1289/ehp.123-A6

40. Blanchet K, Ramesh A, Frison S, Warren E, Hossain M, Smith J, et al. Evidence on public health interventions in humanitarian crises. Lancet. 2017;390(10109):2287-96. DOI: https://doi.org/10.1016/S0140-6736(16)30768-1

41. Ramesh A, Blanchet K, Ensink J, Roberts B. Evidence on the Effectiveness of Water, Sanitation, and Hygiene (WASH) Interventions on Health Outcomes in Humanitarian Crises: A Systematic Review. PLoS ONE. 2015;10(9):e0124688. DOI: https://doi.org/10.1371/journal.pone.0124688

42. Jafari N, Shahsanai A, Memarzadeh M, Loghmani A. Prevention of communicable diseases after disaster: A review. J Res Med Sci. 2011;16(7):956–62.

43. Close RM, Pearson C, Cohn J. Vaccine-preventable disease and the under-utilization of immunizations in complex humanitarian emergencies. Vaccine. 2016;34(39):4649-55. DOI: https://doi.org/10.1016/j.vaccine.2016.08.025

44. Howard N, Chandramohan D, Freeman T, Shafi A, Rafi M, Enayatullah S, et al. Socio-economic factors associated with the purchasing of insecticide-treated nets in Afghanistan and their implications for social marketing. Tropical Medicine and International Health. 2003;8(12):1043-50. DOI: https://doi.org/10.1046/j.1365-3156.2003.01163.x

45. Howard N, Shafi A, Jones C, Rowland M. Malaria control under the Taliban regime: Insecticide-treated net purchasing, coverage, and usage among men and women in eastern Afghanistan. Malaria Journal. 2010;9(1). DOI: https://doi.org/10.1186/1475-2875-9-7

46. Liddle KF, Elema R, Thi SS, Greig J, Venis S. TB treatment in a chronic complex emergency: Treatment outcomes and experiences in Somalia. Transactions of the Royal Society of Tropical Medicine and Hygiene. 2013;107(11):690-8. DOI: https://doi.org/10.1093/trstmh/trt090

47. Burkle F. Lessons learnt and future expectations of complex emergencies. BMJ. 1999; 319: 422-6. DOI: https://doi.org/10.1136/bmj.319.7207.422

48. Sparrow A, Almilaji K, Tajaldin B, Teodoro N, Langton P. Cholera in the time of war: implications of weak surveillance in Syria for the WHO's preparedness—a comparison of two monitoring systems. BMJ Global Health. 2016;1(3):e000029. DOI: https://doi.org/10.1136/bmjgh-2016-000029

49. The Sphere Project. The Sphere Handbook: Humanitarian Charter and Minimum Standards in Humanitarian Response. Geneva: Sphere Association c/o NRC; 2011.

50. Kynn M. The 'heuristics and biases' bias in expert elicitation. Journal of the Royal Statistical Society: Series A (Statistics in Society). 2007;171(1):239-64. DOI: https://doi.org/10.1111/j.1467-985X.2007.00499.x

51. Speirs-Bridge A, Fidler F, McBride M, Flander L, Cumming G, Burgman M. Reducing Overconfidence in the Interval Judgments of Experts. Risk Analysis. 2010;30(3):512-23. DOI: https://doi.org/10.1111/j.1539-6924.2009.01337.x

52. WHO. Communicable disease risk assessment: protocol for humanitarian emergencies. Geneva: WHO; 2007.

53. WHO. Outbreak surveillance and response in humanitarian emergencies. WHO guidelines for EWARN implementation. Geneva: WHO; 2012.