This study aimed to evaluate the impact of integrating Nepali language
EPIWATCHA(r) can detect early signals of disease outbreaks in 51 languages, including English. In August 2021, Nepali language search terms were added to EPIWATCHA(r), and by October 2021, the system began scanning local Nepali news articles. We conducted a study using a descriptive comparative analysis design to analysed EPIWATCHA(r) outbreaks reports from Nepal before and after adding the Nepali search terms.
Of the 517 reports EPIWATCHA(r) identified between January 1, 2018, and June 30, 2023, 43.1% (223) came from Nepali-language articles, mostly detected after adding Nepali search terms. EPIWATCHA(r) identified three times more outbreak reports from Nepal than HealthMap in the year after adding Nepali search terms (271 from October 2021 to September 2022, compared to 42 from October 2020 to September 2021). EPIWATCHA(r) sometimes reported more infectious cases than EWARS Nepal, especially for Dengue (>28,000 vs 8,923) and Cholera (961 vs 58) from October 2021 to September 2022.
The study demonstrated the increased efficiency of EPIWATCHA(r) following the integration of Nepali language search terms, which enhanced its ability to scan local news articles. This research underscores the importance of language-specific Open-source intelligence (OSINT) in detecting infectious disease outbreaks and highlights the need for incorporating local languages into surveillance systems for more effective grassroots-level monitoring.
Epidemiological disease surveillance is the cornerstone of public health and is crucial for tracking, preventing, nowcasting, and forecasting diseases (Aiello et al., 2020). The COVID-19 pandemic has underscored the importance of robust surveillance systems. Active surveillance, while more resource-intensive, is more effective than passive and sentinel surveillance in detecting outbreaks promptly and accurately (Immunization Academy, 2020). Syndromic surveillance, which focuses on specific symptoms without waiting for diagnosis or lab results, is another key strategy (CDC, 2020).
The World Health Organization (WHO) defines public health surveillance as "the continuous and systematic collection, orderly consolidation and evaluation of pertinent data with prompt dissemination of results to those who need to know, particularly those who are in a position to take action" (World Health Organization, 2023). Traditional surveillance systems, though foundational, have limitations. They rely on reported cases, which can underestimate the true disease burden and often involve delays while awaiting diagnoses (Aiello et al., 2020). Open-source intelligence (OSINT) has emerged as a valuable tool to enhance traditional surveillance, though not as a replacement. Integrating OSINT with established systems can add significant value (Milinovich et al., 2014; Yan et al., 2017; Hii et al., 2018).
Surveillance techniques have evolved significantly, especially with the advent of the internet, which has broadened the scope for detecting, predicting, controlling, and preventing disease outbreaks. In today?(tm)s interconnected world, an outbreak in a small area can rapidly spread across regions and even globally, as seen during the COVID-19 pandemic (Lesmanawati et al., 2020; MacIntyre et al., 2022). This highlights the need for continuous efforts and efficient mechanisms to detect outbreaks swiftly.
OSINT in public health surveillance, from data collection to interpretation and dissemination, is increasingly automated thanks to advances in information technology (Hughbank & Githens, 2010). Leveraging user-generated data from the internet and social media allows for maximizing crowd-sourced information (Li et al., 2016). Digital public health surveillance is thus defined as "public health surveillance with the inclusion of digital data, particularly from social media or other internet-based sources" (Aiello et al., 2020, p.103).
While internet-based surveillance systems often have high sensitivity, they may suffer from low specificity. Nonetheless, they are effective at quickly identifying outbreaks and epidemics (Gardy & Loman, 2018). Various internet-based surveillance systems, such as HealthMap, BioCaster, and EPIWATCHA(r), are used to detect outbreaks rapidly. Some systems are open-source, while others require a subscription (MacIntyre et al., 2023a). However, digital surveillance is sometimes criticized for not capturing the actual occurrence of disease but rather public awareness (Aiello et al., 2020).
The rise of digital public health surveillance also brings new ethical concerns, particularly around identifying personal data. Despite these challenges, digital surveillance systems present unprecedented opportunities for improvement. Continuous innovation, adequate training of public health personnel, and integration of AI technology into the public health infrastructure are essential to advance traditional surveillance systems (Aiello et al., 2020; MacIntyre et al., 2023a).
In Nepal, the primary disease outbreak reporting system is the traditional hospital-based sentinel surveillance system known as EWARS Nepal (NHRC, 2019). Established in 1996, EWARS Nepal facilitates the flow of information on vector-borne and other outbreak-prone infectious diseases from across the country's districts (EDCD, 2016). Reports on suspected diseases from the community are sent to the Vector-Borne Disease Research and Training Center (VBDRTC), which then forwards summaries to the Epidemiology and Disease Control Division (EDCD) for weekly publication on its website (EDCD, 2016). Currently, EWARS Nepal operates in 118 hospitals across all 77 districts, focusing primarily on six priority diseases/syndromes: Malaria, Kala-azar, Dengue, Acute Gastroenteritis (AGE), Cholera, Severe Acute Respiratory Infection (SARI), and other epidemic-potential diseases (NHRC, 2019).
EPIWATCHA(r), launched in 2016, is an AI-driven, automated rapid intelligence system for epidemic surveillance (MacIntyre et al., 2022). It reviews epidemic outbreak news from 51 languages daily, including English (MacIntyre et al., 2023b). By utilizing open-source data from online news outlets, EPIWATCHA(r) has proven useful in detecting outbreaks earlier than traditional methods, even assisting countries without formal reporting systems (Bhargavi & Moa, 2020; Lesmanawati et al., 2020; MacIntyre et al., 2020; Mahmood et al., 2021; Mao et al., 2020; Nair et al., 2020; Puca & Trent, 2020; Stone et al., 2020; Sulaiman et al., 2019). To enable early warning EPIWATCHA(r) operates with high sensitivity in real-time data collection, with expert analysts reviewing the database daily (MacIntyre et al., 2022).
This study aims to examine infectious disease outbreaks reported by EPIWATCHA(r) in Nepal, comparing data before and after the inclusion of Nepali language search terms, and to compare these findings with data from EWARS Nepal and HealthMap. This paper first describes the data sources and analytical approach, then compares outbreak detection performance before and after language integration and concludes with implications for digital surveillance in low-resource settings.
The study consists of two parts. The first part involves a descriptive analysis of EPIWATCHA(r) reports i.e. number of outbreak reports for Nepal from January 1, 2018, to June 30, 2023. A comparison was also made between EPIWATCHA(r) reports, as the primary indicator of impact following language integration. from one year before (October 2020 to September 2021) and one year after (October 2021 to September 2022) the inclusion of Nepali language search terms. The study period (January 1, 2018 ?" June 30, 2023) was selected to capture trends before and after the inclusion of Nepali language search terms, allowing for sufficient data representation across multiple epidemic seasons. The one-year pre - and post-comparison (October 2020 - September 2021 and October 2021 - September 2022) was specifically chosen to control for confounding temporal effects and to allow a balanced comparison between equivalent 12-month intervals immediately before and after implementation. The second part compares EPIWATCHA(r) reports/cases with those from EWARS Nepal and HealthMap. The top ten human diseases in Nepal, according to EPIWATCHA(r) and HealthMap, were also listed. The keywords included disease-specific and syndrome-based terms such as ?oCOVID-19,?? ?oDengue,?? ?oCholera,?? ?oScrub Typhus,?? ?oMalaria,?? ?oMeasles,?? ?oKala-azar,?? and ?oAcute Gastroenteritis,?? along with their Nepali equivalents (e.g., ?o Ψ , - ?,?? ?o 1 < ύ ¨ ψ 3/4,?? ?o ψ ? ~ 3/4 - < ???) - Supplementary material.
For the initial phase, the EPIWATCHA(r) team provided outbreak report data in Microsoft Excel format. Data specific to Nepal were extracted for the designated study period. A few reports initially misclassified under locations other than Nepal were corrected and included in the study where relevant. The data were then organized by publication date, disease or syndrome, outbreak location, and associated URL (Uniform Resource Locator) links. Manual deduplication was conducted carefully to minimize subjectivity, with all records independently verified and cross-checked before inclusion in the final dataset.
Google Chrome and Google Translate were used to retrieve and translate articles from the provided URL links. The data were further reviewed for accuracy, duplication, and location verification. Selection criteria were established to exclude reports unsuitable for the study (Appendix A). After removing incorrect and duplicate entries, the data were categorized for descriptive analysis. Each article was manually reviewed to extract the number of cases and deaths, as provided in the reports.
EPIWATCHA(r) began scanning Nepali-language articles in the first week of October 2021. Consequently, the data were divided into two periods: 12 months before and 12 months after the inclusion of Nepali search terms. This division resulted in two distinct datasets for analysis: the first from October 2020 to September 2021, and the second from October 2021 to September 2022. These datasets were then compared by examining the number of reports generated by EPIWATCHA(r) during each period (Table 2).
In the second part, the efficiency of EPIWATCHA(r) was compared with two other outbreak portals: EWARS Nepal and HealthMap. Lists of outbreaks from EWARS Nepal for the two years (October 2020 ?" September 2021 & October 2021 ?" September 2022) were prepared to be compared against EPIWATCHA(r) data for the same period (Table 4). EWARS Nepal, a hospital-based sentinel surveillance system, reports six priority diseases/syndromes and other potential epidemic diseases (Ministry of Health and Population, 2023). Weekly outbreak reports from EWARS Nepal were analyzed to compile case numbers and deaths for the specified period. Similarly, EPIWATCHA(r) data were reviewed to create a line list and extract case numbers and deaths. The analysis focused on outbreak-prone diseases routinely reported in Nepal (Dengue, Cholera, Scrub Typhus, Malaria, Kala-azar, Acute Gastroenteritis, Measles, and others), aligning with EWARS Nepal?(tm)s priority list and WHO surveillance guidance.
HealthMap, an internet-based intelligence system since 2006, was also analyzed. Unlike EPIWATCHA(r), HealthMap does not involve human moderation and reports epidemics and other health events automatically (MacIntyre et al., 2023a). After registering with HealthMap, epidemic alerts/reports for Nepal from January 1, 2018, to June 30, 2023, were analyzed. These data were divided into three periods: October 2020 to September 2021, October 2021 to September 2022, and January 2018 to June 2023 (Table 5). The comparison between HealthMap and EPIWATCHA(r) was limited to reports/alerts, excluding case numbers and deaths.
Finally, reports or cases from all three portals?"EPIWATCHA(r), EWARS Nepal, and HealthMap?"were reviewed for one year (October 2021 ?" September 2022) (Appendix B). The top 10 diseases/syndromes in Nepal were listed based on the number of cases (EPIWATCHA(r)) or reports made (HealthMap) over five and a half years (January 1, 2018 ?" June 30, 2023). No inferential statistics were performed due to the descriptive focus and data characteristics (i.e. open-source, heterogeneous reporting).
A total of 577 reports were reviewed, with 517 reports included in the final analysis (Table 1). The most common reasons for excluding reports were faulty URLs, followed by duplicate articles and reports unrelated to locations in Nepal (Appendix A).
The highest number of reports made during the study period were COVID-19 (192 reports), Dengue (78 reports), and Cholera (71 reports), followed by Measles and Scrub Typhus (Table 1). Of the 517 reports, 62 were due to various animal diseases. Geographically, the source of these reports was divided into Nepal and its seven provinces. Out of the 517 reports, 285 (55.1%) were for Nepal, 138 (26.7%) for Bagmati Province, and only 94 (18.2%) for the remaining six provinces (Figure 1). The 285 reports labelled ?oNepal?? represent those that were national in scope or not geographically specified beyond the country level. The remaining reports (232) were disaggregated by province with no overlap between national and provincial reports.
EPIWATCHA(r) began incorporating Nepali language search terms in August 2021 and extracted the first report from a Nepali language article on October 5, 2021. Of the 517 reports analyzed, 460 were made after the inclusion of Nepali translation terms. EPIWATCHA(r) utilized 31 local newspaper websites in Nepali after incorporating these terms. Among the 517 reports, 275 (53.2%) were based on articles from local Nepali newspapers, with 52 in English and 223 in Nepali. Reports from already curated sources numbered 128 (24.8%), and those from outside Nepal totalled 114 (22%), with 78 in English and 36 in other languages. During the five-and-a-half-year study period, 223 (43.1%) of the 517 reports originated from Nepali language articles.
EPIWATCHA(r) made only 42 reports between October 2020 and September 2021, just before the inclusion of Nepali language terms. In contrast, EPIWATCHA(r) made 271 reports between October 2021 and September 2022, one year after the inclusion of Nepali language terms, a sixfold increase compared to the previous year when no Nepali language terms were present (Table 2).
Table 3 outlines the human cases and deaths due to various diseases and syndromes in Nepal during the study period, as captured by the EPIWATCHA(r) reports. COVID-19 accounted for over a million cases and more than 12,000 deaths. Other significant diseases included Dengue, Scrub Typhus, Cholera, and Acute Gastroenteritis. In the first six months of 2023, there were 1,021 cases and one death due to Measles. During the five-and-a-half-year study period, EPIWATCHA(r) recorded substantial numbers of Malaria (1,938 cases) and Tuberculosis (3,653 cases). Other reported fatalities occurred due to Japanese Encephalitis, Rabies, and potential Swine Flu cases. Lyme Disease was first detected in Nepal, and Monkeypox was identified in one foreigner (Table 3).
EPIWATCHA(r) first reported COVID-19 in Nepal on September 13, 2020, documenting 54,159 cases and 345 deaths. Later, EPIWATCHA(r) identified that the total number of COVID-19 cases in Nepal surpassed 1 million by January 2023, with the total number of deaths reaching 12,031 by May 14, 2023. Following COVID-19, Dengue was the most frequently reported disease in EPIWATCHA(r). By analyzing the articles scanned by EPIWATCHA(r) for Dengue reports, we traced more than 60,000 cases and 73 deaths during the study period, with the majority occurring in 2022 (Table 3).
Although there were fewer reports of Scrub Typhus throughout the study period (Table 1), we identified a total of 2,333 cases and 13 deaths across the country, with Bagmati and Sudurpaschim provinces being the most affected (Table 3). The measles epidemic was first noted in 2023 and was ongoing until the study period ended in June 2023. EPIWATCHA(r) made the first measles report on January 5, 2023, tracing over 1,000 cases across 17 districts in Nepal. A single death due to measles was reported on March 14, 2023.
Cholera outbreaks were concentrated in two periods, one in 2021 and another in 2022: there were 885 cases and four deaths in October 2021, and 76 cases and two deaths from June to September 2022. Cholera and Acute Gastroenteritis were also significant contributors to morbidity and mortality in Nepal (Table 3).
The case numbers reported by EPIWATCHA(r) were compared for the same seven diseases or syndromes (six priority diseases and one with epidemic potential) for which EWARS Nepal published data. Table 4 presents two sets of data (case numbers): one from before and one from after the inclusion of Nepali search terms, referred to as the "prior list" and "after list" from October 2021 onward. There were only 42 reports available for case number exploration in the prior list, compared to 271 reports in the after list (Table 2).
Syndromes like Severe Acute Respiratory Infection (SARI) were not observed in either EPIWATCHA(r) list. EPIWATCHA(r) captured Acute Gastroenteritis (AGE) cases in the after list, but the numbers were significantly lower compared to EWARS Nepal (Table 4). Only one report each for Kala-azar (Leishmaniasis) was found in the prior and after lists of EPIWATCHA(r), but case numbers were not mentioned in either report.
Out of the seven diseases or syndromes for which EWARS Nepal publishes weekly reports (excluding COVID-19), the top two by the number of outbreak cases over the two-year study period were AGE and SARI (Table 4). Case numbers for both AGE and SARI were consistently reported each week. Reported cases of Dengue increased exponentially after June 2022, while Scrub Typhus cases were consistently present throughout the study period. EWARS Nepal sporadically recorded Malaria and Kala-azar (visceral leishmaniasis) cases in both years but registered Cholera cases only in the after list, the second year of the study period (Table 4).
In the prior list, EPIWATCHA(r) captured fewer cases of Malaria, Dengue, and Scrub Typhus compared to EWARS Nepal. In the after list, EPIWATCHA(r) continued to report fewer cases of Malaria and Scrub Typhus but captured more cases of Cholera and Dengue than EWARS Nepal (Table 4).
From October 2020 to September 2021, HealthMap reported 216 disease outbreak alerts for Nepal, including 145 for COVID-19 alone. In comparison, EPIWATCHA(r) made only 42 reports, with 22 related to COVID-19 (Table 5). However, from October 2021 to September 2022, EPIWATCHA(r) produced three times more outbreak reports for Nepal than HealthMap. During this period, HealthMap significantly reduced its COVID-19 alerts, reporting only seven, while EPIWATCHA(r) reported 116 COVID-19 alerts (Table 5). HealthMap articles linked to its reports or alerts were not reviewed to extract case numbers and death tolls.
This study analysed 517 disease outbreak reports in EPIWATCHA(r) for Nepal from January 1, 2018, to June 30, 2023. During this period, most cases and deaths were due to COVID-19, followed by Dengue, Tuberculosis, Scrub Typhus, Malaria, and Measles. EPIWATCHA(r) became more effective than EWARS Nepal and HealthMap in reporting disease outbreaks in Nepal after the inclusion of Nepali language search terms. This underscores the importance of local language capabilities in detecting disease outbreaks in regions with diverse national and traditional languages.
In its early stages, EPIWATCHA(r) issued only 57 reports for Nepal between January 1, 2018, and August 2021, before Nepali translation terms were introduced. As a result, EPIWATCHA(r) missed the first COVID-19 case in Nepal in January 2020, which was first reported by EWARS Nepal on January 26, 2020, in its weekly bulletin. HealthMap, through ProMED-mail, also issued an alert for a suspected COVID-19 case in Nepal on January 20, 2020, and confirmed it on January 24, 2020. EPIWATCHA(r) did not report on COVID-19 cases in Nepal until September 13, 2020, nearly seven to eight months after the first case was reported.
A case study of the Malay language in 2016/2017 by Sulaiman et al. (2019) demonstrated that informal internet-based surveillance could uncover more information than formal surveillance, particularly in low-income countries. They highlighted the need for local language search terms to improve outbreak detection. Similarly, EPIWATCHA(r) detected possible early signals of COVID-19 in Indonesia in 2019-20 using the local Indonesian language, "Bahasa," before the formal notification of the disease (Thamtono et al., 2021).EPIWATCHA(r) initially reported fewer disease outbreaks from Nepal, but its activity increased significantly once Nepali language search terms were incorporated. Reports sourced mainly from Nepali language articles allowed EPIWATCHA(r) to update COVID-19 cases and deaths regularly and promptly. Notably, EPIWATCHA(r) detected and reported the first Omicron variant found in Nepal on December 6, 2021, the same day the initial report was available.
While HealthMap is fully automated, EPIWATCHA(r) is a semi-automated system, requiring analysts to review reports daily (MacIntyre et al., 2022). The human review process in EPIWATCHA(r), conducted by trained researchers, scientists, and health professionals, adds a layer of accuracy, making it more efficient than other systems. However, this process is not without flaws; for example, in May 2019, an error was made when a human case of Avian Influenza A(H5N1) was misclassified as an animal disease, even though it was the first human infection with the bird flu virus in South Asia and the first global case since September 2017 (Acharya et al., 2020; CDC, 2019).
After incorporating local language terms, EPIWATCHA(r) issued six times more reports than before (Table 2). Before this inclusion, EPIWATCHA(r) reported fewer cases than EWARS Nepal for the seven diseases/syndromes EWARS Nepal monitored weekly. However, after the inclusion, EPIWATCHA(r) reported more cases of Dengue and Cholera than EWARS Nepal (Table 4). Furthermore, EPIWATCHA(r) issued three times more outbreak reports than HealthMap after gaining local language scanning capabilities. In some instances, EPIWATCHA(r) reported cholera outbreak cases slightly earlier than EWARS Nepal in the year following the inclusion of the Nepali language. Whilst no inferential statistics were performed, the increase in EPIWATCHA(r) reports coincided with ongoing epidemic activity (particularly COVID-19 and post-pandemic resurgences of Dengue and Cholera), the comparative analysis with HealthMap and EWARS Nepal demonstrates that the increase in EPIWATCHA(r) detections was disproportionately higher than expected from background disease trends alone. This suggests that the observed rise in reporting is primarily attributable to the enhanced local language detection capability rather than solely to changes in disease incidence.
There are limitations to this study. Each article reported by EPIWATCHA(r) was reviewed to determine whether case numbers were mentioned. The credibility of the numbers reported by government sources like EWARS Nepal is generally higher than those in news articles reported by EPIWATCHA(r) or similar portals. While this study used a descriptive comparative approach appropriate to the data characteristics and open-source, heterogeneous reporting, we acknowledge that the absence of formal statistical testing limits the ability to quantify the magnitude and significance of observed differences. Future research should incorporate quantitative validation methods such as trend analysis, sensitivity and specificity testing, and time-lag comparisons to assess the statistical significance of differences in detection performance before and after language integration.
For comparison, the EWARS Nepal database was assumed to be the standard, but EWARS Nepal only reports on seven diseases or syndromes weekly. Therefore, other diseases or syndromes reported by EPIWATCHA(r) could not be compared with EWARS Nepal. Future research will incorporate automated lead-time validation against national surveillance data, and time-series analyses to quantify the impact of local language integration.
Open-source intelligence (OSINT) provides fast, integrated, and timely data for outbreak detection and response. Adding language-specific translation capabilities to disease outbreak reporting portals enhances the utility of these tools. This study demonstrated that after the inclusion of Nepali language search terms, EPIWATCHA(r) began detecting more disease outbreak signals from Nepal.
Compared to EWARS Nepal, a more traditional national disease outbreak reporting portal for Nepal, EPIWATCHA(r) covers a broader range of infectious diseases to report on. Even for the diseases routinely reported by EWARS Nepal, EPIWATCHA(r) was sometimes quicker to report cases. Similarly, compared to HealthMap, EPIWATCHA(r) began issuing three times more outbreak reports after incorporating local language articles scanning capability. By utilizing language-specific intelligence, EPIWATCHA(r) is becoming a vital tool in digital epidemiology, particularly for resource-limited countries.
This study used publicly available, open-source data and did not involve human or animal subjects. No personally identifiable information was collected, and therefore, formal ethical approval was not required.
The authors had full access to all of the data (including statistical reports and tables) in this study.
The study data can be accessed by contacting the corresponding author.
Competing interests
Nothing to declare.
Funding
The Balvi Filantropik Fund supports AQ.
Raina MacIntyre is supported by a NHMRC Investigator Grant 2016907