Introduction

Public health surveillance of communicable diseases is a major cornerstone in International Health Regulations’ (IHR) detection, response and containment strategy adopted by all countries. Since emerging infections like COVID-19 are major threats to human lives, showing high prevalence and incident rates within a short time span (1), an effective, comprehensive and integrated disease surveillance system is required for timely detection and response. Whether it is an indicator based or event based surveillance, an integration with public health laboratories is a must to fully understand disease evolution (2). Not only does it provide early warning, it acts as a platform where surveillance and disease containment implementation strategies can be monitored (3). The surveillance systems should be designed in a way to enhance effective, timely monitoring and control of these diseases.

Since the first case appeared in China in December 2019, emerging novel coronavirus disease (COVID-19) has evolved as a rapidly spreading pandemic with high morbidity and mortality rates across the world. With little information on this new viral disease, the WHO lays special emphasis on uniform data collection with comprehensive and timely information at the different organizational levels. This will subsequently result in an understanding of the disease’s dynamics based on high quality data (4). Development of electronic systems and its integration with laboratories, further facilitates for collecting and reporting disease data in a timely manner (5). This information is critical for detecting and characterizing outbreaks of novel diseases and to determine the size, spread, and tempo of an outbreak after it is detected. Effective and efficient participation of the laboratories in surveillance helps in confirmation of diagnosis, allows a high specificity and provides additional details related to the pathogens.

With the support of developmental partners like the US Center for Disease Control (CDC) and others, a national laboratory-based disease surveillance system at National Institute of Health, Islamabad was established, which acts as a centralized data warehouse. The Ministry of health is directly in charge of coordinating and supporting this system. Different organizations at the local, provincial and state levels work with the ministry of health in terms of sharing information and logistic support.

Pakistan attained testing capacity using real time polymerase chain reactions (RT-PCR) on January 26, 2020, while the first case was tested positive on February 27, 2020 and the testing started at national level at NIH. Several measures have already been implemented to prevent and control possible case importations from different countries (3, 4). To minimize the disease spread, surveillance measures including active contact tracing in communities and Point of Entries (PoEs), were also established (7) as shown in Figure 1.

Figure 1 

Flow of data of national COVID-19 surveillance program across all stakeholder Islamabad

*(NIH- National institute of health, WHO- World Health Organization, FDSRU- Federal disease surveillance and response unit, MNHSR&C- Ministry of national health services regulation and coordination, NCOC- National Command and Control Centre).

By implementing the above mentioned strategies, a continuous monitoring and a quick evaluation was mandated. Our objectives were to evaluate this apical laboratory based surveillance system to understand its strengths and weakness on system attributes designed by the CDC and finally propose some doable recommendations in the context of a resource limited country like Pakistan.

Methods

The surveillance system review was undertaken from February to March, 2020. In order to evaluate the national COVID-19 surveillance system, the CDC’s Updated Guidelines for Surveillance System Analysis - the guidelines for the evaluation of public health surveillance systems published by the US CDC 2001 - were used (8). After defining the objective and system attributes, we identified all stakeholders and key informants. Prior to conducting study an ethical approval was taken from the NIH Institutional review board (IRB). An informed written consent was taken explaining purpose of study and the interviewee’s right to leave the study without any liabilities. Interviews were conducted to explore operational aspects of the system by using a semi structured questionnaire. Other operational managers of allied institutions, field epidemiologists, and other relevant departments were also interviewed to validate the information on system functions.

A desk review was conducted in accordance with system attributes, which includes policy documents, procedural guidelines, database, and performance reports. Case investigation forms, contact tracing and laboratory request forms were also collected and reviewed for data quality completeness and relevance of information. Using a database which includes all the suspected cases coming in this time period of study, system attributes were measured. The measurement of the scale was categorized as good, average and poor using a Likert scale.

Operations of COVID-19 Surveillance

  • The demographics, contact history, clinical and risk factor information of all suspected cases were collected on standardized case investigation forms.
  • The missing data was collected by contacting patients through telephones.
  • Data is managed in LIMS (Laboratory Information Management Software) for prompt referral.
  • Lab staff and rapid response teams (RRTs) are working in three shifts (morning, evening and night).
  • Lab reports are generated in 24-48 hours of sample collection and uploaded on LIMS data repository.
  • Follow-up is done by an expert team for contact tracing and repeat testing.
  • Patients turned positive on RT-PCR are referred to quarantine centers or respective hospitals.
  • RRTs are in liaison with lab workers for sharing of information.
  • A small numbers of private labs. and NIH affiliated public health laboratories located at provincial level are sharing information on a daily basis. A daily situation report and descriptive analysis is being communicated with all national and international stakeholders.
  • Results

    A summary of the evaluation of 10 attributes from the CDC guidelines are presented in Table 1.

    Table 1

    Summary of the evaluation of 10 attributes

    Attributes Justification Evaluation
    Usefulness
  • Standard case definitions with guidelines.

  • Data are properly linked to action.
  • Average
    Data quality
  • Data are taken on standard lab forms.

  • Demographic and socioeconomic status information are sufficient.

  • Few, unknown, and missing values.
  • Good
    Qualitative Analysis
    Simplicity
  • Standard case definitions are followed

  • Short and focused.
  • Good
    Flexibility
  • Easy to add or remove variables.

  • Staff are well trained.

  • System is able to accommodate and detect changes easily.
  • Good
    Acceptability
  • High level of awareness of stakeholders with respect to procedures and their ownership.

  • No refusal and almost 100% reporting rate.

  • All public health facilities participate.
  • Good
    Stability
  • Data management with few reported failure.

  • System is fully functional with sustainable and financial support.

  • Database is stable despite regular electricity and power interruption.
  • Good
    Quantitative Analysis
    Sensitivity
  • Estimated targeted sensitivity 1.25%.
  • Poor
    Positive predictive value
  • Estimated PPV 4.50% *.
  • Poor
    Representativeness
  • All Public and private health facilities are covered.

  • Limited coordination with private labs initially.
  • Good
    Timeliness
  • Daily collection at provincial level and reporting

  • Feedback and response is timely and there is no time lag.
  • Good

    *The PVP is low as it is closely related to the clarity and specificity of the case definition.

    1. Usefulness:

    The system was found useful as the data collected and analyzed was used to track the trend of the epidemic, and for further investigation. Although, due to the novel disease, there is no system defined for the periodic analysis of COVID-19 specific data. Standard case definitions conforming to the WHO’s guidelines were being followed by all organizations for the reporting of cases.

    2. Data Quality:

    The completeness and validity of the recorded data was good. Among all records, 80% of the information was completely filled on the case investigation form and contact tracing forms. Data was received as hard copies from different provinces. Standardized laboratory investigation forms were being used and missing information on the contact tracing forms and case investigation form were later filled through telephonic contact.

    3. Simplicity:

    The system was found to be simple, as a uniform and consistent WHO standard case definition was being used by the laboratory. The system collects necessary information on demographics, clinical signs and symptoms, travel history, number of contacts and co-morbidities from all laboratories. The staff was found trained as the laboratory is IS0 9001-2015 certified which requires semiannual audit assessment of training records.

    4. Flexibility:

    The system was flexible enough to incorporate new respiratory diseases if demanded, though the system is already well-equipped in dealing with the cases of Influenza and SARS in the past years and is able to adopt and accommodate the new changes. The staff is well trained in terms of case investigations, transmitting and analyzing case information, but the time and resources are limited.

    5. Acceptability:

    All stakeholders are on board in this national health emergency. Both private labs and departments were sensitized and pay their due contributions. Since there was an awareness and anticipation of the epidemic, all stakeholders were already taken on board and were well prepared to play their part.

    6. Stability:

    The system is considered stable with sustainable and financial support from both government and international donor agencies. An uninterrupted electronic database system is working 24/7 and has been collecting necessary data without failure so far. The system is well equipped with computers, PPEs, and other logistics for collecting, managing and providing a timely and reliable response.

    7. Sensitivity:

    We defined sensitivity as the ability of the surveillance system to truly detect cases of COVID-19 in the country (sensitivity = cases detected/cases existing × 100). Based on case definitions for COVID-19, the patients were recorded as suspected if they showed fever, cough, sore throat and difficulty in breathing. They were then sent for throat or nasopharyngeal samples. We do not have the prevalence percentage of COVID-19 as it varies from country to country. So we estimated the cases on the basis of symptomatic and asymptomatic cases reported in our database (http://www.covid.gov.pk/).

         Total COVID -19 suspected cases reported till 31 March: 1699

         Total COVID -19 positive cases estimated till 31 March: 35,350

         Total COVID -19 positive cases detected till 31 March: 77

         Sensitivity = No. of cases screened / No. of expected cases x100

         Sensitivity = 1699 / 1, 35000 x100 = 1.25%

         Sensitivity was found to be 1.25%*

         *The sensitivity of the system is low due to the evolving case definition of COVID-19. Questions concerning sensitivity in surveillance systems most commonly arise when changes in disease occurrence are noted.

    8. Predictive Value Positive (PPV):

    We defined predictive value positive (PPV) as the proportion of positive cases out all of cases tested by laboratory facilities. After a review of data in our system we calculated the PPV as 4.5%.

         Total COVID -19 suspected cases reported till 31 March: 1699

         Total COVID -19 positive cases detected till 31 March: 77

         PPV = (No. of positive cases / No. of suspected cases reported to lab) x 100

         PPV= 77 / 1699 x100 = 4.5%

    9. Representativeness:

    Catchment of population renders this system representative. Most of the cases are reported from the public sector and tertiary care hospitals along with private sector tertiary hospitals and small clinics. Primary health care facilities refer suspected cases to nearby government tertiary care hospitals and after conforming to the case definition they send their sample to the national reference laboratory of Islamabad. The initial capacity of the laboratory was 1000 samples/day, which gradually increased to 3000 sample/day over two months. A low PVP for a system in which every reported case is investigated will lead to unnecessary intervention and therefore wasted resources.

    10. Timeliness

    The system was observed working round the clock with a typical reporting time of 24-48 hours. Timeliness was quantified as delays no more than 48 hours after sample collection. The maximum time taken for flow of information from different hospitals to the laboratory was one day. Data is submitted from hospitals to NIH Laboratory on a daily basis. After the case is reported positive, case response measures are taken within 6 hours which includes immediately contacting the positive case, isolating them, sending rapid response teams to the take precautionary measures regarding the isolation of patient, transferring to the hospital and samples from close contacts.

    Discussion

    Our results showed that COVID-19 surveillance was well integrated with active surveillance by providing timely and accurate reports. Low sensitivity and PPV was a limitation.

    Evaluation of a respiratory novel disease surveillance system shows the country’s capacity to detect, respond and contain the disease. Morbidity and mortality data is the main point of measure of the disease burden and shows the effectiveness of response activities and preventive measures taken in accordance. It is crucial to identify weaknesses and strengths of the system in this short span of time and provide decision makers with robust evidence to decide on its future and continuity (9). The system was supposed to identify COVID-19 patients in the early outbreak phase to help flatten the curve (7). All the data is valuable, in the context of the novelty of the disease and it will help the government and stake holders for better policy making, strategic planning and monitoring for these kinds of pandemics. Although it is less useful for detecting the outbreak, the response can be prompt (10). Though inconsistent initially, reports were later on completed through other non-traditional platforms like telephone calls, text messages and toll-free help lines. The only issue was usage of multiple forms by different organizations for data collection which was later on revised (11). Regarding system flexibility, this laboratory is able to adapt changes according to needs and operational demands with minimal additional costs. The laboratory is well equipped with both trained staff and testing capacity attributable to already established Influenza and SARS surveillance.

    Studies conducted on the Australian influenza surveillance system illustrated the importance of timely syndromic and laboratory surveillance of influenza in decreasing the burden of influenza in the Australian community (12). Tucker suggests that defining a certain center for leading and coordinating the entities involved in control and prevention of communicable diseases is one of the most important requirements for launching communicable diseases surveillance systems (13). This goes with the pivotal role of NIH in establishing the high rates of acceptability in the system.

    Likewise, in his review study of surveillance systems, Dato suggests that a number of centers should be set up at different levels for incorporating technical information into the communicable diseases surveillance system (14). These centers must be assigned to management and assessment of COVID surveillance system data, constant analysis of data, redefining criteria for inclusion and exclusion with respect to case definition of diseases into the surveillance system and dissemination of information about the diseases (15). Every organization involved in the process should be in close contact with the surveillance system. Here in our study, limited contribution of the private sector in the management of communicable diseases was one of the many challenges as concluded in the studied conducted previously (16). An evaluative study of the surveillance system in South Africa identified major weaknesses of the system that sentinel sites were unreachable to a proportion of the population (17).

    The system was capable of picking up almost all actual cases of COVID, since gold standard RT-PCR is being used for tests. As prevalence is unknown and sensitivity was less as compared to specificity, the PVP reflects the sensitivity and specificity of the case definition and the prevalence of the condition in the population. The PVP increases with increasing specificity and prevalence. A similar study conducted in China to evaluate influenza surveillance emphasized laboratory-confirmed surveillance and sensitivity tools for developing influenza early warning systems (18).

    In another study of the evaluation of surveillance systems in Pakistan, laboratory-confirmed surveillance is beneficial and an effective way to monitor respiratory disease transmission (19). Therefore, for a protraction of these surveillance systems, a number of issues must be addressed in the alert and watch mode of an outbreak such as shortages of human resources and underfunding, poor coordination among entities involved at different levels, and poor informatic skills of the surveillance staff which may prevent systems from winning the full satisfaction of containment of the disease. The results revealed that the minimum data set, i.e. demographic, laboratory, clinical and vaccination data must be recorded and reported in order to take response measures as done in the COVID-19 outbreak (4).

    Conclusion

    A robust surveillance system provides a good estimation of the burden of the disease. Since COVID-19 has not peaked in Pakistan yet, system modifications over time are expected. So far, the system has been doing well in terms of timely response, contact tracing, testing and quarantine. However, low sensitivity and PPV are limitations. Expansion of the system is necessary to represent the whole population with improved sensitivity and measure of disease trends and the true burden of the disease. Stability should be ensured by continuous supply of reagents, primers, PPEs and transport media, trained staff and other logistics. Based on our results, some areas of weaknesses were identified as discussed. Management of the pandemic needs a departure from paper-based systems and moving towards integrated electronic systems.

    Recommendation

    Expansion of surveillance is recommended by utilizing all public and private facilities so that representativeness could be attained fully and sensitivity can be improved. Provincial ownership and expansion of integrated disease surveillance and response (IDSR) are main areas to address in concerted efforts towards global health security. Representativeness can be further augmented by strengthening relationships between public and private sector organizations. The involvement of medical universities in the designing and execution of surveillance plans and risk communication campaigns will help decrease the burden on laboratories. For stabilizing the system, there is a dire need to reallocate the resources toward these kind of pandemic preparedness and other unforeseen health emergencies.

    Conflict of Interest

    No conflict declared.

    Acknowledgement

    Authors acknowledge the support of the Emergency Operation Center (EOC) and Field Epidemiology and Disease Surveillance Division (FEDSD) at National Institute of Health Islamabad in this evaluation.

    References

    1. Abat C, Chaudet H, Rolain JM, Colson P, Raoult D. Traditional and syndromic surveillance of infectious diseases and pathogens. Vol. 48, International Journal of Infectious Diseases. Elsevier B.V.; 2016. p. 22–8.

    2. De Lusignan S, Correa A, Smith GE, Yonova I, Pebody R, Ferreira F, et al. RCGP Research and Surveillance Centre: 50 years’ surveillance of influenza, infections, and respiratory conditions. Vol. 67, British Journal of General Practice. Royal College of General Practitioners; 2017. p. 440–1.

    3. Sanicas M, Forleo E, Pozzi G, Diop D. A review of the surveillance systems of influenza in selected countries in the tropical region. Vol. 19, Pan African Medical Journal. African Field Epidemiology Network; 2014.

    4. Janati A, Hosseiny M, Gouya MM, Moradi G, Ghaderi E. Communicable disease reporting systems in the world: A systematic review article. Vol. 44, Iranian Journal of Public Health. Iranian Journal of Public Health; 2015. p. 1453–65.

    5. Chandrasekar K. Use of Information Communication Technology (ICT) in communicable disease surveillance – a review of literature. Sri Lanka J Bio-Medical Informatics.

    6. Nisar N, Badar N, Aamir UB, Aamir BM, Kazi SSZ, Zaidi. The Substantial Burden of Influenza in a Tertiary Pediatric Settings in Pakistan: 2008–2014. Open Forum Infectious Diseases, 2015, 2(suppl_1):1721.

    7. Ng Y, Li Z, Chua YX, Chaw WL, Zhao Z, Er B, et al. Evaluation of the Effectiveness of Surveillance and Containment Measures for the First 100 Patients with COVID-19 in Singapore — January 2–February 29, 2020. MMWR Morb Mortal Wkly Rep. 2020 Mar 20 69(11):307–11.

    8. German R, Lee L, Horan J, Milstein R, Pertowski C, Waller M. Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep. 2001;50(RR-13).

    9. Anam LS, Badi MM, Assada MA, Al Serouri AA. Evaluation of Two Malaria Surveillance Systems in Yemen Using Updated CDC Guidelines: Lessons Learned and Future Perspectives. Inq (United States). 2019;56.

    10. Saeed KMI, Bano R, Asghar RJ. Evaluation of national tuberculosis surveillance system in Afghanistan. East Mediterr Heal J. 2013 Feb 1;19(02):200–7.

    11. Dehcheshmeh NF, Arab M, Foroushani AR, Farzianpour F. Survey of Communicable Diseases Surveillance System in Hospitals of Iran: A Qualitative Approach. Glob J Health Sci. 2016;8(9).

    12. Sullivan SG, Raupach J, Franklin LJ, Pennington K, Bareja C, de Kluyver R, et al. A brief overview of influenza surveillance systems in Australia, 2015. Commun Dis Intell Q Rep. 2016 Sep 30;40(3):E351–5.

    13. Lee LM, Thacker SB, Centers for Disease Control. The cornerstone of public health practice: public health surveillance, 1961--2011. MMWR Surveill Summ Morb Mortal Wkly report Surveill Summ / CDC. 2011 Oct 7;60 Suppl 4:15–21.

    14. Dato V, Wagner MM, Fapohunda A. How outbreaks of infectious disease are detected: A review of surveillance systems and outbreaks. Public Health Rep. 2004;119(5):464–71.

    15. Sahal N, Reintjes R, Aro AR. Review article: communicable diseases surveillance lessons learned from developed and developing countries: literature review. Scand J Public Health. 2009 Mar 29 37(2):187–200.

    16. Nogoudalla M, Mahgoub AS. Assessment of communicable disease surveillance system in Gazera State, Sudan, 2009. Int J Infect Dis. 2012 Jun;16:e141.

    17. Budgell E, Cohen AL, McAnerney J, Walaza S, Madhi SA, Blumberg L, et al. Evaluation of Two Influenza Surveillance Systems in South Africa. Costa C, editor. PLoS One. 2015 Mar 30 10(3):e0120226.

    18. Yang X, Liu D, Wei K, Liu X, Meng L, Yu D, et al. Comparing the similarity and difference of three influenza surveillance systems in China. Sci Rep. 2018 Dec 1;8(1):1–7.

    19. Noreen N, Khan AW, Badar N, Khan FK, Khudaidad F, Khan NU, et al. Evaluation of Lab-based Influenza Surveillance System in Pakistan, 2017. Glob Biosecurity. 2019 Jul 3;1(2):24.