Public health surveillance is the systematic collection, analysis and interpretation of health-related data that is required for the planning, implementation and evaluation of public health practice. It can:

  1. Function as an early warning system for imminent public health emergencies;
  2. Provide evidence of the impact of an intervention by tracking the progress towards specified goals; and
  3. Monitor the epidemiology of health problems to help prioritize and inform public health policy and strategies (1, 2).

Influenza is an infectious respiratory illness that ranges from mild to severe and sometimes fatal illness (3). Influenza affects 5-10% of adults and 20-30% of children worldwide each year, causing 3-5 million cases and 650,000 deaths annually. Influenza A (H1N1) virus caused the last global influenza pandemic in 2009 (4).

WHO conducts global influenza surveillance through the Global Influenza Surveillance and Response System (GISRS), which consists of 143 institutions in 113 WHO member states, including national influenza centres, 6 WHO collaborating centres and 13 WHO H5 reference laboratories. GISRS is primarily responsible for monitoring the mutations and trends in influenza viruses, providing updated recommendations for laboratory diagnostics, vaccines and antiviral susceptibility, risk assessment, and serving as a platform for the global alert of the emergence of influenza viruses with pandemic potential (5).

Pakistan has the sixth largest population in the world and is in south Asia, with a tropical to temperate climate. Influenza is a common illness; it has as high as a 20%–30% attack rate in preschool children. Since 2004, influenza surveillance in Pakistan has been conducted by the Pakistan National Institutes of Health (NIH) in conjunction with the US Centres for Disease Control and Prevention (CDC). Sporadic influenza surveillance is done by the National Influenza Centre (NIC) based at NIH, Islamabad. In 2007, with the support of CDC, a national database was established for tracking influenza and hepatitis activity. This aimed to strengthen the national preparedness plans, training and capacity building (6).

In 2008, a sentinel laboratory-based influenza surveillance system was established in collaboration with CDC to assess influenza-like-illness (ILI) and severe acute respiratory illness (SARI) trends, monitor the predominant circulating strains of influenza viruses, characterize the clinical manifestations of influenza, and identify vulnerable high-risk population groups. Lab-based surveillance systems conduct weekly sampling and reporting, using real-time RT-PCR assay to type and sub-type samples. Previously, Pakistan lacked the laboratory capacity for performing influenza diagnostics. However, the National Influenza Lab Based Surveillance Project (NILSP) now provides conventional and advanced diagnostic facilities for influenza and contributes influenza virus isolates to the Global Influenza Surveillance Network for influenza vaccine strain selection. Surveillance for ILI and SARI cases is done at the designated provincial sentinel sites (Figure 1). Influenza specimens from outbreaks are also received from both public and private major hospitals in all cities. The lab-based influenza surveillance system is comprised of seven sentinel sites located in tertiary health facilities in provincial headquarters. Sentinel site location is based on tertiary healthcare facilities, geographic distribution, population density, catchment area, patient turnover, technical expertise, commitment and motivation of staff. Fully equipped labs are available at each sentinel site for PCR testing. A microbiologist, one lab technician and a medical technologist are available at each sentinel site. Staff at sentinel sites are provincial government employees working on the surveillance system on a monthly incentive basis.

We conducted this evaluative study to obtain a comprehensive summary of the operational working of the system through information provided by system representatives and to identify the key strengths, weaknesses and limitations of the system.

Figure 1 

Location of Sentinel sites and Central Lab of Influenza Surveillance System

Figure 2 

Flow chart of lab-based influenza surveillance system, Pakistan*

*(NIH-National institute of health, WHO-World Health Organization, GISRS- Global Influenza Surveillance and Response System, CDC-Centre for Disease Control Peshawar-Hayatabad Medical Complex, DHQ Gilgit-District Headquarter hospital, FGPC -Federal Government polyclinic Islamabad, AMIC-Abbas Institute and Medical Centre Muzzafarabad.BMC-Bolan Medical College Quetta)


To evaluate the influenza sentinel surveillance system in Pakistan.


We conducted this evaluation study from April to July 2017 at NIC, with the aim to evaluate the influenza surveillance system at the national level according to the updated guidelines developed by the CDC for the evaluation of public health surveillance systems (7). We adhered to the following steps:

  1. Defining the purpose of the evaluation.
  2. Identification & engagement of stakeholders in the evaluation process.
  3. Description of the system in terms of: a) purpose of the system, b) operational arrangements, and c) resources used to operate the system
  4. Gather reliable evidence regarding the performance of the system and describe the system in terms of standard attributes with the CDC guidelines.
  5. Conclusions and recommendations.

Evaluation procedure

We developed a semi-structured questionnaire addressing purpose, objective, performance and indicators accessing the system attributes in accordance with CDC guidelines (7). Ten attributes, as described in the updated guidelines, were evaluated in the system: usefulness, simplicity, flexibility, data quality, predictive value, sensitivity, timeliness, acceptability, representativeness and stability

Qualitative and quantitative system attributes were evaluated as per the framework provided by the Updated CDC Guidelines for Evaluation on Public Health Surveillance Systems, 2001 (7).

We reviewed documents related to the system (strategic plan, guidelines, manuals, annual reports and databases) and conducted consultations and interviews with stakeholders. In addition, we interviewed key informants, including microbiologists, lab technician pathologists, virologists and data entry officers from each sentinel site.

We conducted a desk review of relevant system records before interviewing managers. We approached the heads of virology and immunology departments to obtain the necessary information. We also interviewed the operational managers of other relevant departments, such as public health laboratories, field epidemiology and disease surveillance division, and all sentinel sites. We calculated selected indices such as case definition uniformity, data flow and data quality, as described in the guidelines [5].

We used the following formulas to calculate sensitivity and positive predictive value.

Sensitivity = (No of Reported Cases)/(No. of Expected Cases* ) × 100%

*Expected cases include those cases which are not picked by the surveillance system

PVP = (No. of positive cases)/ (No. of reported cases**) × 100%

**Reported cases are all the cases from whom samples are collected

Description of surveillance system

Public health importance

Community influenza surveillance is a key mechanism for defining the beginning and end of the influenza season, estimating the burden of disease in the community and identifying unusual epidemiologic trends. This surveillance data is used for planning at federal level. The benefit of the system is two-fold: epidemiological in terms of trends identified, and clinical, as the laboratory component helps in timely diagnosis of influenza.

Objectives of surveillance system

The objectives of the program are to assess the disease burden, monitor trends of ILI and SARI cases, determine the predominant circulating virus strain, serotype circulating influenza strains, and detect any potential pandemic variant. Other objectives include contributing to global surveillance, providing scientific evidence for preventive measures, strengthening emergency preparedness/response and updating regional strains for vaccine development.

Case definitions

Case definitions used are simple, uniform and consistent throughout the program. They are adapted from WHO case definitions. Influenza like illness (ILI) was defined as an acute respiratory infection with measured fever of ≥ 38 °C and cough (onset within the last 10 days). Severe Acute Respiratory Infection (SARI) was defined as: an acute respiratory infection with measured fever of ≥ 38°C and cough (onset within the last 10 days) requiring hospitalization.

Operations of Surveillance System

  • Standard forms, including history of patient, are filled at sentinel sites and various health facilities in the catchment area of sentinel sites.
  • Samples from suspected cases are collected from the outpatient departments and inpatient wards of most public and private health facilities. In peak season, over 30 samples per week are collected. The minimum number of samples is 10 per week. Then samples are transferred to and processed at sentinel site labs. Shipments are sent biweekly or monthly.
  • Follow up is done by mail and email. If responses are not received weekly, sites are followed up by phone.
  • Data collected through standardized forms are entered into SPSS software by trained staff. Incomplete forms are filled in by staff in the central lab from hard copies of forms. Any missing information is obtained from staff at sentinel sites or from patients by phone call. Data analysis is done in SPSS. Annual reports are generated and shared with CDC.
  • All reported cases are confirmed by laboratories at sentinel sites using real-time PCR and viral culture. Positive samples are sent to the central lab for quality control. The total numbers of received samples and influenza positive samples by subtype are recorded.
  • The system identifies circulating influenza strains to help develop seasonal vaccine formulations.
  • There is a backup mechanism for storing data and records. Records are stored in both hard copy and electronic format. The security of the labs was enhanced by restricted access.
  • Private sector engagement with sentinel sites and the central lab is good. Diagnostic tests are performed free of charge for both the public and private sectors.
  • Changes in case definitions can occur in some situations, such as pandemics. Standard data forms for case definitions are used from sources such as the WHO.
  • Data is shared with partners, Including WHO, sentinel sites, and provincial and executive district health offices (EDHO)
  • The catchment population of surveillance is the population living in the catchment area of tertiary care hospitals.
  • The surveillance system is 100% donor funded (CDC funding).


Results of this evaluation were based on system attributes according to the CDC guidelines for evaluating surveillance systems. The system attributes were evaluated quantitatively as well as qualitatively.

Simplicity (good)

Case definitions were adopted from WHO and were simple, uniform and consistent. The system was simple and easy to understand. Staff were trained in data collection, management of data and dissemination of findings to authorities. Data from sentinel sites includes hard copies of forms and samples from reported cases. The system collects information on demographics, clinical diagnosis, signs and symptoms, comorbidities and risk information for influenza cases. The flow of data and information from sentinel sites to the national lab is efficient. Information is shared with WHO, CDC, and district and provincial officials if action is required. There are regular staff trainings for use of the information and data collection tools. The system is not difficult to operate and uses standard data formats provided by WHO/CDC. Analysis is done in SPSS, which is a user friendly software. The level of integration with other organizations like NARC is minimal, making it easy to operate and manage.

Flexibility (average)

The system has limited flexibility to integrate with other systems. The system has the capacity to be expanded to accommodate new health related events and other respiratory viruses, but additional time and resources would be required.

Data Quality (good)

Forms were randomly selected from two sentinel sites, and 80% were fully completed. About 20% of forms have missing information on radiology, comorbidities, admission details and/or outcomes. Data was assessed as complete and accurate. Samples were collected from persons based on their clinical features, and test results of RT-PCR and viral culture are reported. Paper copies, as well as electronic based data management system (SPSS), are present. Completeness and validity of recorded data reflect good data quality. Data analysis is done at federal/central levels on weekly and monthly basis and the results are uploaded on Flu Net. Data management processes regarding data collection, entry, editing, analysis and feedback were good.

Acceptability (good)

Private hospitals and private labs have good participation in the lab-based influenza surveillance system. All stakeholders participate in the system, but contribution from different stakeholders is not equal. There is a high rate of samples and case reports from private health facilities directly to the central lab. The system is readily accepted by all affiliated healthcare providers.

Sensitivity (low)

Sensitivity is defined as the ability of the surveillance system to identify the true cases of influenza in the country (sensitivity = positive cases/expected cases × 100%). The case definition used for reporting suspected influenza cases can identify almost all the true positive cases by lab detection. Sensitivity is supposed to be low, as the system is sentinel-based and therefore not representative of the true population. People not visiting these sites are not reported and therefore missed. The measurement of the sensitivity of the surveillance system requires the true frequency of the disease in the population under surveillance, and as it is a sentinel site-based surveillance, it is beyond the scope of this evaluative study.

All respiratory swab samples (n = 1497) were tested by real-time RT-PCR assay according to CDC protocol. Of these, 30.4 % were positive for influenza viruses. Out of positive samples, only a few representative samples were grown on MDCK cell lines (virus cultivation) to have isolates for further testing. The performance of both assays cannot be compared as all samples are not cultivated

Figures for existing cases were shared by the data records of central lab Incharge. In total, 1497 samples were received in 2016, and 455 positive cases were detected by the system.

Positive Predictive Value (PVP) = (total samples received)/(total samples positive) = 455/1497 × 100% = 30.39 %

Representativeness (poor)

The representativeness of the lab-based influenza surveillance system is poor because it is sentinel surveillance, not country wide, thus only covering a limited catchment population. People not visiting these sites are therefore not reported and missed. Among reported cases, males accounted for 54%, people aged at 1-12 years accounted for 66%.

Timeliness (good)

The timeliness of the system is good. It was quantified by measuring the time required for sample collection and transfer, the processing of forms, and the availability of laboratory results. Data is generated on a weekly and monthly basis. The analyzed data/reports are shared at provincial and federal level within the required time and submitted to CDC and WHO (Flu Net). CDC influenza experts visit the reference lab twice a year. Outbreak detection and response was observed to be timely and accurate.

Stability (average)

The system is donor funded. Labs were well equipped with computers and other logistics were available at sentinel sites. The system responds to cases in time and is reliable. The system is available when needed and is stable. Uninterrupted working electronic instruments are available. The system can collect and manage data without failure. Monitoring and evaluation of the system is done on a yearly basis. All resources are dependent on CDC funding.

Usefulness (good)

Data from all sentinel sites are analyzed at a national reference lab. The system provides data to the country influenza database. All virological and epidemiology data is updated on WHO GISRS website on a weekly basis (Flu Net). The system provides data on risk factors, and thus helps to make strategic decisions on prevention and control of disease. In addition, serial data can show seasonal and yearly trends of influenza. The system also provides support for outbreaks by timely reporting and continuous feedback from the national lab. Major activities of the system include development of surveillance, laboratory protocols and management guidelines, submission of representative/un-typable samples to WHO for vaccine strain selection, capacity building of human resources through training, quality assurance, laboratory management, enhanced biosafety & biosecurity, improved data entry & management capability.


The results of this study show that the influenza surveillance system provides timely information through weekly and quarterly reporting. A similar evaluative study conducted in tropical countries of Asia and Africa in coordination with CDC and WHO emphasized the importance and necessity of influenza surveillance systems and influenza laboratories to address the challenges and pandemic threat posed by influenza (8). A study conducted in 2015 describing the Australian influenza surveillance system illustrated the importance of timely syndromic and laboratory surveillance of influenza in decreasing the burden of influenza in the Australian community. This allows decision makers to have access to the best available and timely information on circulating strains in relation to vaccine effectiveness (9). In our study, we found that the timeliness of the influenza surveillance system was good. Information shared by surveillance system is used for policy and planning decisions at national and international level. The system has a high degree of acceptability among stakeholders. The public health impact of the surveillance system on community is twofold, both epidemiological and clinical. The lab-based system has a very strong lab component, helps in timely diagnosis, and contributes to vaccine formulation by providing information on circulating strains. Laboratory-confirmed surveillance is beneficial and an effective way to monitor influenza transmission and seasonality. A similar study conducted in China to evaluate influenza surveillance emphasized laboratory-confirmed surveillance as a tool for developing influenza early warning systems (10).

Surveillance data is used for planning at the provincial and federal levels. The provincial coordination network with ownership from federal government is a key feature of the surveillance system. The system contributes to the web-based reporting system Flu Net (northern hemisphere), which is used as a basis for the composition of influenza vaccine. In a study conducted in Alaska, electronic health records were utilized for effective influenza surveillance after the 2009 influenza Pandemic (11).

The surveillance project is primarily foreign funded by CDC, thus having poor sustainability. Sentinel surveillance is not representative of the whole population, as sentinel sites are in major cities of Pakistan. An evaluation study of the surveillance system in South Africa found that a major weakness of the system was that sentinel sites were inaccessible to a proportion of the population (12).


Community influenza surveillance serves as key mechanism for defining the beginning and end of influenza season, estimating the burden of disease in the community, identifying unusual epidemiologic trends, and assisting with vaccine formulation. Given that the system conducts sentinel surveillance covering specific catchment areas, it is not representative of the total population. Sustainability of funds for running the laboratories is an issue. For example, supply of reagents is not reliable or regular, and is totally donor funded. Provincial ownership is lacking. Retention of qualified & trained staff is an issue due to lack of financial incentives.


The surveillance system needs expansion, with inclusion of all the public and private health care facilities to make it truly representative of the entire population. Ownership by provincial government to manage sentinel sites’ laboratories is needed. Strengthening of public-private partnership and involving private sector data into surveillance systems will increase participation and representativeness of the system. Financial and career incentives need to be introduced to improve motivation and retention of trained staff. Capacity building and enhancement of skills can be achieved by regular training of staff and frequent quality checks through field visits. More efforts are required to ensure the sustainability and stability of system.


Author acknowledges the support of Field epidemiology and Laboratory Programme and FEDSD NIH and virology Department (NILSP) in this evaluative study.


1. WHO Public health surveillance (2015) [Feb 2019]. Available from:

2. Bernard C.K. Choi. The Past, Present, and Future of Public Health Surveillance. Hindawi Scientific. Volume 2012, Article ID 875253, 26 pages.

3. CDC. Types of Influenza Viruses - Seasonal Influenza (Flu). U.S. Centres for Disease Control and Protection (CDC). 2014.

4. WHO. Influenza (Seasonal). Bulletin of the World Health Organization. 2013.

5. Katz J. Global influenza surveillance and response system (GISRS). FDA presentation. 2018

6. Nisar N, Badar N, Aamir UB, Aamir BM, Kazi SSZ, Zaidi. The Substantial Burden of Influenza in a Tertiary Pediatric Settings in Pakistan: 2008–2014. Open Forum Infectious Diseases, 2015, 2(suppl_1):1721.

7. Robert R. German, Updated Guidelines for Evaluating Public Health Surveillance Systems.MMWR, July 27, 2001 / 50(RR13);1-35.

8. Melvin Sanicas. A review of the surveillance systems of influenza in selected countries in the tropical region. Pan African Medical Journal. 2014; 19:121. DOI: 10.11604/pamj.2014.19.121.4280

9. Sullivan SG, Franklin LJ, Raupach J, Pennington K, Bareja C, Kluyver R,et al.A brief overview of influenza surveillance systems in Australia, 2015.Commun Dis Intell Q Rep. 2016;40(3):E351-5.

10. Xiaoting Yang, Dongpeng Liu, Kongfu Wei, Comparing the similarity and difference of three influenza surveillance systems in China. Scientific reports (2018) 8:2840. DOI:10.1038/s41598-018-21059-9

11. Keck JW, Redd JT, Cheek JE, Layne LJ, Groom A V, Kitka S, et al. Influenza surveillance using electronic health records in the American Indian and Alaska Native population. 2014;21(1):132–8.

12. Budgell E, Cohen AL, Mcanerney J, Walaza S, Madhi SA, Blumberg L, et al. Evaluation of Two Influenza Surveillance Systems in South Africa. PloS One.2015;10(3): e0120226.