The last decade has seen a rapid increase in the use of telephone health services, which has changed the way we perceive and use health care in Australia. Use of the data collected from telephone services can identify important trends and assist in providing more timely public health action to mitigate morbidity and mortality associated with influenza-like illness (ILI) (1–4).
Healthdirect Australia (HDA) is a government-owned organisation that delivers a range of telephone and digital services in all states and territories of Australia except for Victoria and Queensland. One of the main services is the healthdirect helpline (HH), which receives over 700,000 calls per year. Data are collected on the callers’ personal demographics, location and symptoms, and these data are reported to HDA daily. Approximately 40% of callers to the HH do not visit any other healthcare provider (5), meaning the HH provides a unique insight into community-level disease and is a rapid indicator of syndrome incidence. Such insights are particularly true with syndromes such as influenza-like-illness or gastroenteritis, which are syndromes with high incidence but usually less severe symptoms that are self-limiting, since these patients are less likely to visit other healthcare providers than people who suffer from syndromes with more severe symptoms.
Australia has many sources of data for ILI disease surveillance, including sentinel general practitioner (GP) networks, Emergency Departments (EDs), ambulances, laboratories, specialist surveillance and research (6). However, surveillance systems that collect data on visits to GPs, EDs or ambulances currently do not capture details for every visit to that healthcare practitioner (6). In contrast, the HH has data on every call, and these data are reported to Healthdirect daily.
Studies in the United States (US), Canada, Ireland, and the United Kingdom (UK) have compared syndromic surveillance for ILI in telephone triage data with laboratory confirmed influenza (7–11), ILI in emergency departments (9,12) and ILI in GP presentations (7,8,10,11,13). Correlation has varied across these studies from 0.54 to 0.98. In Australia, one prior study has investigated the correlation between telephone data and laboratory confirmed influenza (10) and other studies have reported on the correlation between other aspects of the influenza surveillance system, for example, correlation between community syndromic surveillance, GPs and laboratory notifications (11,14).
This study describes the process and benefit of building a syndromic surveillance system using the HH to complement other data sources in the influenza surveillance system in Australia. Current research indicates that a combined surveillance approach with multiple data sources has the most value (15–17). We describe the data analysis methods required to test and validate the data, using ILI as an example. The aim of the paper is to determine the validity of HH data for ongoing disease surveillance and public health action.
The HH is a telephone-based health advice service in which a registered nurse provides health advice to participants over the phone. The service has been running since July 2008. Demographic details and characteristics of symptoms are captured in a de-identified, line-listed format. From this, we extracted the daily number of calls with symptoms that correspond to an ILI case definition. ILI cases were defined (in the clinical guidelines) as callers whose main symptoms were either “cough” or “colds and flu.” From the total number of daily encounters we calculated the proportion of calls relating to ILI per day.
The HH was compared with six different data sources listed in Table 1, and described in more detail elsewhere (2). These are: positive laboratory results reported to the National Notifiable Diseases Surveillance System (NNDSS), both in all of Australia and New South Wales (NSW); ED syndromic surveillance conducted in Western Australia (WA) and NSW; a self-reported community-based ILI surveillance system (FluTracking) and the Australian Sentinel Practices Research Network (ASPREN), which conducts syndromic ILI surveillance through sentinel GPs. The data were requested from each data source administrator, and each data source determined its own case definition for ILI. These data sources were chosen for comparison as they vary in terms of their specificity, representativeness, and timeliness. Specificity of the data sources varies, with high specificity in the laboratory data, which is confirmed through a positive influenza test, compared to low specificity in the FluTracking data which is self-reported. Some data sources are reported weekly (FluTracking and ASPREN), others daily (Emergency Departments), while the laboratory data often has a 1-2 week lag in reporting due to the time taken to run the test. Representativeness of the data sources vary, though none capture ILI activity in the whole population.
We estimated the percentage of the sampling frame captured by each data source. This allows us to quantify the effect of increased awareness of influenza leading to more testing or reporting and therefore more positive cases. The percentage of laboratory influenza tests that are captured by the NNDSS is approximately 100%, since there is mandatory reporting of all positive laboratory tests for influenza (6). However, there is no requirement to report the total number of tests requested, which means that the NNDSS only receives notifications of positive laboratory results. The NSW laboratory data source captures both how many positive tests and how many total tests were requested, allowing us to calculate the percentage of tests that were positive. We used this to estimate the total number of tests performed nationally. The percentage of ILI visits to GPs captured by the Australian Sentinel Practices Research Network (ASPREN) was estimated using the number of GPs reporting to ASPREN (18) and the total number of GPs (19). Estimates of the percentage of ILI-related visits to the emergency departments that are captured by the NSW and WA emergency department surveillance systems were calculated using the total number of records in the two data sources each year compared to the Australian Institute for Health and Wellbeing (AIHW) report of all hospital visits in these states (20). The response rate of 85% in FluTracking is reported by Sullivan et al. (21).
Data have been presented using a combination of tables and figures. The Spearman’s, Pearson’s and cross-correlation coefficients between the healthdirect helpline data and each data source are shown as both a table (Table 2) and a box and whisker plot (Figure 2). The comparison between the healthdirect helpline and the NSW Emergency department is also depicted as a plot showing the percentage of influenza-like illness calls or visits in each of these two data sources between 2008 and 2016.
The data were analysed using Spearman’s, Pearson’s and cross-correlation functions in R (version 3.4). The assumptions for the Pearson’s correlation function do not hold in all datasets and all years, whereas the Spearman’s assumptions are true. However, the cross-correlation function has no Spearman’s equivalent for non-linear but monotonic relationships, so we calculated the cross-correlation coefficient, assuming the Pearson assumptions hold, with data lagged by 0-10 weeks. The time lag in weeks that produced the best correlation was recorded for each year. Use of all three correlations allowed us to achieve three aims: to calculate the most accurate correlation assuming Pearson assumptions were not true; to calculate the cross-correlation coefficient; and to determine the reliability of the cross-correlation estimate by comparing the results of the Pearson and Spearman correlations. The study period was 2008 – 2016.
This project was granted ethical approval from the ANU Human Research Ethics Committee number 2016_599, NSW Population and Health Services Research Ethics Committee number 2016_07_646 and the Government of WA Department of Health Human Research Ethics Committee number 2016.39.
We analysed 5,115,050 calls to the HH between July 2008 and December 2016. The 0-5 age group had the largest number of calls and also the highest rate of influenza-related calls. In 2010-2017, the rate of influenza-related calls in the 0-5 age group was 12% of all calls, whereas in all other age groups, the percentage of influenza-related calls was below 5% of all calls. Despite this difference in rate of ILI calls by age, the shape of the yearly epidemic curve of ILI is the same in each age group, except in 2009. In 2009, the percentage of influenza-related calls increased in all age groups, with the largest increase in the 10-60 age groups, as seen in Figure 1. These age groups saw almost a three-fold increase in the percentage of influenza-related calls, with the percentage increasing from below 3% of calls to between 8.5% and 10% of calls.
The HH showed a very high correlation with other data sources of ILI. The strongest correlation was with emergency department surveillance both in NSW (mean correlation 0.96) and WA (mean correlation 0.92) (Figure 2). Figure 3 shows a plot of the NSW ED data and the HH together and visually demonstrates the degree of similarity between the shape of the influenza season, especially the timing and height of the peaks. Sentinel GP surveillance (ASPREN) and self-assessed community reporting (FluTracking) also had high correlation with the HH (mean correlation 0.85 and 0.79 respectively), though correlation with laboratory data varied (mean correlation 0.79 for both laboratory data sources, range in values from 0.49-0.95) (Table 2). Surveillance systems that had a syndromic definition (EDs, GPs and FluTracking) had the best correlation with the HH. The year with the highest correlation coefficients in the telephone data varied, though 2013 had the worst correlation for most data sources. By contrast, 2009 had both the best correlation in some data sources and the worst in others.
The best correlation between HH data and other surveillance data occurred at 0 weeks in most years for every data source except the two laboratory data sources and ASPREN. This indicates that an increase in helpline calls often coincides with an increase in visits to the ED and self-reported ILI on FluTracking. In contrast, the increase in calls to the HH typically occurs 1-2 weeks earlier than a positive influenza test. Similarly, an increase in calls to the HH occurred earlier than visits to the GP in 5 out of 8 years and at the same time in 3 years.
Percentage of calls which are influenza-related in each age group. This shows the comparison between 2009 percentages and 2019-2017 percentages.
The correlation between Healthdirect Australia telephone data and other data sources, 2008-2016.
The range in correlations for each data source is calculated using Spearman’s, Pearson’s and the cross-correlation function individually for each year of data. Data sources: GP Sentinel surveillance (ASPREN), 2008-2016; laboratory confirmed influenza in Australia (Aus Lab) / NNDSS, 2009 -2015; laboratory confirmed influenza in NSW (NSW Lab)/ NNDSS, 2009 - 2016; Emergency department surveillance in NSW (NSW ED), 2008 – 2015; Emergency department surveillance in WA (WA ED), 2008 – 2016; and FluTracking, 2008 – 2015. Time periods for these data sources are detailed in Table 1.
Percentage of influenza-like illness calls or visits: Healthdirect Australia helpline and NSW Emergency Department, 2008 - 2016.
The red line shows ILI-related visits to an NSW Emergency department, as a percentage of all visits in New South Wales from 2008 – 2016. The blue line shows ILI-related calls as a percentage of all calls to the HH in Australia from 2008 – 2016.
The high degree of correlation between the HH and other data sources shows that the HH data are consistent and trustworthy. The HH data could be used to contribute to routine disease surveillance by reporting elevated levels of ILI to emergency departments, public health departments, diseases surveillance groups, researchers, the general public and other interested parties. This information could be used for staffing, infection control, research, public health messaging, surveillance and other public health actions.
The year with the highest correlation coefficients in the telephone data varied, though 2013 had the worst correlation for most data sources. This could be due to the nature of the correlation function, which is strongly influenced by its tails. For example, 2009 provided the best fit in two data sources but the worst fit in one. Data sources matched at the peak in 2009, which is significantly larger than the mean and disproportionately increased or decreased the estimate of correlation. By contrast, the helpline data for 2013 had a flat or double peak (Figure 3), which would lead to lower correlation due to the nature of the correlation function. This flat peak was seen across all data sources.
This is the first time that a national telephone health line in Australia has reported on its contribution to early detection of ILI compared to emergency department and sentinel GP surveillance. This serves to establish some key characteristics of the HH data, including the degree of correlation and the time lag between the HH and other data sources, before the 2020 COVID-19 pandemic. In 2020, the percentage of calls that were influenza-related rose from an average of around 5% to a peak of over 20% in mid-March (22). The National Coronavirus Helpline was established on March 16, 2020, and subsequent to that date all callers with concerns about COVID-19 were diverted to this helpline. Influenza-related calls quickly dropped to below average and stayed below average until November 2020.
Since cough and fever are symptoms of both ILI and COVID-19, the increase in influenza-related calls in January – March of 2020 were likely due to the increasing media attention on the COVID-19 pandemic. This would have caused more people with cold or cough symptoms to call the helpline when they usually would not have been concerned enough to call. There were likely also people calling for general COVID-19 related advice that were being captured in the ILI case definition. This overlap of symptoms between diseases also likely led to the decrease of influenza-related calls to below the average after the Coronavirus helpline was established. Callers with cold or cough symptoms likely called the Coronavirus helpline rather than the usual HH helpline, thus decreasing the amount of influenza-related calls to the HH. Note we would not expect to see a large increase in COVID-related calls to the HH during the Victorian COVID-19 outbreak in July – August of 2020 as the HH does not operate in Victoria.
Other studies have compared telephone triage data to laboratory confirmed influenza in the US (7–10), the UK (11) and Australia (23), with correlations ranging from 0.54 (23) to 0.65 (7,8) and 0.82 (9). The results in these studies are consistent with the correlation of 0.79 between the HH and the NNDSS. The lower correlation for the previous Australian study is likely due to Fitzgerald’s method of finding the correlation of all years combined instead of each year individually (23), which would significantly lower the correlation.
Further studies have compared telephone triage data with Emergency Department surveillance in Canada (correlation of 0.98) (12) and the US (correlation of 0.9) (9). These results are consistent with our study in two aspects: firstly, that there are higher correlations between telephone triage and emergency departments than telephone triage and laboratory confirmed influenza, and secondly, the correlations are similar to this study’s result of 0.96 in NSW EDs and 0.91 in WA EDs. Finally, researchers in Ireland (13), the US (7,8,10) and the UK (11) have compared telephone triage data to GP presentations, with correlations varying from 0.74 (7,8) to 0.80 (13), which is slightly lower than our average correlation of 0.85. This could be due to our study using more years of data (8-9 years compared with 1 in the US, 5 in the UK and 7 in Ireland) or the ASPREN network containing more GPs (ASPREN has 175 GPs, the US study had 753 GPs, the UK study had 50 GPs and the Ireland study had 54).
Four studies also investigated a cross-correlation coefficient to determine the optimal lag for correlation between telephone triage and other ILI data sources (7,8,12,23). A lag of 0 weeks was found to have the highest correlation between the telephone triage data and other data sources in the US (7,8), Australia (23) and the UK (12). On the other hand, a different study in the US found the best correlation with the telephone triage data leading GP and laboratory data by 1 week (10). Other studies used different methods of estimating the time lag between data sources. One study compared the peak of the influenza season across data sources (13), finding that, on average, telephone data led other data sources by 1 week. A further study calculated an alert threshold and found that the telephone triage data triggered an alert between 1-3 weeks earlier than the comparison data source (11). These results match our findings of a lag of 0 weeks in most years, excepting some years or data sources when telephone triage data lead other data sources by 1-3 weeks.
The variation in correlations between datasets could be due to several factors. The lower correlation between the HH and FluTracking and GPs compared to EDs could be due to the smaller sample sizes of these data sources. The lower correlation in the laboratory data could be due to the difference between a syndromic definition for influenza-like illness and laboratory confirmation of influenza.
The HH data has very limited information about symptoms of a case and only describes the main symptom of the caller. Thus, cases could be misclassified, with both callers without ILI identified as cases and callers with ILI not identified. As we were unable to contact callers, we could not obtain more information or test the HH diagnosis against a gold standard. We believe that this misclassification will occur in both directions, however it is possible that it may vary seasonally as interviewers may be more likely to classify a caller to an ILI-related symptom than another symptom during the flu season. For example, Healthdirect Australia marketing targeted at influenza-related symptoms, retraining of nurses in use of the clinical algorithms, and changes in the automatically suggested algorithms would affect the ILI-related activity in the HH, and consequently cause a weaker fit.
Limitations of our comparison datasets include lack of data on age, different numbers of records, and different temporal and geographic variables. To account for changes in the number of records over time and the size of the datasets, we compared the proportion of ILI-related calls instead of the raw number and matched the HH data with the temporal and geographical aspects of each dataset as closely as possible. The inability to age-standardise our datasets may have led to bias, as the HH dataset has a greater proportion of callers under 5 than in the Australian population. Both the Spearman’s and Pearson’s statistical tests are invariant under transformations of scale and so should not be affected where incidence curves by age show the same timing.
To create a syndromic surveillance system that is useful to stakeholders we also convened an expert group consisting of senior members from the Commonwealth and jurisdictional health departments. The group’s role included offering guidance to increase the acceptability of the system and helping to facilitate access to the six different data sources which are detailed in Table 1.
We would like to acknowledge the contribution of the following people to this paper: members of the HATSS advisory group, consisting of Melissa Irwin (NSW Ministry of Health), Robin Gilmour (NSW Ministry of Health), Christina Bareja (Commonwealth Department of Health), Rachael Corvisy (Commonwealth Department of Health), Donna Mak (WA Department of Health), and the data custodians of our data sources: Craig Dalton (FluTracking), Andrew Puljic (WA Emergency Department Data Collection), Melissa Irwin (NSW Public Health Real-Time Emergency Department Surveillance System), Nigel Stocks (ASPREN), Christina Bareja (National Notifiable Diseases Surveillance System), Robin Gilmour (NSW Laboratory Notifications System).
None.
1. Harcourt SE, Morbey RA, Loveridge P, Carrilho L, Baynham D, Povey E, et al. Developing and validating a new national remote health advice syndromic surveillance system in England. J Public Health (Oxf). 2017 Mar 1;39(1):184–92.
2. Cooper DL, Smith GE, Regan M, Large S, Groenewegen PP. Tracking the spatial diffusion of influenza and norovirus using telehealth data: a spatiotemporal analysis of syndromic data. BMC Med. 2008 Jun 26;6:16.
3. Chu A, Savage R, Whelan M, Rosella L, Crowcroft N, Willison D, et al. Assessing the Relative Timeliness of Ontario’s Syndromic Surveillance Systems for Early Detection of the 2009 Influenza H1N1 Pandemic Waves. Canadian Journal of Public Health, Revue Canadienne de Sante Publique. 2013;104(4):340–7.
4. Rolland E, Moore KM, Robinson VA, McGuinness D. Using Ontario’s “Telehealth” health telephone helpline as an early-warning system: a study protocol. BMC Health Serv Res. 2006 Feb 15;6:10.
5. Tran D, Gibson A, Randall D, Havard A, Jorm L. Mapping the outcome of calls to the healthdirect helpline. 2015;(June).
6. Gibeny KB, Cheng AC, Hall R, Leder K. Australia’s National Notifiable Diseases Surveillance System 1991-2011: expanding, adapting and improving. Epidemiology and infection. 2017;145(5):1006–17.
7. Yih WK, Teates KS, Abrams A, Kleinman K, Kulldorff M, Pinner R, et al. Telephone triage service data for detection of influenza-like illness. PLoS ONE. 2009 Apr 17;4(4):e5260.
8. Yih WK, Teates KS, Abrams A, Kleinman K, Pinner R, Harmon R, et al. Nurse call data for detection of influenza-like illness. 2016;(March).
9. Lucero-Obusan C, Winston CA, Schirmer PL, Oda G, Holodniy M. Enhanced influenza surveillance using telephone triage and electronic syndromic surveillance in the Department of Veterans Affairs, 2011-2015. Public Health Reports. 2017;132:16S-22S.
10. Espino JU, Hogan WR, Wagner MM. Telephone Triage: A timely data source for surveillance of influenza-like diseases. AMIA 2003 Symposium Proceedings. 2003;215–9.
11. Cooper DL, Verlander NQ, Elliot AJ, Joseph CA, Smith GE. Can syndromic thresholds provide early warning of national influenza outbreaks? J Public Health (Oxf). 2009 Mar;31(1):17–25.
12. van-Dijk A, Aramini J, Edge G, Moore KM. Real-time surveillance for respiratory disease outbreaks, Ontario, Canada. Emerging Infect Dis. 2009 May;15(5):799–801.
13. Brabazon ED, Carton MW, Murray C, Hederman L, Bedford D. General practice out-of-hours service in Ireland provides a new source of syndromic surveillance data on influenza. Euro Surveill. 2010 Aug 5;15(31).
14. Parrella A, Dalton CB, Pearce R, Litt JCB, Stocks N. ASPREN surveillance system for influenza-like illness - A comparison with FluTracking and the National Notifiable Diseases Surveillance System. Aust Fam Physician. 2009 Nov;38(11):932–6.
15. Owen R, Barr IG, Pengilley A, Liu C, Paterson B, Kaczmarek M, et al. Annual report of the National Influenza Surveillance Scheme, 2007. Commun Dis Intell Q Rep. 2008 Jun;32(2):208–26.
16. Romanowska M, Nowak I, Rybicka K, Brydak LB. The introduction of the SENTINEL influenza surveillance system in Poland - experiences and lessons learned from the first three epidemic seasons. Eurosurveillance. 2008 Feb 21;13(8):7–8.
17. Clothier HJ, Fielding JE, Kelly HA. An evaluation of the Australian Sentinel Practice Research Network (ASPREN) surveillance for influenza-like illness. Commun Dis Intell Q Rep. 2005;29(3):231–47.
18. Surveillance NGS. ASPREN update. 2016.
19. Britt H, Miller GC, Henderson J, Charles J, Valenti L, Harrison C, et al. A decade of Australian general practice activity 2003–04 to 2012–13. General practice series no. 34. Sydney: Sydney University Press,. 2013;(November).
20. Australian Institute of Health and Welfare. Emergency department care 2014-15. 2015.
21. Sullivan SG, Franklin LJ, Raupach J, Pennington K, Bareja C, Kluyver RD. A brief overview of influenza surveillance systems in Australia, 2015. Communicable diseases intelligence. 2016;347–51.
22. Department of Health, Australia. Australian influenza surveillance report No 17 - 16 November to 29 November 2020 [Internet]. Department of Health, Australia; 2020 Dec [cited 2021 May 26]. Report No.: 17. Available from: https://www1.health.gov.au/internet/main/publishing.nsf/Content/9C53D5FF0B103751CA25862600034B44/$File/flu-17-2020.pdf
23. Fitzgerald T. Evaluation of the healthdirect Australia Influenza-like Illness Surveillance System and analysis of the healthdirect Australia Influenza-like Illness Data. 2015.