Expert Perspectives on Outbreak Investigation Training: A Quality Improvement Exercise

Aims: This study aimed to achieve a better understanding of factors contributing to effective training in outbreak investigation, including the characteristics of effective trainees, supervisors, and training activities. Methods: We designed a semi-structured interview aimed at exploring factors in Tynjala’s 3P model related to Presage (which we defined as the qualities of a good FETP trainee), Process (activities that support trainees in attaining competence as outbreak investigators, including effective supervision) and Product (qualities of a good outbreak investigation and investigator). These topics were explored principally within the context of the Australian Master of Philosophy in Applied Epidemiology (MAE) Program. Deductive content analysis was conducted on interview transcripts to construct qualitative themes relating to these topics based on interview data. Results: The principal themes identified as the key components in training a competent outbreak investigator related to: personal qualities and interpersonal skills, alongside prior qualifications; elements of effective supervision including technical and soft skills, flexibility, and personal compatibility with the trainee; and activities to best support trainee development including classroom teaching in preparation for practical experience, and the ideal approach and quantity of practical exercises. Conclusions: This study identified that effective outbreak investigators possess a broad range of skills and knowledge. These span from proficiency at epidemiological tasks to interpersonal communication, underpinned by personal attributes such as perseverance and curiosity. Similarly, supervisors should ideally demonstrate a passion for teaching and investment in the holistic performance and wellbeing of trainees, providing flexibility to adapt to the needs and learning styles of each trainee. Training activities should support trainees to learn theoretical content that complements practical experience, and develop mastery and confidence by having incrementally increased responsibility and independence across successive practical scenarios. The current MAE curriculum focuses heavily on technical content and may benefit from a greater focus on soft skills. The selection criteria for the MAE program has also been altered to reflect the skills identified here.


Introduction
Under the International Health Regulations (IHR) 2005, every country is required to have or establish the capacity to prevent, detect, assess, report and control acute public health events (1). Field epidemiologists are a key component of the workforce required to achieve this capacity. Field epidemiology training programs (FETP), originally based on the Centers for Disease Control and Prevention (CDC) Epidemic Intelligence Service (EIS) (2), are accepted internationally as models for training field epidemiologists and therefore strengthening global health security (3).
FETPs are workplace based learning programs, where trainees spend the majority of their time in a field placement conducting projects to achieve FETP competencies. This placement typically occurs in a Ministry of Health or national public health institute, where trainees build their own learning capacity while also providing service to their field placement organization (4). Learning in the workplace is supported by participation in educational courses where trainees learn key epidemiological skills. A required competency for FETPs is outbreak investigation and response (5), with FETP residents and graduates being key in applying these skills in response to significant acute public health events around the world [6,7,8].
The FETP standard core curriculum, developed by CDC and used or adapted by multiple FETPs worldwide, provides detailed instructional goals and learning objectives related to outbreak investigation (9). The Australian FETP program requires trainees to demonstrate the following core competencies: evaluating a public health surveillance system, analysing public health data, responding to an acute public health event/investigating outbreaks, and conducting an epidemiological study (10). This curriculum is largely oriented towards teaching technical skills and knowledge (11).
However, as FETP trainees spend most of their time in their field placement working in an interactive and dynamic context, it is likely that the majority of their learning occurs in this environment (12). Thriving in this setting requires a variety of other skills and abilities, including teamwork, interpersonal and communication skills, though these skills may not traditionally be emphasized within FETP curricula. There is relatively little empirical evidence as to how to support a trainee to develop competence in outbreak investigation, either within the classroom or the field placement. Evaluations of FETPs have tended to focus on metrics such as enrolment rates, completion rates, subsequent employment and publications of trainees, degree of integration of programs with government public health infrastructure, and trainee self-evaluation of competencies (4,13,14). Where formal or informal evaluations have considered teaching and learning, these have generally focused on appraisal of classroom teaching, technical content and supervisor performance, and have used trainee satisfaction as the main outcome factor (15).
The Australian Master of Philosophy in Applied Epidemiology Program, colloquially known as the MAE Program, is Australia's FETP. As part of ongoing programmatic quality improvement, we aimed to determine how trainees can be best equipped with the right skills, abilities and knowledge to function as effective field epidemiologists on graduation. This was particularly timely as the MAE Program has undergone a number of structural changes over recent years (10), and has recently commenced a funded program to intake trainees from a designated number of Association of Southeast Asian Nations (ASEAN) countries. We were particularly interested in training in outbreak investigation due to its visibility, complexity and particular importance for global health security, which is especially relevant today in the context of the COVID-19 pandemic.
As part of a larger project focused on addressing systems gaps in epidemic control and building capacity in health systems research, we undertook this study to achieve a better understanding of factors contributing to effective training in outbreak investigation.

Study Design
This was designed as a qualitative case study using semi-structured interviews to identify what experienced practitioners consider to be the key components in training a competent outbreak investigator, and how this competence could be achieved. Interviews were conducted with experts in the field to obtain qualitative data on the topics we were interested in, while allowing participants to broaden discussion beyond these areas.

Study Team
The study investigation team was comprised of MAE Program alumni, two of whom were employed as program teaching staff at the time of interviews. Interviews were conducted by MAE Program alumnus CM, who was employed as a research assistant for the purposes of this study.

Theoretical Framework
We framed this project using Tynjala's 3P model for workplace learning (16). Broadly, this model can be used as a tool for understanding how learning occurs in the workplace, by breaking this down into factors related to Presage (learner and contextual factors), Process (activities supporting learning) and Product (learning outcomes).

Interview Content
We designed a semi-structured interview guide aimed at exploring factors related to Presage (which we defined as qualities of a good FETP trainee), Process (activities that support trainees in attaining competence as outbreak investigators, including effective supervision) and Product (qualities of a good outbreak investigation and investigator). These topics were explored principally within the context of the Australian MAE Program.
Interview questions were open-ended and covered a variety of topic domains, including factors that distinguish good outbreak investigators and trainees, factors that distinguish effective supervisors, and the variety and quantity of training activities that best support trainee growth and development. Further information is available in the interview guide (Appendix A).

Sampling and Participants
We used purposive sampling to identify individuals in Australia, through professional networks of the study authors. Participants were experts in the fields of communicable disease control, applied epidemiology, outbreak investigation, and supervision of trainees in these areas. The selection criteria required that interview participants had at least 10 years of experience as a field placement supervisor for MAE trainees (i.e. not MAE academic supervisors who are University employees and who contribute to the overall management and delivery of the MAE Program, but who do not supervise the MAE trainees day to day in their workplace). MAE field supervisors are provided with an initial orientation to the MAE Program, which outlines these details, the required coursework and field work competencies, as well as classroom training content. Interviews were conducted by a member of the research team (author CM) between November 2016 and February 2017. After the initially planned 10 interviews, data collection was ceased as the interviewer and project managers judged that data saturation was reached, with few new emerging themes noted in later interviews.

Analysis
Interviews were recorded verbatim and recordings were professionally transcribed. We then used deductive content analysis to analyse the responses (17), building a thematic framework informed by the structure of Tynjala's 3P model.

Ethics
Ethics approval for this study was provided by the Australian National University (protocol 2016/420) and the University of New South Wales Human Research Ethics Committees (protocol 15571). All participants were provided with an information sheet regarding the study and provided written informed consent, or verbal consent when interviews were conducted by phone.

Results
We interviewed ten participants from several jurisdictions across Australia. All participants held senior positions in their organisations, with roles including senior epidemiologists, directors of public health units and other experienced investigators within communicable disease control. Seven of the participants were public health physicians, and six of the participants were graduates of the Australian FETP or an international FETP (Table 1). Table 1. Summary description of respondent characteristics (n=10) As detailed in Box 1, the principal themes identified as the key components in training a competent outbreak investigator related to: personal qualities and interpersonal skills, alongside prior qualifications; elements of effective supervision including technical and soft skills, flexibility, and personal compatibility with the trainee; and activities to best support trainee development including classroom teaching in preparation for practical experience, and the ideal approach and quantity of practical exercises.

What Makes a Good Outbreak Investigator?
Although this question was asked in relation to outbreak investigators overall, participants answered this largely in relation to a trainee learning about outbreak investigation. Participants repeatedly identified the importance of certain personal characteristics as key to being an effective outbreak investigator and to being a good trainee in outbreak investigation. Trainees who were considered to be good outbreak investigators were perceived to be curious, have the ability to think creatively and laterally, be selfdirected, and have interest in and enthusiasm for the job. Tenacity and attention to detail were also identified as very important qualities. These qualities, whilst desirable in an MAE trainee, were not considered baseline competencies for admission to the MAE Program at the time of this study. While interview participants recognized that all students entering a program such as the MAE Program will have some kind of scientific training as a baseline for admission, specific training and skills were seen as generally less important than the investigator's personal traits and temperament.
"... mainly their creativity and persistence are the key criteria, regardless of their background really. Some of the best investigators haven't had content knowledge at first but they've learnt very rapidly" (Participant 1).
"I think they have to have some biomedical training... But most importantly you need to be at the right temperament and the right inquiring mind. I think that's the most important thing" (Participant 10).
Aligning with this theme, soft skills relating to communication and interpersonal abilities were frequently mentioned in interviews. Interpersonal skills were seen to be critical to the investigator's ability to operate as part of a multidisciplinary team and to liaise with other agencies, as well as critical to being able to function for extended periods in a team environment while managing fatigue and stress. Without these skills "under pressure, to be away from contact with family... if they don't have exceedingly good interpersonal skills, they can start to show and that can be a major detractor" (Participant 7). These skills would assist a trainee in being accepted into the MAE Program throughout the interview process but are not a required baseline competency, with little coursework currently focusing on the development of these soft skills. Conversely, the inability to work within a team, and in particular, strong-headed or competitive trainees were seen to be a challenge to effective outbreak response and well-functioning team dynamics: "I have witnessed a bit of competitiveness in some students that does not fit well ... you really need someone who's good at teamwork" (Participant 3). It was also seen to be vital that investigators and trainees know their own limitations and be willing to seek support from their colleagues whenever needed.

What are the Elements of Effective Supervision?
Participants identified some key areas where supervisors required content knowledge and expertise so that they could effectively guide their trainees. Specifically, participants identified that supervisors needed to have experience in outbreak investigation and response to equip them with requisite technical knowledge on topics such as: how to conduct cohort and case control studies; how to apply appropriate statistical methods and the development of effective written communication for various audiences. Outbreak experience was also seen as important as it meant that supervisors had a professional network that could be used to facilitate access to resources, technical knowledge and practical experience for their trainees.
Similar soft skills relating to communication, teamwork and collaboration were also identified as key supervisory skills. One participant noted that some people "aren't very good supervisors because they're not good at relating to people or communicating or facilitating access for an MAE or FETP fellow. The best ones are those that take an active role in mentoring their trainee" (Participant 4).
The above quote illustrates a recurrent theme of the importance of the supervisor's interest in supervision, more broadly including a genuine interest in building the trainee's skills and knowledge. This theme was strongly reiterated by other participants: "…supervisors should… make a student feel that they're valued and that you're investing in them... it's a two-way street" (Participant 8).
"you want somebody that's not really looking at [the trainee] as free labour, but rather as the future of capacity in the jurisdiction they're working in" (Participant 7).
In building an effective supervisory relationship, interpersonal compatibility leading to good rapport and effective communication between a supervisor and trainee was seen as more important than the supervisor's subject area being aligned with the trainee's interests. Supervisor involvement in recruitment and selection of trainees was seen as an effective step to facilitate supervisor-trainee compatibility and supervisor investment in the relationship.
In addition, the degree of a supervisor's seniority was not seen as critical to positive trainee outcomes, and participants thought that the selection of a supervisor should be adapted to the individual trainee. However, it was recognised that more experienced and senior supervisors could help create better placement opportunities to maximise trainees' practical experience.
Participants described that supervisors need to be adaptable in their supervision style to the trainee's personality, needs and goals. Participants suggested that for trainees with different levels of experience and confidence, supervisors need to be able to tailor their degree of involvement to meet the trainee's needs for support or independence. This included negotiating the frequency of face-to-face contact with the trainee, with some trainees only needing infrequent contact to stay on track, and others benefiting from daily conversation and reassurance. The majority of participants commented that it is crucial for supervisors to have an 'open door policy' and to be available for trainees to contact them for support when needed, ideally being co-located in the workplace. Regular contact was seen as critical to ensure that trainees avoid spending long periods of time working on tasks that may not be important or which need additional direction, and "to make sure the trainee doesn't feel they're floundering around and not have a clear direction or not learning" (Participant 8). These types of soft skills for supervisors, as with trainees, were not part of a structured selection criteria at the time of this study.

What Activities Best Support Trainee Development?
Participants identified 3 main areas as important for trainee learning; classroom exercises, practical teaching and writing up outbreaks.
Classroom teaching exercises were seen as valuable preparation for more applied practical exercises and for outbreak investigation in the field. These include 'desktop' simulated outbreak scenarios, familiarisation with technical tools and standard operating procedures for fieldwork, and developing questionnaires. Classroom teaching was seen to be a "good first step" (Participant 7), allowing trainees to gauge their interest in the area and to have controlled exposure to elements of outbreak response and theory before being placed in a real scenario where 'the real learning takes place'. Many participants repeated the point that classroom learning is secondary in value to practical experience, but that it was useful as part of a tiered approach to prepare for outbreak investigation.
Topics that were viewed as being important to cover in training included communicating risk and managing media enquiries; designing and delivering questionnaires; data system management and data

analysis; and training in interpersonal communication and team dynamics.
Participants spoke at length about the critical importance of practical experience for trainees' development, in alignment with the emphasis they placed on trainees' pragmatic interpersonal skills. The informal nature of this learning, both through doing and through role modelling was emphasized, with the opportunity to collaborate with and learn from more experienced investigators seen as one of the most valuable aspects of investigation.
Participants identified differing numbers of outbreaks (between 1-10) that they viewed as being adequate for competence in outbreak investigation. However, there was universal agreement that more practical experience is always better. Several participants indicated that trainees should have opportunities to cover the range of key investigative study designs including case series, case control studies and cohort studies, and investigate outbreaks caused by a number of different pathogens including "infectious disease emergencies ... run of the mill, notifiable diseases like measles and mumps ... [and] enteric disease outbreaks" (Participant 10).
It was considered important for trainees to have an extensive and diverse practical experience covering the full spectrum of the outbreak investigation process, ranging from planning teams and resourcing, designing and piloting a questionnaire, through to debriefing and report write-up.
Participants described that trainees learn the most when they have ownership and responsibility to manage some aspect of an outbreak investigation. However, they also stated that it is important to balance the responsibilities of the trainees with their experience (i.e. by taking the trainees out of their comfort zone but in a way that was responsible and supported). Participants suggested that there is value in progressively giving trainees more responsibility and independencetrainees should start in a supporting and observing role, and progress through managing more complex and varied tasks, to eventually managing an outbreak response.
Reflection exercises and team debriefing were viewed as being important. Following practical experience, it was thought that formal debriefing and writing up reports were very valuable. Debriefs can be an opportunity for trainees to meet staff from collaborating agencies and learn from them, and "see the different cultural approaches that they get" (Participant 1). Participants thought that involvement in the debrief and write up processes could leverage and maximise the value of practical experiences for trainees, allowing them to critically reflect and identify lessons for future improvement, as well as contributing to improving public health responses -"it does force you to see the whole arc of the outbreak investigation" (Participant 9).

Discussion
The experts interviewed for our study viewed the 'Product' of training -that is becoming an effective outbreak investigator -as requiring a complex and varied skillset. Interview participants identified a variety of important skills and traits, including technical knowledge and professional experience, alongside interpersonal skills and personal qualities such as enthusiasm and tenacity. There is limited literature on the specific skills and attributes that characterise an effective outbreak investigator, however it is noted that curricula for outbreak investigation training programs typically focus on discipline-specific content and technical skills (9). There were no specific selection criteria for trainees at the time of this study, with coursework focusing heavily on technical skills. Given the importance placed on soft skills for an individual to be an effective outbreak investigator, these competencies could be addressed and developed within the MAE training program.
To be a good trainee for outbreak investigation, study participants identified personal qualities and interpersonal skills as being more important than prerequisite technical knowledge in outbreak investigation. Our findings indicate that beyond their training background, a good trainee needs to know their own limits, function well in a team environment under conditions of stress and fatigue, and demonstrate creativity, curiosity and tenacity. This emphasis on nontechnical skills and personal traits echoes research that identifies "dedication to hard work and having an intrinsic curiosity and a sense of discovery" as key factors for success in epidemiology careers (18). This is also reflected in other published work where good communication and interpersonal skills were emphasised for trainees and supervisors in field epidemiology training (19). Given the focus of outbreak investigation training on technical competencies, it is somewhat surprising that our participants did not place more emphasis on technical skills in distinguishing good trainees. It may be that these skills are a baseline expectation, or that these technical skills -unlike 'soft' personal and interpersonal qualities -are viewed as being able to be taught and learned more easily. It was difficult to determine, when viewing these results through the 3P model of workplace learning, whether these non-technical skills should be related to Presage, that is factors that trainees bring to the program, or to 'Process', that is factors that should be developed through training activities.
Supervision was seen as a key activity to support effective outbreak investigation. Our participants identified that both technical competencies and personal, non-technical skills are important for effective supervision. Our results also indicate that the supervisor's attitude to supervision is also very important, including their interest in and approach to supervising, and their flexibility in adapting to individual trainees' needs. These aspects are supported by research literature on workplace supervision that indicates the supervisory role should be holistic, including elements of pastoral care and supporting personal development for the trainee (20). While participants identified these various qualities and attitudes being valuable for supervisors, they also indicated that the dynamic of the relationship between supervisor and trainee is critically important for effective training outcomes. This reflects other findings which state that "the quality of the supervisory relationship strongly affects the effectiveness of supervision" (20), and that the compatibility of personalities between supervisor and trainee is an important predictor of later professional outcomes for trainees in applied epidemiology (21). Similarly, the findings of our study indicate that a compatible and positive relationship between supervisor and trainee is critical for effective supervision and learning to occur. In this way it is important that supervisors also have strong interpersonal skills, in order to take an active and empathetic mentoring role in the trainee's development and to adapt their supervision style according to the trainee's ability and needs.
With regards to other Process factors, our findings regarding classroom learning and practical training activities indicate that theoretical and classroom-based learning are valuable as part of a tiered approach to learning, which progressively gives trainees greater responsibility and exposure to outbreak investigation. Classroom learning was described as being of lesser importance than practical experience, but valuable as preparation for the real thing. These findings align with research literature on workplace learning, describing that close linkages between "expanding one's theoretical knowledge and insights" and "externalising one's practical and theoretical insights" are critical for effective learning and growth of expertise (16). This body of research also suggests that there is value in gradually increasing exposure and responsibilities over subsequent learning experiences, to acquire expertise (22,23). This approach was valued by our interview participants, suggesting that trainees best develop mastery and confidence by having incrementally increased responsibility and independence in each practical outbreak investigation or other workplace scenario.
The findings of our study have practical implications for the MAE Program, and potentially for other FETPs. The MAE Program strives to improve curriculum quality, and our findings on the perceived importance of interpersonal skills and personal qualities in trainees has been and will continue to be important considerations when reviewing curriculum materials. They are also important to consider in the trainee recruitment process. The relative degree to which these are inherent skills that a trainee brings to the program compared with qualities that can be acquired through either purposeful practical experience and/or classroom-based learning would be a very useful area for further research to consider. At the time of this study there were no concrete selection criteria for acceptance into the MAE Program. Selection was based on previous experience and a set of standard interview questions. The practical application of these findings from interviews is the development of a set selection criterion for the MAE Program which now focuses on problem solving and initiative, as well as technical skills. With regards to learning in the workplace, our interviewees' views on a structured approach towards greater autonomy and responsibility may be useful to guide new field supervisors in approaching supervision. However, this may need to be adapted depending on the workplace and outbreaks to which the trainee will be exposed. Within the MAE trainee recruitment process, workplaces are already closely involved in the selection of trainees, and participant views on the importance of compatability of the supervisor-trainee relationship in helping achieve good training outcomes reinforces the importance of continuing this practice.
A limitation of this study is the use of a small purposive sample to assess subjective views of the important aspects of outbreak investigation training, with no measure of objective training outcomes related to these proposed elements of effective supervision and training. Further investigation in this area would be strengthened by including more objective measures of program success and linked to more objective measures of longitudinal trainee outcomes. Studies of supervision in other contexts have used measures such as trainees' later publication rates and job attainment in relevant fields, which could be explored when developing more direct indicators of efficacy in outbreak investigation training [21].

Conclusion
This study identified that effective outbreak investigators possess a broad range of skills and knowledge. These span proficiency at epidemiological tasks to interpersonal communication, underpinned by personal attributes such as perseverance and curiosity. Similarly, supervisors should ideally demonstrate passion for teaching and investment in the holistic performance and wellbeing of trainees, providing flexibility to adapt to the needs and learning styles of each trainee. For both supervisors and trainees, questions around which of these skills can be taught versus which are innate remain unanswered. However, it does appear that some of these personal attributes are inbuilt. This information has been used to develop specific selection criteria for entry into the MAE Program, where previously no such criteria was used. In addition, there appears to be a disconnect between the current coursework which is technically focused, whilst many of the factors considered to make an individual a good outbreak investigator are personal qualities and soft skills. This should be an important consideration for future adaptation and development of the MAE coursework and curriculum. Further research in this area would be useful to guide training activities to ensure that outcomes of FETPs and outbreak training more generally achieves the best possible results in an era of increasing threats to global biosecurity. What Activities Best Support Trainee Development? Classroom Teaching -classroom learning seen as secondary in value for trainees, but important in preparing them for practical scenarios;

Box 1 -Summary of Results
Specific Competencies -key skills to be trained included: multi-day simulated exercises; media engagement; questionnaire design; data management & analysis; teamwork and interpersonal skills Practical Experience -'learning by doing' is the most important aspect of applied epidemiology training; the more practical experience, the better; diverse experiences important to cover the full spectrum of outbreak investigation; level of involvement and responsibility should be gradually increased over subsequent practical scenarios Debriefing -good opportunity to meet and learn from workers in other agencies; allows critical reflections and maximises learning for future; debriefing should be a standardised structured process to consistently cover key necessary elements