If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Point prevalence surveys of healthcare-associated urinary tract infections: Development, pilot testing and evaluation of face-to-face and online educational packages
School of Nursing, Midwifery and Paramedicine, Australian Catholic University, Canberra, Australian Capital Territory, Australia1Nursing Research Institute, Australian Catholic University and St Vincent's Health Australia (Sydney), Australian Capital Territory, Australia2Lifestyle Research Centre, Avondale College of Higher Education, Cooranbong, New South Wales, Australia
Faculty of Arts, Nursing and Theology, Avondale College of Higher Education, Wahroonga, New South Wales, AustraliaSchool of Nursing and Midwifery, Griffith University, Nathan, Queensland, Australia
Few studies investigate training and education of staff involved in surveillance.
•
Mastery learning supports training suited to a broad range of knowledge and skills.
•
Training health staff facilitates consistent data collection for HAUTI surveillance.
Abstract
Objective
To describe the development, pilot testing and evaluation of face-to-face and online educational training packages for healthcare staff undertaking point prevalence surveys (PPS) of healthcare-associated urinary tract infections (HAUTIs) in Australian hospitals and aged care facilities.
Methods
The study involved two phases. A face-to-face educational training package was developed and used in Phase I of the HAUTI PPS data collection conducted in six hospitals. In Phase II, the training package was expanded and modified for online use by healthcare staff in 82 hospitals and 17 aged care facilities. Ten staff evaluated the face-to-face training package in Phase I. For Phase II, 38 staff evaluated the online training package. After each phase, staff completed an online evaluation survey about the usefulness of the training package and ease of data collection.
Results
For Phase I, usefulness of the training package was rated highly (100%, n = 10) with all respondents rating the training useful in preparing for data collection. Staff in Phase II also reported the online training useful in preparing for data collection and was rated very useful by 21% (n = 8) of respondents and useful by 66% (n = 25). Some respondents (Phase I, n = 4 and Phase II, n = 25) provided small amount of text data that was triangulated with quantitative data. Qualitative feedback reinforced quantitative ranking of usefulness of the training package.
Conclusion
The training packages were sufficient to train healthcare staff with varying levels of knowledge and skills in undertaking HAUTI PPS in hospitals and/or aged care facilities.
]. Most healthcare-associated urinary tract infections (HAUTIs) are caused by use of indwelling urinary catheters and are termed catheter-associated urinary tract infections (CAUTIs) [
The use of point prevalence surveys of healthcare-associated infection to identify risk factors and facilitate infection prevention and control planning.
]. To provide the foundation for a national PPS, the Surveillance to Reduce Urinary Tract Infections (STRUTI) study was developed. This is a three-phase study with Phase I conducted in six Australian hospitals [
]. Phase II aimed to provide proof of concept by testing the protocol using an online data collection process. This phase involved development, pilot and validation of an online database for hospitals and aged care facilities to conduct point prevalence UTI surveillance. Data collection was again conducted in acute care settings and was extended to include aged care facilities which were under-represented in Phase I.
Training of healthcare staff is an important part of an infection control program [
]. Besides broader infection control training, specific training of healthcare staff in infection control surveillance is essential. Findings from a recent study showed that just over half of all surveyed participants had been trained in HAI surveillance and those who had been trained were significantly more likely to undertake prospective surveillance and perform risk adjustment [
]. These findings emphasise the benefits of surveillance training which extends beyond the interpretation and application of surveillance definitions but more importantly the use of appropriate methods in collecting surveillance data and analysis of these data [
]. Ensuring high quality and complete data sets requires adequate and consistent training of staff involved in undertaking data collection. There is a lack of training of healthcare staff in Australia in undertaking HAI surveillance [
] and to our knowledge, there no available studies reporting on the acceptability of online HAI surveillance training. This paper aims to describe the development, pilot testing and evaluation of face-to-face and online training packages for staff that undertook a HAUTI PPS in Australian acute and aged care facilities.
Methods
A training package was developed in Phase I to provide educational support to those undertaking data collection for the HAUTI PPS in acute care. The Phase I package included both face-to-face presentations and electronic resources. After completion of Phase I, data collectors provided feedback that informed development of the online training package for Phase II. Given the much greater number of participating institutions in Phase II, face-to-face delivery was not feasible. The Phase II package comprised a module for acute care and another for aged care. Data collectors in both Phases were employed in hospitals and/or aged care facilities and were primarily nurses, some but not all with a background in infection control. The training packages had to be appropriate for both staff with extensive knowledge and training in infection control, prevalence surveys and the methodologies governing accurate and reliable data collection, and staff who had limited post-secondary qualifications.
Whilst there was a minimum mandatory level of content to ensure consistency of data collection, the training packages were largely developed based on the principle of adult learning theory [
]. Adult learning characteristics were summarised by Knowles as: autonomous and self-directed; accumulated a foundation of experiences and knowledge; goal oriented; relevancy oriented; practical and a need to be shown respect. Use of a variety of formats was important to cater for differences in learning preferences and to maintain interest for busy staff [
], the training package allowed participants to undertake training at their preferred speed, and employed assessment to ensure consistency in knowledge. These principles were also followed in Phase II where the training package was completely online but included a range of resources and mastery assessment using several randomly selected multiple choice questions (MCQs) with feedback when incorrect answers were given.
Phase I: Face-to-face training
Development of acute care training package
Development of the training package was partly informed by resources from the Health Protection Scotland Education and Training Events [
]. The data collection items further informed the content of the package. The training package included a paper-based manual consisting of five learning modules (Table 1).
Table 1Outline of learning modules for Phases I and II training packages.
Learning modules
Phase I
Phase II
Module 1
Basic epidemiology principles including prevalence and incidence
Introduction to HAUTIs and Prevalence surveys
Module 2
Data collection principles and the methodology for PPS data collection
Data collection methods for STRUTI PPS data collection
Module 3
The use of Centre of Disease Control and Health Protection Agency case definitions
The use of the Health Protection Agency and McGeer case definitions
Module 4
Practical application of the definitions using case studies
Practical application of the definitions using case studies
Module 5
Face-to-face demonstration session of data entry using the data collection forms
In developing this package for Phase I, it was assumed that data collectors had some prior clinical and infection control knowledge as majority were registered nurses. The training package explained all necessary PPS procedures. Data collectors were provided with the paper based manual prior to commencement of the training session.
Delivery of training
The training was delivered face-to-face over 2 h. It was mandatory for all data collectors to undertake the training. The five learning modules were delivered as separate power point presentations. At the completion of the training session, participants were expected to:
i.
Have an understanding of the aims of the PPS.
ii.
Have knowledge of principles and approaches to PPS epidemiology inclusive of strengths and limitations.
iii.
Understand the PPS data collection process.
iv.
Be able to apply surveillance definitions of UTI in acute care facilities.
v.
Understand the concept of reliability in the context of the PPS.
vi.
Be able to collect accurate data on the PPS.
vii.
Have developed data entry skills as they relate to PPS data, in addition to gaining knowledge and confidence in utilising the PPS regarding their study site.
viii.
Have an understanding of the reporting output from the PPS.
Immediately following completion of the face-to-face training session, data collectors completed post-training case study assessments. This face-to-face case study style assessment comprised two case scenarios based on detection of HAUTIs with five questions asked of each scenario. A total of 20 min was allocated for completing both case study questions. Data collectors were required to achieve at least 80% mastery in the assessment before proceeding with data collection. We planned for those who did not achieve this mark to be asked to complete another two case studies before being approved to participate in data collection; however, this was not required.
Post-data collection evaluation survey
After completing the HAUTI PPS data collection, all data collectors completed an anonymous online survey. The evaluation survey was designed to gain feedback on the usefulness of the training package and ease of data collection. The survey comprised 17 items of which four were open questions, nine were closed questions and the remaining four were closed questions with an option for providing comments. Quantitative data were collected using Likert scales while qualitative responses were entered in text boxes. Table 2 provides a summary of domains assessed in the evaluation surveys.
Table 2Summary of domains assessed in Phases I and II evaluation surveys.
Domains
Phase I
Phase II
Data collection site
Specify acute care hospital
Specify acute care hospital, aged care facility or both
Domain in evaluation survey relevant to training package.
Rate usefulness of the training package based on scale responses
Rate usefulness of the training package based on scale responses
Identification of patients with UTI
Tick all steps that apply
Tick all steps that apply
Survey form and data collection process
Rate ease of use of survey form based on scale responses; identify strengths of the survey form and areas for improvement in the form and data collection process
Rate ease of registration on the STRUTI website and ease of use of online survey form; identify areas for improvement in the form and data collection process
Surveillance definitions
Rate usability of definitions and identify any gaps with the definitions
Urinary catheter insertion
Provide feedback on the reasons for catheter insertion and reviewing the need for catheter insertion
a Domain in evaluation survey relevant to training package.
Development of acute and aged care training packages
The Phase II acute care training package was based on the Phase I package with modifications made for an online-only format, and was also informed by feedback from Phase I post-data collection survey described above. An additional training package was developed for conducting HAUTI PPS in aged care facilities. The aged care training package was based on the acute care training package template where possible but included incorporation of the McGeer surveillance definitions for UTIs which are specific to aged care [
]. Specific consideration was given to the fact that not all data collectors would be registered nurses.
Delivery of training
Two weeks prior to commencement of online data collection, data collectors were sent the online training package. There were four online modules (Table 1). Data collectors could access the modules at any time via the STRUTI website. Prior to data collection, data collectors were prompted to complete an online post-training assessment of MCQs (see Fig. 1 for screen shot of webpage). The assessment comprised ten MCQs, which were designed to ensure that data collectors had an understanding of surveillance definitions and could apply them to undertake the PPS. The MCQ structure and content was reviewed by an academic with expertise in MCQ development and pilot tested with a small number of infection control practitioners.
Figure 1STRUTI training package assessment screen.
Similar to Phase I, data collectors were expected to complete the questions within 20 min. Again, 80% mastery was required and data collectors were given a total of three attempts to successfully complete the post training assessment. If all three attempts of the assessment were unsuccessful in gaining a pass of 80%, the study team would be notified and the data collector would be given additional training and testing prior to undertaking data collection; however, this was not required.
Post-data collection evaluation survey
At the completion of online PPS data collection, data collectors were asked to complete an anonymous online evaluation survey (Table 2) to provide feedback on the online training package and online data collection process. There were 10 items, comprising one open question, eight closed questions and one closed question with an ‘other’ option allowing for additional information to be collected. Quantitative data were collected using Likert scales and qualitative responses were entered in text boxes.
Data management and analysis of evaluation surveys
Data from Phase I were collected using Survey Monkey™. Data from Phase II were collected using Qualtrics™. Quantitative data were analysed using descriptive statistics (with SPSS version 21). Some respondents entered comments in Phase I (n = 4) and Phase II (n = 25) providing a small amount of text data that were triangulated with quantitative data. About 50% of comments related to the quality and content of the training package and the remainder related directly to the data collection and data entry process and are not reported in detail here. Comments related to the training package were sorted, compared with quantitative ratings and are summarised descriptively.
Results
Phase I
Eleven data collectors assisted with data collection for the PPS. A response rate of 91% (n = 10) was achieved for completion of the post-data collection evaluation survey. The training package was well received by respondents. Usefulness of the training package was rated highly (100%, n = 10) with all respondents rating the training useful in preparing them for data collection.
Four respondents provided brief qualitative feedback. No formal data analysis was conducted. The feedback reinforced the high rating for the usefulness of the training package in undertaking the PPS:“The training was essential to understand what was required of me and a useful time to ask question[s] so the hopefully there were no misunderstanding.”
One respondent suggested that it would have been helpful to provide more information about how to use data collection forms. This feedback prompted inclusion of specific instructions in the Phase II package.
Phase II
From the 82 acute and 17 aged care facilities, 38 data collectors completed the evaluation survey. A total of 92 data collectors passed the acute and/or aged care online training assessment hence an estimated response rate of 41% was achieved for completion of the post-data collection evaluation survey. We consider this to be an estimate because there were some data collectors who did both the acute and aged care training and the surveillance but would have only responded to the evaluation survey once. It is also likely that some data collectors may have undertaken the training but not the surveillance. The majority of respondents (79%, n = 30) collected data from acute care facilities, 5% (n = 2) from aged care facilities and 16% (n = 6) from both acute and aged care facilities.
When asked about the usefulness of the online training package in preparing for data collection, 21% (n = 8) rated the training very useful, over half of respondents (66%, n = 25) rated the training useful and 13% (n = 5) rated the usefulness of the training package as ‘neutral’. None rated the package as not useful.
Qualitative feedback generally reinforced quantitative ranking of the usefulness of the training package. A small number of respondents appeared to find the multiple choice structure frustrating, recognising that in some instances they had to make a final choice between two potentially correct answers, or that the questions were too simplistic. One respondent suggested that:‘there should be a few more tricky questions in your training package’.
A significant amount of feedback related to difficulties with the process of online data entry. For example, two respondents requested training about basic login processes and navigation of the website.
Discussion
We have described the development, pilot testing and evaluation of a face-to-face and an online training package separately for conducting HAUTI PPS in acute and aged care facilities. Our findings indicate that the training packages adequately trained healthcare staff with different levels of prior education to conduct data collection for a HAUTI PPS. The training packages were perceived as important to conduct the PPS.
Infection control is a health and safety issue; therefore all those working in hospitals and aged care facilities are responsible for providing a safe environment for patients, residents and other staff [
]. Training of healthcare staff in both acute and aged care facilities is a key component of an infection control program and collection of surveillance data have been identified as a good training starting point [
]. Undertaking HAUTI PPS requires a well-developed training package providing useful information for a wide range of healthcare staff.
Our results show that data collectors evaluated the training packages as essential to successfully conduct PPS. While comparison of the different modes of delivery of the training was not feasible given the descriptive nature of the study, usefulness of the training package was still rated highly using face-to-face and online training approaches. Although face-to-face teaching is widely used in healthcare facilities, there is growing evidence of the effectiveness of online training programs [
] and our study adds to this body of knowledge. Due to the busy work environment in healthcare facilities, the need to acquire new knowledge and skills has resulted in a shift from traditional in-class teaching to online training. This allows training based on job needs and provides flexibility to healthcare staff [
]. Evidence shows that nurses generally have positive perceptions of online education, related to convenience and individual self-directed preparedness for learning [
]. Online training also allows consistency of data collection across many institutions.
The evaluation findings from both phases showed that the majority of data collectors found the training packages useful in preparing them for collecting data for the PPS. Although ascertaining the usefulness of the training package would also require assessment of and comparison with outcomes from untrained data collectors, the absence of any detailed feedback about specific components of the training package suggests that the overall content was appropriate. The use of MCQs as an assessment format as opposed to case study scenario used in Phase I also appears to have been generally well accepted, despite suggestions that MCQs are a more superficial learning approach as opposed to a deep learning approach [
]. In particular, application of the mastery approach valued previous learning and was effective and appropriate for health care staff with varied existing levels of knowledge [
]. Achievement of mastery in the assessment allowed for variations in speed of learning and is supported by the concept of ‘mastery learning’ which refers to teaching methods that allow a consistent level of performance that all students must master before moving to the next task [
]. The feedback about difficulties with basic procedures such as login, suggests that future training packages for online PPS should include a module about use of computer software.
Data collection in Phase II was undertaken in both hospital and aged care facilities. The shift of health care delivery from hospitals to other settings has widened the types of healthcare staff who require training in infection control practices. Infection control training is essential for a broad range of staff that work in different healthcare settings and may be involved in the management of patients and residents [
]. It is important to both develop a broadly appropriate training package and assess education provided to different healthcare staff, as this may not only improve knowledge and detection of HAUTIs but may lead to a reduction in HAUTIs and ultimately improvement in patient and resident outcomes.
There are a few limitations noted in our study. This study was not designed to assess any correlation between improvements in quality of HAUTI surveillance and training modality. We could not determine if there were any differences in the feedback between hospital and aged care staff given that some respondents undertook data collection in both areas. Although majority of data collectors in both phases were nurses, data on professional level was not collected for the evaluation survey. This may have provided information on differences in feedback between infection control practitioners, registered nurses or enrolled nurses. For Phase II, there was at least 2 weeks between training and post-training assessment. Given that the Phase II post-training assessment was not undertaken immediately after delivery of the online training there is the potential for recall bias in the study findings. The estimated 41% response rate for the Phase II post-data collection evaluation survey limits generalisability of the study findings for this phase. Despite these limitations, standardised online training packages have now been developed for participating hospitals and aged care facilities in Australia to undertake HAUTI PPS. The use of specific assessment cut-offs to determine knowledge attainment and participation in the PPS also added strength to ensure staff had an adequate and standardised level of knowledge prior to being allowed to undertake data collection. The training for aged care staff in Phase II has provided capacity building with aged care facilities providing informal feedback that the training increased workforce capability for aged care staff who rarely receive specific infection control training.
Development and delivery of a robust HAUTI PPS training or educational package will ensure a consistent methodological approach in data collection and improve efficiency in surveillance processes for healthcare staff [
]. Consistency in training and assessment improves data reliability and this is especially true for HAI data which are now frequently publicly reported [
]. Training packages are also not often evaluated, and evaluation is an important aspect as it determines effectiveness of the training and ensures that training objectives have been achieved.
This study has demonstrated that both face-to-face and online training packages were sufficient to train healthcare staff in undertaking HAUTI PPS. This study provides important evidence about the methodological approach being taken in developing and implementing national HAUTI PPS in acute and aged care facilities, leading to potential health benefits to thousands of Australians requiring urinary catheters as part of their care. Results of a proposed national PPS will inform updates to the training packages. Further studies are needed to explore whether face-to-face or online training of healthcare staff improves quality of HAUTI surveillance data.
Ethics
Approval for undertaking Phases I and II of the study was obtained from the Australian Catholic University Human Research Ethics Committee (HREC) as well the HRECs of participating healthcare sites. This approval included permission for implementation of the PPS and participation in the post data collection evaluation surveys.
Authorship statement
All authors conceived the idea for this manuscript and contributed to manuscript preparation. All authors have seen and approved the final version.
Conflicts of interest
The authors declare that they have no conflicts of interest. Two of the authors have an editorial affiliation with the journal. They played no role whatsoever in the review or decision rendering process for this paper.
Funding
Phase I of this study was funded by a Covidien 2012 Infection Control Scholarship and an Australian Catholic University Faculty of Health Sciences Research Grant. Phase II was funded by an Ian Potter Foundation Health and Disability Grant and a second Australian Catholic University Faculty of Health Sciences Research Grant.
Acknowledgements
The authors would like to thank the staff of the participating hospitals and aged care facilities. Thanks also to A, B and C for their assistance in conducting this research study. Thanks to D for assistance with development of the Qualtrics survey.
Provenance and peer review
Not commissioned; externally peer reviewed.
References
Pittet D.
Donaldson L.
Challenging the world: patient safety and health care-associated infection.
The use of point prevalence surveys of healthcare-associated infection to identify risk factors and facilitate infection prevention and control planning.
30039-1&id=gr1.jpg){kind=link}