Eric E. Brown, MD, MSc, FRCPC1, 2, 3*, Tarek K. Rajji, MD, FRCPC1, 2, 3, 4 and Renee Logan, MD, CCFP1,5
1. Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Canada
2. Department of Psychiatry, University of Toronto, Toronto, Canada
3. Adult Neurodevelopment and Geriatric Psychiatry Division, Centre for Addiction and Mental Health, Toronto, Canada
4. Toronto Dementia Research Alliance, University of Toronto, Canada
5. Department of Family and Community Medicine, University of Toronto, Toronto, Canada
*Corresponding author:
Eric Brown
Centre for Addiction and Mental Health
80 Workman Way
Toronto, ON M6J 1H4
Email: This email address is being protected from spambots. You need JavaScript enabled to view it.
ABSTRACT
Background: Significant COVID-19 transmission occurs from people not experiencing symptoms. We aimed to investigate whether rapid antigen testing among pre-symptomatic or asymptomatic healthcare workers along with symptom and exposure screening may help to identify COVID-19 cases.
Methods: We invited staff on two inpatient geriatric mental health units to voluntarily undergo testing with the Abbott PanbioTM rapid antigen test kit on site, up to three times weekly for up to eight weeks per participant. We expanded the study to two more units due to low recruitment.
Results: From March 30 to July 23, 2021, we tested 28 participants who underwent on average 16.5 tests participating for an average of 46 days. Of 462 rapid tests,
one was positive, which was followed by a negative confirmatory polymerase chain reaction (PCR) test. The participation rate among the initially targeted population
was low (16.7%).
Discussion: Our study occurred during initially moderate to high but dropping community incidence rates. The positive result occurred when community transmission was low. It was likely a false positive. There were no detected or reported cases of COVID-19 among staff participants.
Conclusions: Rapid antigen testing did not identify cases when used in addition to symptom- and exposure-based screening in asymptomatic staff in our small study. Recruitment was low and we suspect repeated testing by appointment was perceived as onerous.
KEYWORDS COVID-19; rapid antigen testing; institutional outbreaks; occupational health
INTRODUCTION
Hospitals, and in particular psychiatric hospitals, have been impacted by outbreaks during the COVID-19 pandemic [1]. For example, in Ontario, Canada, between February 16, 2020 and June 12, 2021, there were 568 hospital COVID-19 outbreaks [2]. In addition to the resulting patient and staff morbidity and mortality, each COVID-19 outbreak was associated with a closure of a median 11 days, disrupting the function of the healthcare system. To reduce the risk of COVID-19 outbreaks introduced by staff, hospitals have implemented policy changes including universal masking and point-of-entry screening for symptoms, risk factors, and exposures. A challenge to symptom-based screening is that the period of infectiousness of COVID-19 begins before the onset of symptoms (i.e., is pre-symptomatic) [3]. Additionally, a proportion of cases may not ever be associated with symptoms (asymptomatic cases) [4]. A majority of onward transmission in COVID-19 outbreaks may be “silent”, i.e., transmitted from an individual who is unaware of potential illness [5]. Symptom- and exposure-based screening programs may, therefore, miss asymptomatic infectious cases [6]. This may be particularly true in healthcare settings in which most or all of the staff are vaccinated, as breakthrough infections are more likely to be mild or asymptomatic [7]. Testing for COVID-19 infection in asymptomatic healthcare staff may, therefore, prevent hospital COVID-19 outbreaks.
Our hospital is the Centre for Addiction and Mental Health (CAMH), a large academic mental health hospital in Toronto, Ontario, with subspecialized inpatient and outpatient services. Prior to our study, 9 COVID-19 outbreaks occurred in the inpatient units, the majority of which started with staff illness. The 48-bed geriatric unit experienced a COVID-19 outbreak resulting in six infections causing serious morbidity to two patients, including one patient death. At CAMH, Infection Prevention and Control (IPAC) protocols implemented during the pandemic include front-door screening of staff, patients, and visitors. During our study, screening protocols included daily questions or staff self-attestations regarding symptoms and known case contacts (Table 1). Positive responses to either question excluded staff from attending work and would qualify them for polymerase chain reaction (PCR) COVID-19 testing. We investigated whether additional voluntary, repeated testing of asymptomatic staff with rapid antigen testing is feasible and whether it may help to identify cases and prevent COVID-19 outbreaks.
METHODS
With age and cognitive impairment being important risk factors for severe COVID-19, our study initially targeted the two geriatric mental health inpatient units at CAMH. These units contain 23 and 25 beds each for a total of 48 subspecialized tertiary care geriatric mental health inpatient beds. The patient population varies but generally includes patients over the age of 60 with major neurocognitive disorder requiring acute assessment and treatment, and individuals over the age of 65 with age-related mental health disorders. A variable proportion of patients may also be awaiting long term care or more supportive housing and have fewer acute issues.
Our inclusion criteria broadly included any staff working on the two geriatric inpatient floors willing to provide deep nasal samples. Due to low recruitment, this criterion was expanded with a research ethics board protocol amendment, to allow the participation of trainees and expand the setting to any inpatient unit at our hospital. We expanded recruitment to two long-stay forensic units (31 and 16 beds each) for a total of four participating inpatient units. Participants also needed to be asymptomatic for COVID-19 and pass the hospital-wide standard screening protocols (Table 1). Participants were excluded if they previously had tested positive by PCR for COVID-19.
We ensured all eligible geriatric inpatient staff were aware of the study by sending recruitment e-mails, posting flyers on site, discussing the study at team meetings, discussing the study with individuals and by having study personnel present on the units.
All participants provided written informed consent following a discussion of the risks and benefits of participation. Participants were compensated for their participation. Participants were allowed to participate during work hours. Our study received ethics approval from the CAMH research ethics board, internal hospital approval, and union support.
The study period started with the first recruitment email on March 26, 2021 and ended with the final test on July 23, 2021. During the 119-day study period, participants were invited to participate for up to 8 weeks starting from their first rapid test, with a pause in participation permitted for vacation or other leaves. Participation could end early if the participant withdrew, was unable to comply with the study requirements, or no longer met criteria for inclusion (i.e., they tested positive by PCR for COVID-19 during the study). During their participation, participants were asked to book and undergo testing up to 3 times per calendar week. Testing could occur at any time mutually acceptable to the participant and research assistant, and research assistants provided daily availability including at the beginning and end of nursing shifts. The total duration of the study was determined by the availability of funding and human resources.
Testing was performed with appropriate personal protective equipment including surgical mask, face shield, gown and gloves for the research assistant, and surgical mask for the participants. Rapid testing was done according to the manufacturer’s instructions [8] and using the original manufacturer’s swabs and the deep nasal technique. We used the Abbott PanbioTM COVID-19 rapid antigen test kit, which detects the presence of the nucleocapsid protein of SARS-CoV-2 on a membrane-based immunochromatography assay. Testing was administered by research assistants who were trained and evaluated by a physician who delegated the task. The tests were provided by the Ontario Ministry of Health.
The test characteristics of the Panbio rapid antigen test have been reported using real-time PCR as a reference standard. Among studies adherent to the manufacturer’s instructions, sensitivity and specificity in symptomatic patients is 0.741 (0.608-0.840) and 0.998 (0.995-0.999) respectively as reported in a Cochrane review (n = 3699 samples) [9]. Among asymptomatic participants, average sensitivity was lower at 0.581 (0.417-0.729) and specificity was 0.984 (0.922-0.997) (n = 1097 cases) [9].
RESULTS
Testing
In total, 28 participants were enrolled: 19 from the geriatric units and 9 from the forensic units. The mean tests completed per participant was 16.5 (range 4-24), with 462 tests completed during the study. Rapid antigen testing began with the first participant on March 30, 2021. The mean participation time (time from first to last test) for each participant was 46 days (range 13-93). The duration of the study during which screening occurred was 115 days (16.4 weeks), with the final test on July 23, 2021. Each test is plotted in Figures 1 and 2 in relation to the local epidemiological context of COVID-19 cases. No participant reported positive screening at the screening station since their previous study visit. All participants had at least two doses of a COVID-19 vaccine.
Test results
Of the 462 rapid tests performed, one was positive (0.22%). Per protocol, the participant went for confirmatory PCR testing through a local testing centre on the same day as the positive antigen test result (July 14, 2021). The result of the PCR test was negative. The participant was cleared to return to work and to continue the study but elected to withdraw.
Feasibility and tolerability
No participants needed to be withdrawn from the study, for example due to testing positive by PCR for COVID-19. Six of 28 participants (21.4%) dropped out from the study before completion, due to participant preference (n = 2) or being ineligible or unavailable for future study visits (n = 4).
Our original target population were all staff working on the geriatric inpatient units. At the onset of the study, we estimated that there were 108 eligible staff members (including physicians but not trainees whose eligibility was later added). We hoped to recruit as many participants as possible with a target of 80%, but from the original defined population we only recruited 18 (16.7%) despite the recruitment efforts outlined in Methods. We were not able to quantify the total eligible population when the study was expanded.
There were no adverse events in the study. As expected, some participants reported that the testing procedure was uncomfortable.
Background COVID-19 incidence
To roughly estimate the approximate number of cases expected during the study if the staff risk were equal to the population average, an estimate of background COVID-19 incidence was calculated based on the staggered enrolment and participation of participants, the different public health units in which they reside, and the time period each participant was undergoing screening. This was done by taking the sum of the number of new cases reported in the public health unit per person (cases divided by population) during the time they participated in the study. The overall estimate was the sum of these per-person incidence rates. The data was obtained from Ontario Public Health [10].
The estimate for COVID-19 incidence during the study, assuming study participants reflected the average population rates of the public health units in which they reside was
0.23 cases per 28 people.
Outside of our rapid testing procedure, there were no reported staff cases of COVID-19 among participants during their study participation from any sources. Further, there were no reported positive cases of non-participating staff from units who were participating in the study at the time of the units’ participation.
DISCUSSION
We conducted a pilot study to the utility and feasibility of using rapid antigen testing to augment front door screening protocols in our mental health hospital, targeting the highest risk inpatient units. We screened 28 participants and performed 462 rapid antigen tests. Recruitment was well below target (16.7% of the initially targeted population) despite repeated and varied recruitment efforts. One rapid antigen test was positive and follow up confirmatory PCR testing on this test was negative. Six participants dropped out of the study. No cases of COVID-19 were identified on the units during the study among participants and among non-participating staff during the study. Thus, we found that in our context during the study period, poor feasibility is reflected by recruitment challenges and high dropout, and the testing program did not identify any asymptomatic cases of COVID-19.
The study occurred at a period of a waning third wave of COVID-19. At the beginning of the study, COVID-19 incidence in the community was moderate, with the rolling seven-day average peaking around 40-50 cases per 100,000 per day in late April 2021 in Toronto and York Region, and around 50-60 cases per 100,000 per day shortly after in Peel (Figures 1, 2). In all 3 regions, cases quickly dropped off through June and July.
Based on average incidence rates in the respective public health units of our participants, the number of cases expected during their participation is 0.23. However, this estimate is biased by several factors, including the under-ascertainment of cases in the community and the differences in demographics and risk factors of our healthcare worker participants compared to the average population. Further, our study participants had all received a full series of vaccination for COVID-19, potentially reducing their risk of contracting COVID-19 depending on the recency of their most recent dose [7]. These limitations aside, the estimate suggests it is reasonable that a case of COVID-19 was not detected in the study. This is further supported by the fact that no staff positive cases were reported from the units participating even among non-participating staff. This suggests that the actual rate of cases was very low during our study.
While specificity of the Abbott PanbioTM test is high, a low but non-zero rate of false positive results are still expected and have been reported elsewhere. For example, seven of 159 asymptomatic individuals in another study had a positive antigen test with negative PCR. That study reported overall specificity of 0.949 (0.912-0.986) [11]. However, a large study of 824 asymptomatic individuals found specificity of the Abbott PanbioTM rapid test was still excellent (> 0.99) [12].
While sensitivity of the Abbott PanbioTM rapid antigen test is generally good, it is lower in asymptomatic individuals, as applicable to our study. A range of sensitivities is reported for asymptomatic individuals. For example, the sensitivity was only 0.333 (0.196-0.503, n = 296) and 0.545 (0.25-0.84, n = 296) in the asymptomatic cases of two studies respectively [13,14]. A larger (n = 824) study of asymptomatic individuals found the sensitivity to be 0.909 (0.783-0.975) in detecting early infections [12], while another study of asymptomatic case contacts found sensitivity to be 0.481 (0.374-0.589) [15]. Combining symptom and exposure screening (Table 1) with targeted PCR and repeated rapid antigen testing, should be more sensitive than the use of a single rapid test alone.
Interpretation of the positive result
Given the high specificity of the Abbott PanbioTM test, and the imperfect sensitivity of PCR, a positive antigen test with negative PCR test cannot exclude the possibility of COVID-19 with certainty. In other studies, this has been suspected to account for some of the antigen-positive and PCR-negative tests [16]. The lack of a second confirmatory test is a limitation of this study. In our study, the positive antigen result occurred a time when local community transmission was low (< 1 case per 100,000 per day). At that time, the pre-test probability in an asymptomatic person who additionally screens negative to the hospital’s COVID-19-risk screening questions was estimated to be well below 0.1%. Using this as a benchmark and given the average sensitivity and specificity for asymptomatic individuals reported in a Cochrane review, the post-test probability would be 3.5%. Thus, the subsequent negative PCR test in this case would be expected and our positive result was likely a false positive.
Feasibility of implementation
Our recruitment fell well short of our target of 80%. We assumed that interest would be high as participation provided participants with information about their COVID-19 status that may help them navigate risk in their personal lives, in addition to helping them keep their patients and colleagues safe. We attempted to make participation as easy as possible by providing testing on site (in the same building as the participants’ units) at a time of the participants’ selection during or adjacent to business hours, and by being flexible with respect to the frequency of weekly tests (zero to three tests weekly). We advertised the study broadly and directly. We were not able to provide flexible testing times during overnight/evening shifts for nurses and personal support workers.
Anecdotally, we heard that potential participants were concerned about the implications around missing work should they test positive. We also excluded participants who had previously tested positive with COVID-19 and this may have reduced our eligible population by a small number. Future research could include survey information to systematically identify the reasons why a healthcare worker may not want to participate in a voluntary asymptomatic testing program.
The COVID-19 pandemic has had a major impact on our local hospital, as it has to the healthcare system more broadly, including increased burden of work and staffing shortages. Participation in this study may have been perceived as an additional burden for staff resulting in low uptake.
Since our rapid testing study, the guidance on the use of rapid antigen test in Ontario has changed to allow self-swabbing by non-healthcare providers who are trained, which includes those who have watched a training video provided by the Ministry. This allows more flexible timing of testing, and potentially increase feasibility [17]. A study that allowed participants to self-report home rapid test results found that the testing enabled the earlier identification of cases [18].
Since the conclusion of our study, an increase in cases driven by the omicron variant of SARS-CoV-2 led to the highest daily case and hospitalization rates seen in Ontario to date. This wave included high numbers of hospital COVID-19 outbreaks [19]. This context differed in aspects that would impact both the feasibility and uptake and the number of cases identified, if a similar screening program had occurred during this omicron wave. Important differences include higher infectiousness and increased risk of vaccine-breakthrough cases (healthcare workers in our hospital are vaccinated), reduced access to testing in the community as reflected by test positivity increasing to over 30% (potentially increasing the risk of staff having undetected contact with infected individuals who are not aware of their status), and suspected reduced sensitivity of rapid antigen tests to the omicron variant [19].
Other studies evaluating the use of rapid antigen testing to identify asymptomatic SARS-CoV-2 infections in healthcare workers reported challenges with feasibility and limited utility [20]. A large study using rapid antigen tests to screen asymptomatic continuing care healthcare providers found a high false positive rate (30%) with the Abbott PanbioTM test we used in the present study, and a higher false positive rate (70%) using BD Veritor, another antigen-based test. High false positive rates limit the utility of this testing strategy due to the burden on healthcare workers and the healthcare system.
Limitations
We are unable to estimate sensitivity and specificity of the rapid antigen testing used in our study as we only applied PCR testing for a single positive rapid antigen test. Further, the use of PCR as a gold-standard is limited by its imperfect sensitivity. We used the recruitment rate and drop-out rate as measures of feasibility. However, we were not able to hear from those who did not elect to join the study to systematically understand the barriers to participate.
CONCLUSIONS
Our pilot use of rapid antigen testing to augment the hospitals screening for COVID-19 was intended to increase the sensitivity of our overall program to detect COVID-19 on some of our most vulnerable units in our mental health hospital. However, we did not detect any confirmed cases of COVID-19. We had relatively low interest and enrollment in this voluntary compensated research study despite thorough recruitment efforts, suggesting there are barriers to participation we did not systematically identify. Our study did not identify any positive cases of COVID-19 as the single positive rapid antigen test was likely a false positive, and no cases were identified in staff (participants or not) on the participating units during the study.
REFERENCES
1. Xiang, Y.T., Zhao, Y.J., Liu, Z.H., Li, X.H., Zhao, N., Cheung, T., et al. The COVID-19 outbreak and psychiatric hospitals in China: managing challenges through mental health service reform. Int J Biol Sci. 2020 Mar 15;16(10):1741–4.
2. Public Health Ontario. COVID-19 Outbreaks and Cases in Ontario, by Setting: February 16, 2020 to June 12, 2021 [Internet]. Toronto: Ontario Agency for Health Protection and Promotion (Public Health Ontario); 2021 [cited 2021 Oct 26] p. 45. Available from: https://www.publichealthontario.ca/-/media/documents/ncov/epi/ covid-19-settings-based-outbreaks-epi-summary. pdf?sc_lang=en.
3. Arons, M.M., Hatfield, K.M., Reddy, S.C., Kimball, A., James, A., Jacobs, J.R., et al. Presymptomatic SARS-CoV-2 Infections and Transmission in a Skilled Nursing Facility. N Engl J Med. 2020 May 28;382(22):2081–90.
4. Yanes-Lane, M., Winters, N., Fregonese, F., Bastos, M., Perlman-Arrow, S., Campbell, J.R., et al. Proportion of asymptomatic infection among COVID-19 positive persons and their transmission potential: A systematic review and meta-analysis. PLOS ONE. 2020 Nov 3;15(11):e0241536.
5. Moghadas, S.M., Fitzpatrick, M.C., Sah, P., Pandey, A., Shoukat, A., Singer, B.H., et al. The implications of silent transmission for the control of COVID-19 outbreaks. PNAS. 2020 Jul 28;117(30):17513–5.
6. Schwartz, K.L., McGeer, A.J., Bogoch, I.I. Rapid antigen screening of asymptomatic people as a public health tool to combat COVID-19. CMAJ. 2021 Mar 29; 193 (13): E449–52.
7. Bergwerk, M., Gonen, T., Lustig, Y., Amit, S., Lipsitch, M., Cohen, C., et al. Covid-19 Breakthrough Infections in Vaccinated Health Care Workers. N Engl J Med. 2021 Oct 14;385(16):1474–84.
8. Abbott. COVID-19 Ag Rapid Test Device [Internet]. Germany; 2020 [cited 2021 Nov 25]. Available from: https://www.who.int/diagnostics_laboratory/eual/eul_0564_032_00_panbi_covid19_ag_rapid_test_ device.pdf.
9. Dinnes, J., Deeks, J.J., Berhane, S., Taylor, M., Adriano, A., Davenport, C., et al. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2021 Mar 24;2021(3):CD013705.
10. Government of Ontario. Confirmed positive cases of COVID-19 in Ontario – Confirmed positive cases of COVID19 in Ontario – Ontario Data Catalogue [Internet]. [cited 2022 Jan 20]. Available from: https://data.ontario.ca/dataset/f4112442-bdc8-45d2-be3c-12efae72fb27/resource/455fd63b-603d-4608-8216-7d8647f43350.
11. Fenollar, F., Bouam, A., Ballouche, M., Fuster, L., Prudent, E., Colson, P., et al. Evaluation of the Panbio COVID-19 Rapid Antigen Detection Test Device for the Screening of Patients with COVID-19. Journal of Clinical Microbiology. 59(2):e02589-20.
12. Winkel, B., Schram, E., Gremmels, H., Debast, S., Schuurman, R., Wensing, A., et al. Screening for SARS-CoV-2 infection in asymptomatic individuals using the Panbio COVID-19 antigen rapid test (Abbott) compared with RT-PCR: a prospective cohort study. BMJ Open. 2021 Oct;11(10):e048206.
13. Linares, M., Pérez-Tanoira, R., Carrero, A., Romanyk, J., Pérez-García, F., Gómez-Herruz, P., et al. Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms. Journal of Clinical Virology. 2020 Dec 1;133:104659.
14. Masiá, M., Fernández-González, M., Sánchez, M., Carvajal, M., García, J.A., Gonzalo-Jiménez, N., et al. Nasopharyngeal Panbio COVID-19 Antigen Performed at Point-of-Care Has a High Sensitivity in Symptomatic and Asymptomatic Patients with Higher Risk for Transmission and Older Age. Open Forum Infectious Diseases [Internet]. 2021 Mar 1 [cited 2021 Oct 25];8(3). Available from: https://doi.org/10.1093/ofid/ofab059
15. Torres, I., Poujois, S., Albert, E., Colomina, J., Navarro, D. Evaluation of a rapid antigen test (PanbioTM COVID-19 Ag rapid test device) for SARS-CoV-2 detection in asymptomatic close contacts of COVID-19 patients. Clinical Microbiology and Infection. 2021 Apr;27(4):636.e1-636.e4.
16. Stokes, W., Berenger, B.M., Portnoy, D., Scott, B., Szelewicki, J., Singh, T., et al. Clinical performance of the Abbott Panbio with nasopharyngeal, throat, and saliva swabs among symptomatic individuals with COVID-19.
Eur J Clin Microbiol Infect Dis. 2021 Mar 20;1–6.
17. Ontario Health. COVID-19 Health System Response Materials [Internet]. 2021 [cited 2021 Nov 29]. Available from: https://www.ontariohealth.ca/COVID-19/Health-System-Response-Resources#testing-and-assessment-centres
18. Lamb, G., Heskin, J., Randell, P., Mughal, N., Moore, L.S., Jones, R., et al. Real-world evaluation of COVID-19 lateral flow device (LFD) mass-testing in healthcare workers at a London hospital; a prospective cohort analysis. Journal of Infection. 2021 Oct;83(4):452–7.
19. Public Health Ontario. COVID-19 Variant of Concern Omicron (B.1.1.529): Risk Assessment, January 26, 2022. Toronto, Ontario: Ontario Agency for Health Protection and Promotion (Public Health Ontario).; 2022 Jan.
20. Šterbenc, A., Tomi, V., Bidovec Stojkovi, U., Vrankar, K., Rozman, A., Zidarn, M. Usefulness of rapid antigen testing for SARS-CoV-2 screening of healthcare workers: a pilot study. Clin Exp Med. 2022 Feb;22(1):157–60.