ABSTRACT
Background: Ventilator-associated pneumonia (VAP) is a leading cause of antibiotic prescribing and mortality in intensive care units. However, its diagnosis remains challenging because of non-specific clinical signs and subjective surveillance criteria. This study aimed to evaluate the diagnostic accuracy of four epidemiological surveillance algorithms: NHSN/CDC (USA), PNEU CDC (USA), RHOVE (México), and HELICS (Europe).
Bertha Patricia Tijerina-Soto, MD1, Magaly Padilla-Orozco, MD2, and Adrián Camacho-Ortiz, MD, PhD1*
1 Department of Infectious Diseases, Facultad de Medicina y Hospital Universitario (Dr. José Eleuterio González), Universidad Autónoma de Nuevo León, México
2 Epidemiology Coordination, Facultad de Medicina y Hospital Universitario (Dr. José Eleuterio González), Universidad Autónoma de Nuevo León, México
*Corresponding author
Adrián Camacho-Ortiz MD, PhD
Head of Infectious Diseases
Facultad de Medicina y Hospital Universitario
“Dr. José Eleuterio González”
Universidad Autónoma de Nuevo León
Ave. Francisco I. Madero y Gonzalitos, S/N,
Mitras Centro, 64460, Monterrey, NL, México
email: adrian.camachoort@uanl.edu.mx
Article history:
Received 20 March 2026
Received in revised form 15 June 2026
Accepted 17 June 2026
ABSTRACT
Background: Ventilator-associated pneumonia (VAP) is a leading cause of antibiotic prescribing and mortality in intensive care units. However, its diagnosis remains challenging because of non-specific clinical signs and subjective surveillance criteria. This study aimed to evaluate the diagnostic accuracy of four epidemiological surveillance algorithms: NHSN/CDC (USA), PNEU CDC (USA), RHOVE (México), and HELICS (Europe).
Methods: A retrospective observational study was conducted at a tertiary care hospital in northern México between August 2023 and August 2024. We analyzed the clinical records of 102 adult patients who received mechanical ventilation for ≥48 hours. Data collection included demographic, clinical, biochemical, radiological, and microbiological variables. Statistical analyses included sensitivity, specificity, and Cohen’s kappa coefficient to assess agreement among the diagnostic algorithms.
Results: Of the 102 records, 80.3% yielded positive cultures, predominantly Acinetobacter baumannii (47%). VAP was identified in 40% of cases by NHSN/CDC, 54.9% by both PNEU CDC and RHOVE, and 55.8% by HELICS. Using NHSN/CDC as the reference standard, the PNEU CDC, RHOVE, and HELICS algorithms demonstrated 100% sensitivity but lower specificities (73.77%–75.41%). Cohen’s kappa showed substantial agreement between NHSN/CDC and the other methods, with values ranging from 0.68 to 0.69 (p < 0.001). Notably, 45 patients who were clinically diagnosed by physicians did not meet sufficient criteria for a VAP surveillance diagnosis.
Conclusions: There is substantial concordance among the NHSN/CDC, PNEU CDC, RHOVE, and HELICS algorithms for VAP diagnosis. Although the latter three methods are highly sensitive, their lower specificity (average 74.5%) indicates a higher rate of false-positive results compared with the stricter NHSN/CDC criteria. These findings underscore the need to refine surveillance criteria to improve epidemiological comparability.
KEYWORDS:
Ventilator-associated pneumonia, epidemiological surveillance, diagnostic algorithms, ventilator-associated events (VAE), concordance analysis
INTRODUCTION
Ventilator-associated pneumonia (VAP) is clinically defined as a parenchymal lung infection in patients receiving invasive mechanical ventilation (MV) for at least 48 hours (Papazian
et al., 2020) and is the leading reason for antibiotic prescribing in the intensive care unit.
VAP is estimated to affect 5% to 45% of patients receiving mechanical ventilation, with variation depending on the country and the criteria used to classify VAP (American Thoracic Society [ATS] & Infectious Diseases Society of America [IDSA], 2005; Koulenti et al., 2017). It is associated with increased length of hospitalization, morbidity, and mortality (Fihman et al., 2015; Vincent et al., 2009; Rello et al., 2002). Determining the precise incidence is challenging because of the lack of standardized pneumonia definitions, variability in their application, and diagnostic and microbiological limitations (Ego et al., 2015; Eggimann
et al., 2003).
Moreover, in adult patients receiving mechanical ventilation, the accurate identification of VAP is often hindered by the presence of concurrent pulmonary complications such as acute respiratory distress syndrome (ARDS), pulmonary edema, and atelectasis (Klompas, 2007).
VAP surveillance is, therefore, necessary to determine its prevalence and measure the success of prevention efforts. However, the surveillance process is complex and subjective, as the inherent non-specific clinical signs and symptoms contribute to high diagnostic sensitivity. These sources of variability make VAP rates difficult to interpret and compare within and among institutions (Klompas & Platt, 2007; Rahimibashar et al., 2021; Kirtland et al., 1997).
To address these challenges, efforts have been made to develop standardized diagnostic algorithms that incorporate clinical, radiographic, and microbiological data.
To date, several surveillance criteria for VAP have been developed. In the United States, the original VAP surveillance definitions were replaced in 2013 by the National Healthcare Safety Network of the Centers for Disease Control and Prevention (NHSN/CDC) definitions of ventilator-associated events (VAE). The NHSN definitions use more objective and readily measurable criteria to improve reproducibility (Magill et al., 2013; Centers for Disease Control and Prevention [CDC], 2024; Klompas, 2013; Papazian et al., 2020; ATS & IDSA, 2005).
Another surveillance strategy is the PNEU/VAP algorithm, which is based on imaging, clinical, and laboratory criteria. In Mexico, the Hospital Network for Epidemiological Surveillance (RHOVE, for its Spanish acronym), which forms part of the National Epidemiological Surveillance System (SINAVE), published updated definitions for health care-associated infections (HAIs) in 2024, implementing a new algorithm for the diagnosis of health care-associated pneumonia (Table 1). In 2023, RHOVE reported 8,439 cases of VAP, making it the most prevalent health care-associated infection (HAI) in the country. VAP accounted for 14% of all reported HAIs, with an incidence of 14.0 cases per 1,000 mechanical ventilation days. Finally, the Hospital in Europe Link for Infection Control through Surveillance (HELICS) provides another diagnostic algorithm for ventilator-associated pneumonia and has served as a reference standard in previous studies (Rahimibashar et al., 2021; Xie et al., 2011; López-Pueyo et al., 2013). Therefore, the aim of the present study was to evaluate the concordance among these four epidemiological surveillance algorithms (NHSN/CDC, PNEU CDC, RHOVE, and HELICS) for the detection of ventilator-associated pneumonia and to assess their diagnostic accuracy.
METHODS
A retrospective observational study was conducted based on the analysis of clinical records from patients aged 18 years or older who underwent invasive mechanical ventilation (MV) for at least 48 hours and were admitted to a medical/surgical ICU at a tertiary care hospital in northern Mexico between August 19, 2023 and August 31, 2024. During the study period, 745 patients admitted to the ICU received MV for more than 48 hours. Over this period, the institution recorded an overall VAP rate of 18.3 cases per 1,000 ventilator-days. Patient selection and medical record review were performed independently by an enrolment team of two infectious diseases physicians. In cases of disagreement, the opinion of a third evaluator was sought to reach a consensus. Participants were included if they had a diagnosis of ventilator-associated pneumonia (VAP) documented either by the Hospital Epidemiological Surveillance Unit, which uses the RHOVE algorithm for case identification and classification or by the treating physician during the hospital stay, as documented in the medical record. For the analysis, only the first episode of VAP experienced by each patient was considered.
The data collection tool consisted of a two-part checklist that included demographic variables; clinical variables (cough, shortness of breath, increased respiratory secretions, respiratory rate, heart rate, temperature, rales on auscultation, ventilator parameters such as positive end-expiratory pressure [PEEP] and fraction of inspired oxygen [FiO2], and initiation of antibiotic treatment); biochemical variables (total leukocyte count and C-reactive protein); radiological findings from chest radiographs or computed tomography (CT) scans; and microbiological variables.
Positive quantitative cultures of lower respiratory tract secretions were identified from the patient medical records. A positive result was defined as ≥104 colony-forming units per millilitre (CFU/mL) for bronchoalveolar lavage (BAL) samples and ≥105 CFU/mL for tracheobronchial aspirates. Bacterial isolates were identified by matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-TOF MS; Microflex LT system, Bruker Daltonics, Bremen, Germany) as part of routine clinical practice.
Ethics approvStatistical analysisal
The Research and Ethics Committees of Hospital Universitario
“Dr. José Eleuterio González” approved the study (registration number IF25-00001).
Frequencies and percentages were used to describe categorical variables. Continuous variables with a normal distribution were expressed as mean and standard deviation (SD), whereas non-normally distributed variables were presented as median and interquartile range (IQR). In addition, 2 × 2 contingency tables were used to calculate sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratios. Agreement among the evaluated diagnostic algorithms was assessed using Cohen’s kappa coefficient. Data were analyzed using SPSS version 31.0 (IBM Corp., Armonk, NY, USA).
RESULTS
A total of 102 clinical records from patients diagnosed with VAP by the attending physician were evaluated. The mean age was 45.6 ± 12 years, and most patients were male (68.6%). Positive cultures were identified in 80.3% of cases, with Acinetobacter baumannii (47%), Pseudomonas aeruginosa (28.4%), Klebsiella spp. (14.7%), and Stenotrophomonas maltophilia (8.8%) being the most frequently isolated organisms. Multidrug-resistant (MDR) organisms were identified in 49% of isolates.
Overall, 57 patients met the criteria for at least one surveillance algorithm for ventilator-associated pneumonia. Specifically, 41 patients (40%) met the ventilator-associated event (VAE) definition according to the National Healthcare Safety Network of the Centers for Disease Control and Prevention (NHSN/CDC) algorithm, 56 (54.9%) met the PNEU CDC and Hospital Network for Epidemiological Surveillance (RHOVE) criteria, and 57 (55.8%) met the Hospital in Europe Link for Infection Control through Surveillance (HELICS) criteria.
In addition, 45 patients were clinically diagnosed with VAP by the attending physician but did not meet sufficient criteria under any of the surveillance algorithms. This was primarily due to the absence of radiographic progression and the lack of microbiological isolation on culture. For the purposes of this analysis, these patients were classified as false-positive cases (Table 2).Using the NHSN/CDC criteria as the reference standard, the sensitivity and specificity of the evaluated diagnostic algorithms were as follows: PNEU CDC (sensitivity 100%; specificity 75.41%), RHOVE (sensitivity 100%; specificity 75.41%), and HELICS (sensitivity 100%; specificity 73.77%) (Table 3).
Agreement between the NHSN/CDC algorithm and the other diagnostic methods was assessed using Cohen’s kappa coefficient. The agreement between NHSN/CDC and PNEU CDC yielded a kappa value of 0.69 (p < 0.001), as did the agreement between NHSN/CDC and RHOVE (K = 0.69, p < 0.001). Agreement between NHSN/CDC and HELICS yielded a kappa value of 0.68 (p < 0.001). These findings indicate substantial agreement among the evaluated diagnostic methods for case classification (Table 2).
DISCUSSION
Historically, the diagnosis of ventilator-associated pneumonia (VAP) has been based on a tripartite assessment of clinical, radiological, and microbiological parameters (Tejerina et al., 2010).
The inherent challenges of clinical diagnosis are mainly attributable to the non-specific signs and symptoms, which frequently overlap with other complications associated with mechanical ventilation, including pulmonary edema and atelectasis (Vincent et al., 2009; Klompas, 2007; Rahimibashar et al., 2021).
Evaluating the correlation and diagnostic performance of VAP surveillance algorithms is crucial for ensuring accurate epidemiological assessments and facilitating robust comparisons across studies (Waltrick et al., 2015; Ego et al., 2015).
The agreement among the surveillance algorithms observed in our study is consistent with previous reports. A prospective study comparing the Johanson criteria, the Clinical Pulmonary Infection Score (CPIS), and the NHSN/CDC definitions against HELICS reported that, using HELICS as the reference standard, the sensitivity and specificity of the evaluated diagnostic algorithms were as follows: NHSN/CDC (sensitivity 54.2%; specificity 100%), CPIS (sensitivity 68.75%; specificity 95.23%), and Johanson criteria (sensitivity 67.69%; specificity 95%) (Rahimibashar et al., 2021).
Another study of 168 ICU patients evaluated the agreement between CPIS, using a score of ≥7 points to diagnose VAP, and the NHSN/CDC criteria. The NHSN/CDC method demonstrated high specificity (100%), a positive predictive value (PPV) of 100%, and a negative predictive value (NPV) of 84%, but relatively low sensitivity (37%). These differences resulted in substantial discrepancies in the estimated incidence density (NHSN/CDC: 5.2 per 1,000 mechanical ventilation days; CPIS >7 points: 13.1 per 1,000 mechanical ventilation days) (Waltrick et al., 2015).
These studies concluded that the low sensitivity of the NHSN/CDC method for identifying VAP was attributable to its stringent criteria, including higher thresholds for positive end-expiratory pressure (PEEP) and fraction of inspired oxygen (FiO2), as well as the requirement for a 48-hour period of ventilatory stability, which is not observed in all patients with VAP (Rahimibashar et al., 2021; Waltrick et al., 2015).
Additionally, the international EUVAE cohort examined the relationship between ventilator-associated events (VAE) and VAP and found that only 59% of patients with suspected pulmonary infection met the criteria for possible ventilator-associated pneumonia (PVAP). This finding suggests that the VAE algorithm identifies only the most severe cases and excludes patients who do not experience sufficient deterioration in oxygenation to meet the diagnostic thresholds based on ventilator support requirements (Tejerina et al., 2010).
In the present study, concordance analysis using Cohen’s kappa coefficient demonstrated substantial agreement between the NHSN/CDC algorithm and the PNEU CDC, RHOVE, and HELICS criteria. The kappa values (0.68–0.69) indicate agreement beyond chance, suggesting that these algorithms classify cases similarly. The statistical significance of these findings (p < 0.001) further supports the robustness of the observed agreement.
Nonetheless, although the PNEU CDC, RHOVE, and HELICS algorithms demonstrated 100% sensitivity relative to the NHSN/CDC reference standard, their specificities ranged from 73.77% to 75.41%. This translates to a false-positive rate of 24.59% to 26.23%, indicating that a considerable proportion of patients without VAP according to the reference standard would be classified as positive by these algorithms. These findings underscore the inherent trade-offs in surveillance design: highly specific algorithms such as NHSN/CDC, minimize subjectivity but may underestimate the true clinical burden of VAP.
Our findings underscore the importance of comparative evaluations of VAP surveillance algorithms. Although substantial concordance was observed among the evaluated methods, meaningful differences in specificity remained. These observations highlight the need for further research to refine surveillance criteria and improve the consistency and reproducibility of VAP surveillance across institutions.
This study has several limitations that should be acknowledged. First, the retrospective design is susceptible to selection bias and residual confounding. Second, the single-centre setting may limit the generalizability of the findings. Finally, interobserver reproducibility of the algorithm assessments was not evaluated.
CONCLUSION
Our findings demonstrate substantial concordance among the NHSN/CDC, PNEU CDC, RHOVE, and HELICS algorithms for the diagnosis of ventilator-associated pneumonia. Using the NHSN/CDC algorithm as the reference standard, the evaluated algorithms demonstrated 100% sensitivity and an average specificity of 74.5%.
REFERENCES
American Thoracic Society, & Infectious Diseases Society of America. (2005). Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. American Journal of Respiratory and Critical Care Medicine, 171(4), 388–416 https://doi.org/10.1164/rccm.200405-644ST
Centers for Disease Control and Prevention. (2024, January). Pneumonia (ventilator-associated [VAP] and non-ventilator-associated pneumonia [PNEU]) event. National Healthcare Safety Network.
Ego, A., Preiser, J. C., & Vincent, J. L. (2015). Impact of diagnostic criteria on the incidence of ventilator-associated pneumonia. Chest, 147(2), 347–355. https://doi.org/10.1378/chest.14-0610
Eggimann, P., Hugonnet, S., Sax, H., Touveneau, S., Chevrolet, J.-C., & Pittet, D. (2003). Ventilator-associated pneumonia: Caveats for benchmarking. Intensive Care Medicine, 29(11), 2086–2089. https://doi.org/10.1007/s00134-003-1991-9
Fihman, V., Messika, J., Hajage, D., Tournier, V., Gaudry, S., Magdoud, F., Barnaud, G., Billard-Pomares, T., Branger, C., Dreyfuss, D., & Ricard, J. D. (2015). Five-year trends for ventilator-associated pneumonia: Correlation between microbiological findings and antimicrobial drug consumption. International Journal of Antimicrobial Agents, 46(5), 518–525. https://doi.org/10.1016/j.ijantimicag.2015.07.010
Kirtland, S. H., Corley, D. E., Winterbauer, R. H., Springmeyer, S. C., Casey, K. R., Hampson, N. B., & Dreis, D. F. (1997). The diagnosis of ventilator-associated pneumonia: A comparison of histologic, microbiologic, and clinical criteria. Chest, 112(2), 445–457. https://doi.org/10.1378/chest.112.2.445 Klompas, M. (2007).
Does this patient have ventilator-associated pneumonia? JAMA, 297(14), 1583–1593.https://doi.org/10.1001/jama.297.14.1583
Klompas, M. (2013). Complications of mechanical ventilation— The CDC’s new surveillance paradigm. New England Journal of Medicine, 368(16), 1472–1475.
https:// & Platt, R. (2007). Ventilator-associated pneumonia—the wrong quality measure for benchmarking. Annals of Internal Medicine, 147(11), 803–805.
https://doi.org/10.7326/0003-4819-147-11-200712040-00013
Koulenti, D., Tsigou, E., & Rello, J. (2017). Nosocomial pneumonia in 27 ICUs in Europe: Perspectives from the EU-VAP/CAP study. European Journal of Clinical Microbiology & Infectious Diseases, 36(11), 1999–2006. https://doi.org/10.1007/s10096-016-2703-z
López-Pueyo, M. J., Olaechea-Astigarraga, P., Palomar-Martínez, M., Insausti-Ordeñana, J., Alvarez-Lerma, F., & ENVIN–HELICS Study Group. (2013). Quality control of the surveillance programme of ICU-acquired infection (ENVIN-HELICS registry) in Spain. Journal of Hospital Infection, 84(2), 126–131. https://doi.org/10.1016/j.jhin.2013.02.018
Magill, S. S., Klompas, M., Balk, R., Burns, S. M., Deutschman, C. S., Diekema, D., Fridkin, S., Greene, L., Guh, A., Gutterman, D., Hammer, B., Henderson, D., Hess, D., Hill, N. S., Horan, T., Kollef, M., Levy, M., Septimus, E., VanAntwerpen, C., Wright, D., & Lipsett, P. (2013). Developing a new, national approach to surveillance for ventilator-associated events. Critical Care Medicine, 41(11), 2467–2475. https://doi.org/10.1097/CCM.0b013e3182a262db
Papazian, L., Klompas, M., & Luyt, C. E. (2020). Ventilator-associated pneumonia in adults: A narrative review. Intensive Care Medicine, 46(5), 888–906.
https://doi.org/10.1007/s00134-020-05980-0
Rahimibashar, F., Miller, A. C., Yaghoobi-Azari, M. H., & Vahedian-Azimi, A. (2021). A comparison of diagnostic algorithms and clinical parameters to diagnose ventilator-associated pneumonia: A prospective observational study. BMC Pulmonary Medicine, 21(1), Article 161.https://doi.org/10.1186/s12890-021-01527-1
Ramírez-Estrada, S., Lagunes, L., Peña-López, Y., Vahedian-Azimi, A., Nseir, S., Arvaniti, K., Bastug, A., Totorika, I., Oztoprak, N., Bouadma, L., Koulenti, D., Rello, J., & EUVAE Study Investigators Group. (2018). Assessing predictive accuracy for outcomes of ventilator-associated events in an international cohort: The EUVAE study. Intensive Care Medicine, 44(8), 1212–1220. https://doi.org/10.1007/s00134-018-5269-7
Rello, J., Ollendorf, D. A., Oster, G., Vera-Llonch, M., Bellm, L., Redman, R., Kollef, M. H., & VAP Outcomes Scientific Advisory Group. (2002). Epidemiology and outcomes of ventilator-associated pneumonia in a large US database. Chest, 122(6), 2115–2121. https://doi.org/10.1378/chest.122.6.2115
Tejerina, E., Esteban, A., Fernández-Segoviano, P., Frutos-Vivar, F., Aramburu, J., Ballesteros, D., & Rodríguez-Barbero, J. M. (2010). Accuracy of clinical definitions of ventilator-associated pneumonia: Comparison with autopsy findings. Journal of Critical Care, 25(1), 62–68. https://doi.org/10.1016/j.jcrc.2009.05.008
Vincent, J. L., Rello, J., Marshall, J., Silva, E., Anzueto, A., Martin, C. D., Moreno, R., Lipman, J., Gomersall, C., Sakr, Y., Reinhart, K., & EPIC II Group of Investigators. (2009). International study of the prevalence and outcomes of infection in intensive care units. JAMA, 302(21), 2323–2329. https://doi.org/10.1001/jama.2009.1754
Waltrick, R., Possamai, D. S., de Aguiar, F. P., Dadam, M., de Souza Filho, V. J., Ramos, L. R., Laurett, R. S., Fujiwara, K., Caldeira Filho, M., Koenig, Á., & Westphal, G. A. (2015). Comparison between a clinical diagnosis method and the surveillance technique of the Center for Disease Control and Prevention for identification of mechanical ventilator-associated pneumonia. Revista Brasileira de Terapia Intensiva, 27(3), 260–265. https://doi.org/10.5935/0103-507X.20150047
Xie, D. S., Xiong, W., Lai, R. P., Liu, L., Gan, X. M., Wang, X. H., Wang, M., Lou, Y. X., Fu, X. Y., Wang, H. F., Xiang, H., Xu, Y. H., & Nie, S. F. (2011). Ventilator-associated pneumonia in intensive care units in Hubei Province, China: A multicentre prospective cohort survey. Journal of Hospital Infection, 78(4), 284–288. https://doi.org/10.1016/j.jhin.2011.03.009