Announcements
The new Eurosurveillance website is almost here.

Eurosurveillance is on the updated list of the Directory of Open Access Journals and in the SHERPA/RoMEO database. Read more here.

On 6 June 2017, the World Health Organization (WHO) published updates to its ‘Essential Medicines List’ (EML). Read more here.

Follow Eurosurveillance on Twitter: @Eurosurveillanc


In this issue


Home Eurosurveillance Edition  2017: Volume 22/ Issue 11 Article 3
Back to Table of Contents
Previous Download (pdf)
Next

Eurosurveillance, Volume 22, Issue 11, 16 March 2017
Research article
Natale, Stelling, Meledandri, Messenger, and D'Ancona: Use of WHONET-SaTScan system for simulated real-time detection of antimicrobial resistance clusters in a hospital in Italy, 2012 to 2014

+ Author affiliations


Citation style for this article: Natale A, Stelling J, Meledandri M, Messenger LA, D'Ancona F. Use of WHONET-SaTScan system for simulated real-time detection of antimicrobial resistance clusters in a hospital in Italy, 2012 to 2014. Euro Surveill. 2017;22(11):pii=30484. DOI: http://dx.doi.org/10.2807/1560-7917.ES.2017.22.11.30484

Received:01 February 2016; Accepted:21 August 2016


Introduction

Antimicrobial resistance (AMR) is considered a public health threat as it is increasingly hampering effective treatment of bacterial and fungal diseases worldwide [1,2]. According to the Global Report on Surveillance of Antimicrobial Resistance, rates of resistance are increasing in all World Health Organization (WHO) Regions in pathogens causing infections in both healthcare and community settings [3]. A better strategy to enhance surveillance and strengthen collaborations at a global level is needed in order to coordinate efficient control strategies and to complete the current gaps in surveillance caused by lack of standard methodologies for data collection and failure of data sharing at local, national and international levels [4].

Despite multiple efforts for harmonisation and centralisation of clinical data, lack of data standardisation and poor data accessibility still constitute a worldwide problem. There is also a current need for a standardised interpretation of microbiology data as exemplified by the recent breakpoint harmonisation process promoted by the European Committee on Antimicrobial Susceptibility Testing (EUCAST) [5]. Clinical microbiology reports represent an important resource for the detection of ongoing dissemination of resistant (and even susceptible) pathogens. In spite of this, they are often underutilised not only at local hospital level, but also in national surveillance systems or across countries [6].

With the aim to centralise and coordinate European surveillance of AMR, the European Centre for Disease Prevention and Control (ECDC) coordinates the European Antimicrobial Resistance Surveillance Network (EARS-Net), a network of national surveillance systems [7]. This network collects routine clinical antimicrobial susceptibility data from 28 European Union (EU) and two European Economic Area (EEA) countries (Norway and Iceland) concerning invasive isolates (blood and cerebrospinal fluid, CSF) of eight organisms considered of public health concern [8]. This network has promoted the regular collection of clinical data in the participating countries and further highlighted the need of a standardised data format. To address such a need, and to facilitate data sharing, the WHO Collaborating Centre for Surveillance of Antimicrobial Resistance based at the Brigham and Women’s Hospital and Harvard Medical School, Boston (United States) developed a software to manage microbiology test results, the WHONET software [9], free to download, (www.whonet.org/software.html) that allows data entry into a standard format, or via BacLink utility software, a conversion tool [10]. Thanks to the software’s automated data entry and its ability to handle large datasets as well as to rapidly generate trends and patterns, WHONET has become the official component of many national surveillance programmes and is now used as a support tool in up to 120 WHO member states [11].

As a further application, WHONET has embedded the free SaTScan software (www.satscan.org) developed by Martin Kulldorff together with Information Management Services, Inc. and supported by various United States’ National Institutes of Health for the detection of spatial and temporal data clustering, using spatial, temporal or space-time scan statistics [12]. This algorithm is designed to evaluate random distribution or spatial and temporal clustering of diseases and to test their statistical significance, applied to surveillance of diseases and their geographic/spatial determinants or prospectively to timely detect outbreaks [13,14]. In combination, the WHONET-SaTScan system allows for timely detection of clusters of AMR pathogens in space and time facilitating outbreak investigations locally in a single hospital [15], in the community [16], or at national scale for real-time surveillance purposes [17]. The system also enables to study transmission of resistance between wards [18].

In this study, the WHONET-SaTScan software was applied for the first time within the Italian surveillance system. Since 2001, Italy has in place a national antibiotic resistance surveillance project coordinated by the Istituto Superiore di Sanità (AR-ISS), based on sentinel microbiological laboratories, integrating more than 50 hospitals throughout the country. Approximately 20 laboratories have been part of a sub-network called MICRONET and until the end of 2014, automatically submitted clinical data every night to a central server [19]. MICRONET included clinical data for all bacterial pathogens and all kind of samples. Furthermore, in Italy, the WHONET-BacLink software was already used at national level to aggregate and analyse data collected from all the laboratories belonging to the AR-ISS, making the Italian system ideal for the application of the WHONET-SaTScan system. In this work, data collected retrospectively between 2012 and 2014 from one hospital in Rome were analysed using a simulated prospective method to detect statistically significant clusters of pathogens of public health importance. The alerts generated by this method were then compared with the ones generated by the detection method currently in place in the hospital to assess the validity of the WHONET-SaTScan for a possible future implementation within the surveillance of AMR in a real-time and predictive manner.

Methods

Setting of the study

San Filippo Neri Hospital (SFNH) is a public hospital, with predominant surgical activity, located in the northern urban area of Rome with a capacity of 457 beds. Control and response to infections are responsibility of a hospital infections control team (Commissione Prevenzione e Controllo delle Infezioni Ospedaliere, CPCIO), composed of clinicians, microbiologists and virologists, infection preventionists, pharmacists and nurses. The CPCIO is coordinated by the hospital’s health manager, and collects microbiology data, detects epidemiological alerts and implements standardised control measures within the hospital. A procedure called ‘EpiD’ is activated when the definition of an outbreak is met (‘three or more samples of the same organism, isolated from three different patients within 5 days in the same operating unit’; ‘3 by 5’ rule) and containment measures are then set in place.

Extraction of microbiology data and susceptibility test results

Microbiology data were extracted from the MICRONET database, using date of test request as main parameter and setting restrictions to location (SFNH) and time (between January 2011 and the most recent data available at the time of the study, i.e. 30 May 2014). Data fields extracted included laboratory identity (ID), patient ID, sex, date of birth, age, pathogen type, ward, institution code, department, ward type, specimen number, specimen date, specimen type, specimen code, isolate number, admission date and susceptibility test results which were further described qualitatively as resistant (R), intermediate (I) and susceptible (S) based on minimum inhibitory concentration (MIC) test results and assigned as per EUCAST breakpoints [20]. Data were then converted to WHONET compatible format using the BacLink software. During the conversion the dataset was restricted to the first isolate per patient – including outpatients and inpatients admitted to the hospital any time before specimen collection (i.e. with no distinction between hospital-acquired or community infections) – over a 365 days period and all R and I results were combined as ‘non susceptible’ (NS) for purposes of resistance phenotype analysis.

Resistance profiles were adapted to this setting by choosing a panel of antibiotics for the main groups of pathogens, according to SFNH’s frequency of performed/reported antimicrobial tests per each group. The number of tests was obtained by performing a per cent resistant-intermediate-susceptible (%RIS) analysis on a sample of data from January to June 2013, assuming consistency of testing protocols across years. A 75% frequency was chosen as cut-off value.

Statistical analysis

The SaTScan cluster detection tool integrated into the WHONET software was used to retrospectively identify clusters of antimicrobial resistant pathogens in SFNH. SaTScan can identify clusters of cases in terms of spatial only, temporal only, or combined spatial and temporal distributions. In this work, we used the SaTScan space-time permutation scan statistics for the evaluation of the statistical significance of identified clusters [14]. In this analysis, the temporal parameter was the ‘specimen date’ while the spatial parameters included a specific location within the hospital, such as the actual ‘ward’ or a group of wards with communal care characteristic defined as ‘service’. Non-spatial variables were the ‘pathogen type’ or ‘resistance profile’ based on antibiotic susceptibility test results. Clusters were identified using the categorical variables ‘pathogen type’, ‘resistance profile’, ‘ward’ and ‘service’ plus a combination of such variables. The statistical significance of clusters was evaluated by a Monte-Carlo maximum likelihood test using SatScan’s space-time permutation model. The parameters chosen for this analysis had been already assessed in previous studies [15,17]. A maximum cluster length of 60 days cut-off was chosen, corresponding to the maximum temporal scanning window size for signal generation. The statistical likelihood of signals is determined by the recurrence interval, which corresponds to the inverse of the p-value, expressed in days, signifying the time during which a similar signal would occur by random variation only. In this study, only clusters with a recurrence interval of > 365 days were included in the analysis. The baseline parameter (i.e. the temporal baseline preceding the maximum temporal window against which is compared) was set to 365 days. Thus data from 2011 were considered exclusively as baseline data (as they contributed to the first 365 days of the baseline) for the subsequent 2012 time period, and any clusters detected in 2011 were not included in the analysis.

Dataset generation and comparison of WHONET-SaTScan results with the SFNH infection prevention and control system

Overlapping signals generated by the WHONET-SaTScan analysis were combined into a single ‘signal cluster’. In particular, clusters including more information (more types of signal at the same time), more epidemiologically relevant (in terms of duration, number of cases etc.) and with higher recurrence interval, were chosen as representative clusters provided by the system. Cluster summary and cluster detail tables were generated and line listings of all the isolates involved in the alerts were also produced. The summary table of the alerts compiled by WHONET-SaTScan was compared with the CPCIO official list of microbiology alerts from 2012 to 2013 and an extract of the semester report of 2014. Because the CPCIO’s analysis of the alert reports from previous years revealed that more than 75% of all episodes within the hospital were caused by three pathogens: Clostridium difficile, multidrug resistant (MDR) Acinetobacter baumannii and carbapenem-resistant Klebsiella pneumoniae, the latest hospital reports, including the ones covered in our study, were restricted to such pathogens. Moreover, as C. difficile was not included in the SaTScan-WHONET list of organisms at the time of this study, our comparison could only be based on A. baumannii and K. pneumoniae.

A questionnaire, adapted from a Brigham and Women’s Hospital’s, was used to assess whether there were any clusters detected by the WHONET-SaTScan of epidemiological or clinical importance. These alerts were further classified according to the level of concern caused (1 – no concern, disregard; 2 – low concern, await more cases; 3 – moderate concern, action; 4 – high concern, action) and for moderate and high concern, on the type of action (1 – notify other members of the CPCIO to increase awareness; 2 – assess background frequency of organism; 3 – start investigating by assessing medical records to find a common source; 4– activate containment measures). The questionnaire was completed by the head of the microbiology and virology laboratory who was a member of the CPCIO at the time of this study.

Results

Dataset

The microbiology dataset from SFNH collected from the beginning of 2011 to the end of May 2014 included a total of 11,777 samples, of which 7,994 from 2012 and 2014 were included in the final analysis, while 3,783 from 2011 were used as baseline data only. Specimen types were mainly urine (37.2%), pus (20.0%), and blood (11.2%). Table 1 depicts a summary of isolates’ characteristics between 2012 and 2014. Overall, isolates included 139 species, the most common being Escherichia coli (n = 2,092, 26.2%), Staphylococcus aureus (n = 742, 9.3%), Enterococcus faecalis (n = 656, 8.2%), K. pneumoniae (n = 554, 6.9%) and Pseudomonas aeruginosa (n = 506, 6.3%).

Table 1

Characteristics of isolates from San Filippo Neri Hospital extracted from MICRONET, Italy, January 2012–May 2014 (n = 7,994 isolates)

Isolates characteristics Number of isolates Percentage
Year
2012     3,419     42.7
2013     3,327     41.7
2014a     1,248     15.6
Sex
Female     4,340     54.3
Male     3,616     45.2
Missing information     38     0.5
Specimen type
Urine     2,972     37.2
Pus     1,598     20.0
Blood     893     11.2
Tracheal aspirate     578     7.2
Vaginal swab     367     4.6
Cervical test     254     3.2
Sputum     238     3.0
Aspirateb     203     2.5
Nasal swab     168     2.1
Throat swab     161     2.0
Others     562     7.0
Organism group
Gram-negative     4,483     56.0
Gram-positive     2,984     37.3
Mycoplasma     272     3.4
Anaerobe     183     2.3
Fungi     57     0.7
Mycobacterium (non tuberculosis)     15     0.2
Department of origin
Outpatient     2,720     34.0
Medicine     1,970     24.6
Surgery     1,767     22.1
Intensive/intermediate care unit     1,108     13.8
Obstetric/gynaecology     160     2.0
Neonatology     121     1.5
Haematology/oncology     85     1.1
Emergency     49     0.6
Psychiatry     14     0.2

a Data are from the first 5 months of 2014 only.

b Aspirates other than tracheal aspirates.

Signals created by WHONET-SaTScan

The WHONET-SaTScan analysis generated a total of 287 signals from 2012 to 2014 grouped into 90 ‘cluster summaries’, among which some, overlapping in the spatial components of service/ward and resistance phenotype, were further merged manually into 71 final clusters. Table 2 shows the summary characteristics of the final 71 clusters. Of these: 18 were caused by E. coli strains mostly fully susceptible to all antibiotics except for three, one of which being an extended-spectrum beta-lactamase (ESBL) strain; 13 by E. faecalis with different combinations of resistance phenotypes; seven by K. pneumoniae, one of which with resistance to four different classes of antibiotics and one in the intensive care unit (ICU) caused by a carbapenem-resistant strain; four by P. aeruginosa, one in ICU by a possible extensive drug-resistant (XDR) strain; three by S. aureus, one being a meticillin-resistant S. aureus (MRSA) strain; two by A. baumannii, one of which involving 13 cases of an XDR organism over two months; lastly, two by Enterococcus faecium including one by an MDR strain and the other including two cases of a vancomycin-resistant (VRE) strain in a neonatology ward.

Table 2

Summary characteristics of clusters generated by WHONET-SatScan in San Filippo Neri Hospital, Italy, 2012–2014 (n = 71 clusters)

Cluster characteristics Number Percentagea
Total number 71 100
Average number of clusters per month 4.5 NA
Year
2012 17         24.0
2013 42         59.1
2014 12         16.9
Pathogen type
Escherichia coli 18 25.4
Enterococcus faecalis 13 18.3
Klebsiella pneumoniae 7 9.9
Pseudomonas aeruginosa 5 7.0
Staphylococcus aureus 4 5.6
Acinetobacter baumannii 2 2.8
Other 22 31.0
Type of alerts
Ward and resistance profile 24 33.8
Resistance profile 21 29.5
Serviceb and resistance profile 16 22.5
Serviceb 4 5.7
Pathogen type 4 5.7
Ward 2 2.8
Mean number of signals per cluster (95% CI) 1.73 (1.53–1.93)
Number of cases
Total 700 100
Median per cluster (range) 4 (2–143)
Cluster length in days
1 10 14.1
2–5 17 24.0
6–10 10 14.1
11–50 21 29.5
> 50 13 18.3

CI: confidence interval; NA: not applicable.

a Unless otherwise specified in the row heading.

b A group of wards with communal care is defined as ‘service’.

Comparison of alerts generated by WHONET-SaTScan with the hospital response system

In order to assess the validity of the method we compared the signal alerts generated by our analysis with the ‘EpiD’ procedure activated by the CPCIO. The total number of potential outbreaks detected by the WHONET-SaTScan system per year was higher than the number of activated ‘EpiD‘ (respectively, including C. difficile in ‘EpiD’, 17 vs 4 in 2012, 42 vs 6 in 2013 and 12 vs 4 in 2014). Table 3 summarises the comparison between the two systems, by year. In 2012, of two alerts detected by CPCIO (i.e. two activated ‘EpiD’), only the one involving K. pneumoniae is possibly in common between the two systems. However, this cluster was detected by WHONET-SaTScan in a different ward (outpatient) than by the CPCIO (which found the cluster in the ICU) and at a later time. As the CPCIO detected the K. pneumoniae cluster 11 days earlier, this outbreak was probably contained as result of the activation of the ‘Epid’ procedure. In 2013, three outbreaks were detected with a 100% agreement between the CPCIO and WHONET-SaTScan; one of these outbreaks involved A. baumannii in a cluster of long duration, which lasted from 20 May 2013 to 1 August 2013 with a recurrence interval of 2 years. This large outbreak, however, included a smaller signal outbreak clustered by service and resistance between 20 May 2013 and 25 June 2013 in the ICU with recurrence interval of 2.75 years (more rare) probably corresponding to the same signal that activated a response within the hospital. The signal of this cluster as generated by the WHONET-SaTScan is shown in the Figure. The other two outbreaks were caused by K. pneumoniae and seem to have activated the ‘EpiD’ procedure only months after the start of the outbreak, according to WHONET-SaTScan. In 2014, there was no official report from the hospital at the time of the study, and the alerts we could obtain were only from incomplete reports. However, none of the CPCIO alerts was detected by WHONET-SaTScan.

Table 3

Comparison between Acinetobacter baumannii and Klebsiella pneumoniae alerts detected by San Filippo Neri’s CPCIO and WHONET-SaTScan systems, Italy, 2012–2014

Year Organism Detected by the CPCIO Detected by WHONET-SaTScan Agreement between the two systems
(%)
Number of alerts Date of activation Ward Number of casesa Number of alerts Start date Ward Number of cases
2012 Acinetobacter baumannii MDR 1 14 Aug ICU ≥ 3 0 NA NA NA 50
Klebsiella pneumoniae
MDR
1 11 Sep ICU ≥ 3 1 22 Sep OUT 2
2013 Acinetobacter baumannii
XDR
1 27 Jun ICU ≥ 3 1 20 May NSW 13 100
Klebsiella pneumoniae
MDR
1 11 Oct ICU ≥ 3 1 6 May ICU 6
Klebsiella pneumoniae
KPC
1 27 Nov ICU ≥ 3 1 3 Aug ICU 3
2014 Klebsiella pneumoniae
KPC
1 Aprb ICU ND 0 NA NA NA NA

CPCIO: Commissione Prevenzione e Controllo delle Infezioni Ospedaliere (hospital infections control team); ICU: intensive care unit; KPC: Klebsiella pneumoniae carbapenem-resistant; MDR: multidrug resistant; NA: not applicable; ND: no data; NSW: no specific ward; OUT:  outpatient ward; XDR: extensive drug-resistant.

a The number of cases detected by the CPCIO is at least three to trigger the activation of control response as per outbreak definition (see text for details).

b The exact date of activation was not available at the time of this study and only an unofficial report from 2014 was available.

Figure

Distribution of Acinetobacter baumannii XDR cases and cluster alerts, San Filippo Neri Hospital, Italy, January 2013–May 2014

/images/dynamic/articles/22740/16-00085-f1

XDR: extensive drug-resistant.

The figure shows number of cases per month and the empty bars represent the signals that generated the alert in 2013 by WHONET-SaTScan; the sharp increase in number of cases in June 2013 compared with the previous months is statistically significant (see Table 4). The cases in July and August are part of the same cluster, as detected by the SaTScan temporal window and possibly part of the same epidemiological cluster.

Evaluation of the alerts generated by WHONET-SaTScan

To assess the benefit of WHONET-SaTScan we asked a member of the CPCIO to evaluate the alerts generated by this system. Table 4 shows the alerts considered worth knowing by the hospital and by type of crucial characteristic (pathogen type, resistance phenotype or location). Of the 71 clusters detected by the WHONET-SaTScan analysis, 18 were considered of importance, of which only three had been initially detected by the CPCIO. The majority of these clusters were deemed relevant for the hospital because of the combined characteristics of pathogen type, resistance phenotype and location (n = 8, 44%), but also for their location alone (n = 4, 22%). Of these 18 clusters, including the ones detected by the CPCIO, six were considered of low concern, eight of moderate concern and four of high concern. For the eight alerts of moderate concern only two of the four types of possible actions were activated (i.e. 1 – notification of other members of CPCIO and 4 – start response measures), while the four alerts of high concern would trigger all four types of action. Among the high concern alerts, one caused by an E. faecium VRE strain in June 2013 and one by a P. aeruginosa MDR/XDR strain in July 2013, occurred completely undetected by the CPCIO.

Table 4

Cluster alerts detected by WHONET-SaTScan in San Filippo Neri Hospital considered relevant by the hospital’s infection control system (CPCIO) and critical characteristics of the alerts for the evaluation, Italy, 2012–2014 (n = 18 alerts)

Organisma Type of alert Alert characteristicsb Dates of outbreak (start–end) Recurrence interval
(1/n years)
Observed cases Observed/ expected case ratio Time span in days CPCIO evaluation
2012
E. coli Ward/res Gen med
Fully susceptible
5 Sep–22 Sep 1/1.14 18 2.51 19 Pat/res
P. aeruginosa Ward/res Vascular surgery
CTX, SXT
12 Oct–22 Oct 1/1.25 4 7.02 11 Pat/res
E. aerogenes Ward/res Vascular surgery
Fully susceptible
22 Oct–25 Oct 1/19.85 2 6.06 4 Ward
S. marcescens Res SXT 13 Nov–15 Nov 1/22.40 2 5.56 3 Pat/res
2013
E. coli Ward/res Neuro-rehab 20 Apr–20 Apr 1/1.52 2 153.85 1 Ward
K. pneumoniae Res CTX, CAZ, CIP, GEN, TZP, SXT 6 May–6 Jun 1/23.69 6 46.15 32 Pat/res
E. faecium Serv/res Neonatology
AMP, ERY, GEN, IPM, LVX, MFX, VAN
10 Jun–12 Jun 1/1.30 2 21.05 3 Pat/res/serv
S. aureus Res LVX, OXA, PEN 26 Apr–22 Jun 1/1.44 10 4.13 58 Pat/res
P. aeruginosa Ward/res ICU
CTX, CAZ, IPM, MEM, TZP, SXT
15 Jun–1 Jul 1/1.44 3 23.08 17 Pat/res/ward
S. marcescens Serv/res ICU
AMK
19 May–15 Jul 1/2.11 3 4.76 58 Pat/res/serv
A. baumannii Res CTX, CAZ, CIP, GEN, IPM, MEM, SXT 20 May–1 Aug 1/2.00 13 3.56 74 Pat/res
K. pneumoniae Serv/res ICU
AMK, CTX, CAZ, CIP, GEN, IPM, MEM, TZP, SXT
3 Aug–7 Aug 1/2.78 3 38.96 5 Pat/res/serv
P. aeruginosa Serv Interm care unit 26 Aug–27 Sep 1/4.13 4 9.52 33 Serv
K. pneumoniae Serv Interm care unit 5 Oct–18 Oct 1/1.25 3 15.79 14 Serv
S. aureus Ward/res Ortho-Trauma
Fully susceptible
2 Dec–2 Dec 1/2.11 2 142.86 1 Pat/res/ward
S. marcescens Serv/res Gen Med
AMK
24 Dec–3 Jan 2014 1/1.37 2 9.09 11 Pat/res/serv
2014
P. aeruginosa Serv/res Surgery
CIP, IPM, SXT
31 Mar–31 Mar 1/2.49 2 105.26 1 Pat/res/serv
S. maltophila Ward/res ICU 14 Apr–28 May 1/3.26 2 4.35 45 Pat/res/ward

Gen med: general medicine; ICU: intensive care unit; interm care unit: intermediate care unit; neuro-rehab: neuro-rehabilitation; ortho-trauma: orthopaedic trauma; pat: pathogen type; res: resistance; serv: service.

a The organisms are abbreviated as follows: A. baumannii: Acinetobacter baumannii; E. aerogenes: Enterobacter aerogenes; E. coli: Escherichia coli; E. faecium: Enterococcus faecium; K. pneumoniae: Klebsiella pneumoniae; P. aeruginosa: Pseudomonas aeruginosa; S. aureus: Staphylococcus aureus; S. maltophila: Stenotrophomonas maltophila; S. marcescens: Serratia marcescens.

b Antibiotics listed in this column are abbreviated as follows: AMK: amikacin; AMP: ampicillin; CAZ: ceftazidime; CIP: ciprofloxacin; CTX: cefotaxime; ERY: erythromycin; GEN: gentamicin; IPM: imipenem; LVX: levofloxacin; MEM: meropenem; MFX: moxifloxacin; OXA: oxacillin; PEN: penicillin; SXT: trimethoprim/sulfamethoxazole; TZP: tazobactam; VAN: vancomycin.

Discussion

Timeliness is one of the main attributes of a good surveillance system, representing the ability to take appropriate public health action based on urgency [21]. Electronic data systems for the collection and analysis of microbiology data are becoming essential tools for surveillance to guarantee reliability, timeliness and standardisation across different compartments [22]. The aim of this work is to show the utility of a new tool, the WHONET-SaTScan, for surveillance of AMR in healthcare settings, especially in a context in which national surveillance programmes facilitate automated routine data collection, as the case of the Italian MICRONET [23].

When compared with traditional surveillance methods, the automated system used in this study showed a discrepancy in detected signals, as previously observed in other studies [15,24]. The higher number of signals produced by WHONET-SaTScan could be due to methodological differences compared to the CPCIO approach. WHONET-SaTScan generates a list of statistically significant signals, using an arbitrary choice for the cut-off value of significance (the recurrence interval), that affects sensitivity and specificity of the method, therefore meaning that statistically significant signals could not be necessarily indicative of a real outbreak or vice versa. Furthermore, the space-time permutation statistics cannot distinguish underlying fluctuations of local population sizes or temporal variations of detection frequency, leading to biased p-values [14]. In contrast, the CPCIO’s method is based on the classic definition of outbreak based on the ‘3 by 5’ rule, irrespective of the baseline incidence of the organism or the specific resistance phenotypes. In this case, its sensitivity is determined by the complexity of the case definition and personal interpretation, particularly in case of complex resistance phenotypes, while its specificity can be affected by baseline incidence. As a consequence, detection of clusters could be either delayed or even missing, especially if cases are spread throughout the hospital or, alternatively, infection control responses could be triggered when not needed, drawing staff and resources from the hospital and causing unnecessary distress to patients. On the other hand, traditional methods allow case-by-case interpretations based on personal experience and hospital background, identifying clusters not statistically but epidemiologically significant, like for example the cluster of A. baumannii in 2012, detected only by the CPCIO. Lack of information on the evolution of outbreaks after activation of the ‘EpiD’ procedure in the CPCIO reports, besides providing no indication on the efficacy of the measures adopted, interferes with the comparison between extent of outbreaks, as clusters detected by WHONET-SaTScan may result in higher case numbers and longer time spans.

The WHONET-SaTScan system showed some advantages compared to the CPCIO’s. The ‘3 by 5’ rule applied to a single ward at the time, in particular to critical care units, seems to be restrictive when compared with the WHONET-SaTScan ability to include groups of wards together or cover the whole hospital simultaneously. In this study, the ‘EpiD’ activated by the CPCIO occurred mainly in the ICU, while the clusters detected by WHONET-SaTScan were more homogeneously distributed throughout the hospital. The evaluation by the CPCIO coordinator showed that the main factor to trigger a response was the organism resistance profile, followed by pathogen type, location and source of specimen. The WHONET-SaTScan analysis allows for the investigation of clusters according to a specific resistance profile in combination to a specific location (‘resistance/ward’ and ‘resistance/service’), useful when an outbreak is occurring in a critical care ward. In addition, within the same analysis WHONET-SaTScan identifies clusters of susceptible strains, otherwise neglected due to a higher focus on resistance. Such clusters could be, in fact, of great interest to the infection control team for their routes of transmission and to the medical team in terms of pathogen characteristics and for offering different therapeutic options.

This study is not exempt of limitations and bias. Its retrospective nature undermines the efficacy of the WHONET-SaTScan system in the ‘field’. If conducted in real-time, it would have detected two clusters of MDR K. pneumoniae on average 126 days (95% confidence interval (CI): 66–186; n = 2), i.e. four months earlier, than the standard hospital control system, plus additional ones (two outbreaks of E. faecium VRE strain and P. aeruginosa MDR/XDR) that had occurred unnoticed within the hospital. The possibility to investigate prospectively the list of statistically significant alerts in combination with the clinical and epidemiological expertise of the hospital control team would provide a better evaluation of its benefits. Moreover, the inclusion of C. difficile, at the time not included in the list of organisms in the WHONET-SatScan analysis, would have better met the needs of the facility under investigation.

Reporting bias occurs as a consequence of selective reporting and control within the hospital due to a combination of resource availability, therapeutic choices and background prevalence data. For example, because of the endemic distribution in Italy of MRSA or ESBLs and the lack of appropriate resources for a prompt and effective intervention, the hospital adopted the policy of not reporting alerts triggered by these organisms. Again, the choice of antibiotics routinely used would reflect the panel of antibiotics tested and included in the configuration of WHONET, thus generating a list of alerts biased by the hospital policy on testing and reporting microbiology data. Lack of representativeness is another limitation of this study, as SFNH has in place an official procedure for infection control and a regular collection of standardised microbiology data, which most likely does not reflect the situation of other hospitals in Italy, a country with high between-hospital and regional variation.

Nevertheless, this work represents the first application of the WHONET-SaTScan system in a healthcare facility in Italy with the potential to be applied to other hospitals, extended to multiple hospitals in the same area or region or even on a larger scale to the whole national territory. Although the WHONET software is implemented within the surveillance systems of other European countries [25-27], this pilot study represents the first example of its application to the detection of clusters of resistant pathogens within a national surveillance system in Europe.


Acknowledgements

There was no financial support for the completion of this work. The contributions of Dr. Stelling to this publication were supported by the National Institute of General Medical Sciences of the National Institutes of Health under award number R01GM103525.

Conflict of interest

None declared.

Authors’ contributions

Alessandra Natale, led all data analysis and writing of the manuscript. John Stelling, project co-supervisor, gave strong guidance on data analysis and use of the WHONET-SatScan software plus overall revision of the work. Marcello Meledandri, kindly filled the questionnaire on the comparison analysis between WHONET-SaTScan data and the hospital infection control system, and provided access to the CPCIO reports. Louisa Messenger, provided overall ideas. Fortunato D'ancona, project supervisor, provided facilities, MICRONET data access and supervision on data analysis, plus strong guidance on the writing of the manuscript.


References

  1. Spellberg B, Blaser M, Guidos RJ, Boucher HW, Bradley JS, Eisenstein BI,  et al. , Infectious Diseases Society of America (IDSA). Combating antimicrobial resistance: policy recommendations to save lives.Clin Infect Dis. 2011;52(Suppl 5):S397-428. DOI: 10.1093/cid/cir153 PMID: 21474585

  2. Grundmann H, Klugman KP, Walsh T, Ramon-Pardo P, Sigauque B, Khan W,  et al.  A framework for global surveillance of antibiotic resistance. Drug Resist Updat. 2011;14(2):79-87. DOI: 10.1016/j.drup.2011.02.007 PMID: 21482177

  3. World Health Organization (WHO). Antimicrobial resistance. Geneva: WHO. [Accessed 29 Apr 2014]. Available from: http://www.who.int/mediacentre/factsheets/fs194/en/

  4. Weinberg J. Surveillance and control of infectious diseases at local, national and international levels.Clin Microbiol Infect. 2005;11(Suppl 1):12-4. DOI: 10.1111/j.1469-0691.2005.01083.x PMID: 15760438

  5. Brown D, Canton R, Dubreuil L, Gatermann S, Giske C, MacGowan A,  et al.  Widespread implementation of EUCAST breakpoints for antibacterial susceptibility testing in Europe. Euro Surveill. 2015;20(2):21008. DOI: 10.2807/1560-7917.ES2015.20.2.21008 PMID: 25613780

  6. O’Brien TF, Stelling J. The world’s microbiology laboratories can be a global microbial sensor network.Biomedica. 2014;34(Suppl 1):9-15.PMID: 24968031

  7. Antimicrobial resistance interactive database (EARS-Net). [Accessed 6 Aug 2016]. Available from: http://ecdc.europa.eu/en/healthtopics/antimicrobial_resistance/database/Pages/database.aspx

  8. European Centre for Disease Prevention and Control (ECDC). Antimicrobial resistance surveillance in Europe 2014. Annual Report of the European Antimicrobial Resistance Surveillance Network (EARS-Net). Stockholm: ECDC; 2015.

  9. World Health Organization (WHO). WHONET Software. Geneva: WHO. [Accessed 30 Jan 2016]. Available from: http://www.who.int/drugresistance/whonetsoftware/en/

  10. WHONET. Documentation. [Accessed 26 Jan 2016]. Available from: http://www.whonet.org/www/documentation.html

  11. Stelling JM, O’Brien TF. Surveillance of antimicrobial resistance: the WHONET program.Clin Infect Dis. 1997;24(Suppl 1):S157-68. DOI: 10.1093/clinids/24.Supplement_1.S157 PMID: 8994799

  12. SaTScan - Software for the spatial, temporal, and space-time scan statistics. [Accessed 25 Oct 2015]. Available from: http://www.satscan.org/

  13. Kulldorff M. Prospective time periodic geographical disease surveillance using a scan statistic.J R Stat Soc A.2001;164(1):61-72. DOI: 10.1111/1467-985X.00186

  14. Kulldorff M, Heffernan R, Hartman J, Assunção R, Mostashari F. A space-time permutation scan statistic for disease outbreak detection.PLoS Med. 2005;2(3):e59. DOI: 10.1371/journal.pmed.0020059 PMID: 15719066

  15. Huang SS, Yokoe DS, Stelling J, Placzek H, Kulldorff M, Kleinman K,  et al.  Automated detection of infectious disease outbreaks in hospitals: a retrospective cohort study. PLoS Med. 2010;7(2):e1000238. DOI: 10.1371/journal.pmed.1000238 PMID: 20186274

  16. Viñas MR, Tuduri E, Galar A, Yih K, Pichel M, Stelling J,  et al. , Group MIDAS - Argentina. Laboratory-based prospective surveillance for community outbreaks of Shigella spp. in Argentina.PLoS Negl Trop Dis. 2013;7(12):e2521. DOI: 10.1371/journal.pntd.0002521 PMID: 24349586

  17. Stelling J, Yih WK, Galas M, Kulldorff M, Pichel M, Terragno R,  et al. , Collaborative Group WHONET-Argentina. Automated use of WHONET and SaTScan to detect outbreaks of Shigella spp. using antimicrobial resistance phenotypes.Epidemiol Infect. 2010;138(6):873-83. DOI: 10.1017/S0950268809990884 PMID: 19796449

  18. Vlek ALM, Cooper BS, Kypraios T, Cox A, Edgeworth JD, Auguet OT. Clustering of antimicrobial resistance outbreaks across bacterial species in the intensive care unit.Clin Infect Dis. 2013;57(1):65-76. DOI: 10.1093/cid/cit192 PMID: 23549524

  19. Sisto A, D’Ancona F, Meledandri M, Pantosti A, Rossolini GM, Raglio A,  et al. , Micronet network participants. Carbapenem non-susceptible Klebsiella pneumoniae from Micronet network hospitals, Italy, 2009 to 2012.Euro Surveill. 2012;17(33):20247.PMID: 22913976

  20. The European Committee on Antimicrobial Susceptibility Testing (EUCAST). Breakpoint tables for interpretation of MICs and zone diameters. Version 6.0, 2016. Växjö: EUCAST; 2016. Available from: http://www.eucast.org.

  21. Centers for Disease Control (CDC). Guidelines for evaluating surveillance systems.MMWR Suppl. 1988;37(5):1-18.PMID: 3131659

  22. van Mourik MS, Troelstra A, van Solinge WW, Moons KG, Bonten MJ. Automated surveillance for healthcare-associated infections: opportunities for improvement.Clin Infect Dis. 2013;57(1):85-93. DOI: 10.1093/cid/cit185 PMID: 23532476

  23. Sisto A, D’Ancona F, Meledandri M, Pantosti A, Goglio A. Acinetobacter baumanni in Italian hospitals: results from a retrospective data analysis from the Micronet Network laboratories. 2011; Presentation at the European Congress of Clinical Microbiology and Infectious Diseases (ECCMID) and International Congress of Chemotherapy (ICC). Milan: 21st ECCMID; 27th ICC. 7 – 10 May 2011. P701.

  24. Faires MC, Pearl DL, Ciccotelli WA, Berke O, Reid-Smith RJ, Weese JS. The use of the temporal scan statistic to detect methicillin-resistant Staphylococcus aureus clusters in a community hospital.BMC Infect Dis. 2014;14(1):375. DOI: 10.1186/1471-2334-14-375 PMID: 25005247

  25. Falagas ME, Mourtzoukou EG, Polemis M, Vatopoulos AC, Greek System for Surveillance of Antimicrobial Resistance. Trends in antimicrobial resistance of Acinetobacter baumannii clinical isolates from hospitalised patients in Greece and treatment implications.Clin Microbiol Infect. 2007;13(8):816-9. DOI: 10.1111/j.1469-0691.2007.01761.x PMID: 17610601

  26. Chaniotaki S, Giakouppi P, Tzouvelekis LS, Panagiotakos D, Kozanitou M, Petrikkos G,  et al. , WHONET Study Group. Quinolone resistance among Escherichia coli strains from community-acquired urinary tract infections in Greece.Clin Microbiol Infect. 2004;10(1):75-8. DOI: 10.1111/j.1469-0691.2004.00807.x PMID: 14706091

  27. Oteo J, Baquero F, Vindel A, Campos J, Spanish members of the European Antimicrobial Resistance Surveillance System. Antibiotic resistance in 3113 blood isolates of Staphylococcus aureus in 40 Spanish hospitals participating in the European Antimicrobial Resistance Surveillance System (2000-2002).J Antimicrob Chemother. 2004;53(6):1033-8. DOI: 10.1093/jac/dkh214 PMID: 15128722



Back to Table of Contents
Previous Download (pdf)
Next

The publisher’s policy on data collection and use of cookies.

Disclaimer: The opinions expressed by authors contributing to Eurosurveillance do not necessarily reflect the opinions of the European Centre for Disease Prevention and Control (ECDC) or the editorial team or the institutions with which the authors are affiliated. Neither ECDC nor any person acting on behalf of ECDC is responsible for the use that might be made of the information in this journal. The information provided on the Eurosurveillance site is designed to support, not replace, the relationship that exists between a patient/site visitor and his/her physician. Our website does not host any form of commercial advertisement. Except where otherwise stated, all manuscripts published after 1 January 2016 will be published under the Creative Commons Attribution (CC BY) licence. You are free to share and adapt the material, but you must give appropriate credit, provide a link to the licence, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

Eurosurveillance [ISSN] - ©2007-2016. All rights reserved
 

This website is certified by Health On the Net Foundation. Click to verify. This site complies with the HONcode standard for trustworthy health information:
verify here.