1887
  1. Home
  2. Eurosurveillance
  3. Volume 27, Issue 39, 29/Sep/2022
  4. Article
Research Open Access
Like 0
Download

Abstract

Background

The European Centre for Disease Prevention and Control (ECDC) systematically collates information from sources to rapidly detect early public health threats. The lack of a freely available, customisable and automated early warning tool using data from Twitter prompted the ECDC to develop epitweetr, which collects, geolocates and aggregates tweets generating signals and email alerts.

Aim

This study aims to compare the performance of epitweetr to manually monitoring tweets for the purpose of early detecting public health threats.

Methods

We calculated the general and specific positive predictive value (PPV) of signals generated by epitweetr between 19 October and 30 November 2020. Sensitivity, specificity, timeliness and accuracy and performance of tweet geolocation and signal detection algorithms obtained from epitweetr and the manual monitoring of 1,200 tweets were compared.

Results

The epitweetr geolocation algorithm had an accuracy of 30.1% at national, and 25.9% at subnational levels. The signal detection algorithm had 3.0% general PPV and 74.6% specific PPV. Compared to manual monitoring, epitweetr had greater sensitivity (47.9% and 78.6%, respectively), and reduced PPV (97.9% and 74.6%, respectively). Median validation time difference between 16 common events detected by epitweetr and manual monitoring was -48.6 hours (IQR: −102.8 to −23.7).

Conclusion

Epitweetr has shown sufficient performance as an early warning tool for public health threats using Twitter data. Since epitweetr is a free, open-source tool with configurable settings and a strong automated component, it is expected to increase in usability and usefulness to public health experts.

© Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Closure
A corrected article has been published for this content:
Erratum for Euro Surveill. 2022;27(39)
Loading

Article metrics loading...

/content/10.2807/1560-7917.ES.2022.27.39.2200177
2022-09-29
2024-11-21
http://instance.metastore.ingenta.com/content/10.2807/1560-7917.ES.2022.27.39.2200177
Loading
Loading full text...

Full text loading...

/deliver/fulltext/eurosurveillance/27/39/eurosurv-27-39-5.html?itemId=/content/10.2807/1560-7917.ES.2022.27.39.2200177&mimeType=html&fmt=ahah

References

  1. Decision No. 1082/2013/EU of the European Parliament and of the Council of 22 October 2013 on serious cross-border threats to health and repealing Decision No 2119/98/EC. Luxembourg: Official Journal of the European Union; 5 Nov 2013. Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013D1082&from=EN
  2. European Centre for Disease Prevention and Control (ECDC). Communicable disease threats to public health in the European Union - Annual epidemiological report for 2019. Stockholm: ECDC; 2020. Available from: https://www.ecdc.europa.eu/en/publications-data/communicable-disease-threats-public-health-european-union-2019
  3. European Centre for Disease Prevention and Control (ECDC). Sources - Worldwide data on COVID-19. Stockholm: ECDC; 2020. Available from: https://www.ecdc.europa.eu/en/publications-data/sources-worldwide-data-covid-19
  4. Li C, Chen LJ, Chen X, Zhang M, Pang CP, Chen H. Retrospective analysis of the possibility of predicting the COVID-19 outbreak from Internet searches and social media data, China, 2020. Euro Surveill. 2020;25(10). . https://doi.org/10.2807/1560-7917.ES.2020.25.10.2000199  PMID: 32183935 
  5. Rocklöv J, Tozan Y, Ramadona A, Sewe MO, Sudre B, Garrido J, et al. Using Big Data to Monitor the Introduction and Spread of Chikungunya, Europe, 2017. Emerg Infect Dis. 2019;25(6):1041-9. . https://doi.org/10.3201/eid2506.180138  PMID: 31107221 
  6. Șerban O, Thapen N, Maginnis B, Hankin C, Foot V. Real-time processing of social media with SENTINEL: A syndromic surveillance system incorporating deep learning for health classification. Inf Process Manage. 2019;56(3):1166-84.  https://doi.org/10.1016/j.ipm.2018.04.011 
  7. Jordan S, Hovet S, Fung I, Liang H, Fu K-W, Tse Z. Using Twitter for Public Health Surveillance from Monitoring and Prediction to Public Response. Data (Basel). 2018;4(1):6.  https://doi.org/10.3390/data4010006 
  8. de Araujo DHM, de Carvalho EA, da Motta CLR, da Silva Borges MR, Gomes JO, de Carvalho PVR. Social Networks Applied to Zika and H1N1 Epidemics: A Systematic Review. Presented at: Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018). Cham. IEA. 2018.
  9. Masri S, Jia J, Li C, Zhou G, Lee MC, Yan G, et al. Use of Twitter data to improve Zika virus surveillance in the United States during the 2016 epidemic. BMC Public Health. 2019;19(1):761.  https://doi.org/10.1186/s12889-019-7103-8  PMID: 31200692 
  10. van de Belt TH, van Stockum PT, Engelen LJLPG, Lancee J, Schrijver R, Rodríguez-Baño J, et al. Social media posts and online search behaviour as early-warning system for MRSA outbreaks. Antimicrob Resist Infect Control. 2018;7(1):69. . https://doi.org/10.1186/s13756-018-0359-4  PMID: 29876100 
  11. Dang T, Nguyen NVT, Pham V. HealthTvizer: Exploring Health Awareness in Twitter Data through Coordinated Multiple Views. Presented at: IEEE International Conference on Big Data (Big Data); New York: IEEE; 2018.
  12. Kannan R, Govindasamy MA, Soon L, Ramakrishnan K. Social Media Analytics for Dengue Monitoring in Malaysia. Presented at: 8th IEEE International Conference on Control System, Computing and Engineering (ICCSCE); New York: IEEE; 2018.
  13. Tsao S-F, Chen H, Tisseverasinghe T, Yang Y, Li L, Butt ZA. What social media told us in the time of COVID-19: a scoping review. Lancet Digit Health. 2021;3(3):e175-94.  https://doi.org/10.1016/S2589-7500(20)30315-0  PMID: 33518503 
  14. Lopreite M, Panzarasa P, Puliga M, Riccaboni M. Early warnings of COVID-19 outbreaks across Europe from social media. Sci Rep. 2021;11(1):2147. . https://doi.org/10.1038/s41598-021-81333-1  PMID: 33495534 
  15. European Centre for Disease Prevention and Control (ECDC). epitweetr tool. Stockholm: ECDC; 1 Oct 2020. Available from: https://www.ecdc.europa.eu/en/publications-data/epitweetr-tool
  16. European Centre for Disease Prevention and Control (ECDC). epitweetr. [Accessed: 16 Nov 2020]. Available from: https://github.com/EU-ECDC/epitweetr
  17. European Centre for Disease Prevention and Control (ECDC). Epitweetr: user documentation. Stockholm: ECDC; 2020. Available from: https://www.ecdc.europa.eu/sites/default/files/documents/epitweetr_vignette.pdf
  18. Bojanowski P, Grave E, Joulin A, Mikolov T. Enriching Word Vectors with Subword Information. Trans Assoc Comput Linguist. 2017;5:135-46.  https://doi.org/10.1162/tacl_a_00051 
  19. Facebook Inc. FastText word vectors for 157 languages. [Accessed: 24 Jan 2022]. Available from: https://fasttext.cc/docs/en/crawl-vectors.html
  20. GeoNames. In: Unxos GmbH, editor. Wollerau, Switzerland. [Accessed: 24 Jan 2022]. Available from: https://www.geonames.org/about.html
  21. Apache Software Foundation. Apache Lucene. 8.5.0 ed. Wilmington. [Accessed: 24 Jan 2022]. Available from: https://lucene.apache.org/
  22. Chang W, Cheng J, Allaire JJ, Xie Y, McPherson J. Shiny: Web Application Framework. 1.4.0.2 ed 2020. [Accessed: 13 Mar 2022]. Available from: https://www.rdocumentation.org/packages/shiny/versions/1.4.0.2
  23. Fricker RD Jr, Hegler BL, Dunfee DA. Comparing syndromic surveillance detection methods: EARS’ versus a CUSUM-based methodology. Stat Med. 2008;27(17):3407-29.  https://doi.org/10.1002/sim.3197  PMID: 18240128 
  24. Salmon M, Schumacher D, Höhle M. Monitoring Count Time Series inR: Aberration Detection in Public Health Surveillance. J Stat Softw. 2016;70(10).  https://doi.org/10.18637/jss.v070.i10 
  25. Farrington CP, Andrews NJ, Beale AD, Catchpole MA. A Statistical Algorithm for the Early Detection of Outbreaks of Infectious Disease. J R Stat Soc Ser A Stat Soc. 1996;159(3):547.  https://doi.org/10.2307/2983331 
  26. van Stralen KJ, Stel VS, Reitsma JB, Dekker FW, Zoccali C, Jager KJ. Diagnostic methods I: sensitivity, specificity, and other measures of accuracy. Kidney Int. 2009;75(12):1257-63.  https://doi.org/10.1038/ki.2009.92  PMID: 19340091 
  27. Kaiser R, Coulombier D, Baldari M, Morgan D, Paquet C. What is epidemic intelligence, and how is it being improved in Europe? Euro Surveill. 2006 Feb 2;11(2):E060202 4.  https://doi.org/http://dx.doi.org/10.2807/esw.11.05.02892-en 
  28. Zhu Y, Wang W, Atrubin D, Wu Y. Initial evaluation of the early aberration reporting system--Florida. MMWR Suppl. 2005;54(54):123-30. PMID: 16177703 
  29. Lifna CS, Vijayalakshmi M. Identifying Concept-drift in Twitter Streams. Procedia Comput Sci. 2015;45:86-94.  https://doi.org/10.1016/j.procs.2015.03.093 
  30. Pätsch S. ECDC developed epitweetr to find public health signals in the Twitter noise. Brussels: European Commission; June 2021. Available from: https://joinup.ec.europa.eu/collection/open-source-observatory-osor/news/searching-infectious-diseases-open-source
Epitweetr: Early warning of public health threats using Twitter data
<p class="citation"> <span>Citation style for this article:</span> <span class="meta-value authors"> <a href="/search?value1=Laura+Espinosa&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Espinosa Laura</a>, <a href="/search?value1=Ariana+Wijermans&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Wijermans Ariana</a>, <a href="/search?value1=Francisco+Orchard&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Orchard Francisco</a>, <a href="/search?value1=Michael+H%C3%B6hle&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Höhle Michael</a>, <a href="/search?value1=Thomas+Czernichow&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Czernichow Thomas</a>, <a href="/search?value1=Pietro+Coletti&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Coletti Pietro</a>, <a href="/search?value1=Lisa+Hermans&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Hermans Lisa</a>, <a href="/search?value1=Christel+Faes&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Faes Christel</a>, <a href="/search?value1=Esther+Kissling&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Kissling Esther</a>, <a href="/search?value1=Thomas+Mollet&option1=author&noRedirect=true" class="nonDisambigAuthorLink">Mollet Thomas</a></span>. Epitweetr: Early warning of public health threats using Twitter data. <a href="/content/ecdc">Euro Surveill.</a> 2022;27(39):pii=2200177. <a href="https://doi.org/10.2807/1560-7917.ES.2022.27.39.2200177" title="doi link" target="_blank">https://doi.org/10.2807/1560-7917.ES.2022.27.39.2200177</a> <span class="ES_Article_citation"> <span class="generated">Received</span>: 11 Feb 2022;&nbsp;&nbsp; <span class="generated">Accepted</span>: 04 Jul 2022 </span> </p>
/content/10.2807/1560-7917.ES.2022.27.39.2200177
/content/10.2807/1560-7917.ES.2022.27.39.2200177
Loading

Data & Media loading...

Supplementary data

Submit comment
Close
Comment moderation successfully completed
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error