Detection of Infectious Disease Outbreaks in Search Engine Time Series Using Non-Specific Syndromic Surveillance with Effect-Size Filtering

Novel infectious disease outbreaks, including most recently that of the COVID-19 pandemic, could be detected by non-specific syndromic surveillance systems. Such systems, utilizing a variety of data sources ranging from Electronic Health Records to internet data such as aggregated search engine queries, create alerts when unusually high rates of symptom reports occur. This is especially important for the detection of novel diseases, where their manifested symptoms are unknown. Here we improve upon a set of previously-proposed non-specific syndromic surveillance methods by taking into account both how unusual a preponderance of symptoms is and their effect size. We demonstrate that our method is as accurate as previously-proposed methods for low dimensional data and show its effectiveness for high-dimensional aggregated data by applying it to aggregated time-series health-related search engine queries. We find that in 2019 the method would have raised alerts related to several disease outbreaks earlier than health authorities did. During the COVID-19 pandemic the system identified the beginning of pandemic waves quickly, through combinations of symptoms which varied from wave to wave. Thus, the proposed method could be used as a practical tool for decision makers to detect new disease outbreaks using time series derived from search engine data even in the absence of specific information on the diseases of interest and their symptoms.