Comparing West Nile Virus Forecasts

Global changes affect disease distribution and dynamics. Quantitative tools may provide risk information and guide interventions to improve human and ecological health outcomes. However, a variety of tools exist, and formally comparing methods could allow researchers and stakeholders, such as public health workers, to quickly identify the most appropriate model(s) or modeling approach for a given situation. For vector-borne diseases, it has not been possible to systematically evaluate and compare forecasting methods because forecasts are often based on different data sources, spatiotemporal scales, and measurement precision. Assessments also use different evaluation metrics, making meta-analysis impossible. Therefore, it is essential that a formal approach to disease forecast comparison be developed. At the same time, stakeholders often apply these models without understanding their strengths, weaknesses, and assumptions. This could lead to sub-optimal public health and vector control decision-making. Therefore, it is crucial to rigorously explore the models using simulated data sets. We propose a workshop in which data managers and model forecasters cooperatively develop a collection of common data sets and evaluation metrics to compare model performance from their various approaches to improve the capacity to forecast when and where West Nile virus may occur. We believe this integrative workshop approach will prove an effective template for disease forecast improvement and application across a wide range of systems.