Supplementary Information File:

The RAPIDD Ebola Forecasting Challenge:

Synthesis and Lessons Learnt

Cécile Viboud 1, Kaiyuan Sun 2, Robert Gaffey 1, Marco Ajelli 3, Laura Fumanelli 3, Stefano Merler 3, Qian

Zhang 2, Gerardo Chowell 1,4, Lone Simonsen 1,5, Alessandro Vespignani 2,6,7, and the RAPIDD Ebola

Forecasting Challenge group*

Supplementary Table1: Performance statistics for individual teams and ensemble predictions (BMA: Bayesian Mean, ENS_mean: mean of all available point predictions). Teams are ranked according to their performance for all scenarios combined.

Supplementary Table 2. Select error metrics for Ebola incidence forecasts using the challenge model fitted to its own data (benchmark of prediction). Table displays values for 3 different error metrics (Mean absolute percentage error, R square, and Pearson’s correlation), by scenario of the challenge model. Each value represents a summary error averaged over 20 incidence targets (1-4 week ahead incidence forecasts for each of 5 prediction time points).

Model Scenario 1 Scenario 2 Scenario 3 Scenario 4

All scenarios combined

Mean absolute percentage errorChallenge model fitted to its own data 0.23 0.13 0.46 0.15 0.24

R squaredChallenge model fitted to its own data 0.87 0.92 0.67 0.73 0.86

Pearson's correlationChallenge model fitted to its own data 0.93 0.98 0.82 0.88 0.93

Supplementary Figure 1: Prediction accuracy vs uncertainty. Top. Proportion of ground-truth incidence values that fell within the IQR of predictions, by team. Bottom. Mean relative uncertainty across teams. Relative uncertainty is measured by the ratio of the size of an interquartile range by the point estimate, for each prediction. The CDC model did not provide uncertainty measures.

Supplementary Figure 2: Agreement between synthetic and predicted 1-4 week ahead incidences for scenario 2, combining all prediction time points.

Supplementary Figure 3: Agreement between synthetic and predicted 1-4 week ahead incidences for scenario 3, combining all prediction time points.

Supplementary Figure 4: Agreement between synthetic and predicted 1-4 week ahead incidences for scenario 4, combining all prediction time points.

Supplementary Figure 5: Timing and magnitude of Ebola peaks, and final size predictions, for scenario 2.

Supplementary Figure 6: Timing and magnitude of Ebola peaks, and final size predictions, for scenario 3.

Supplementary Figure 7: Timing and magnitude of Ebola peaks, and final size predictions, for data-poor scenario 4.