This first-of-its kind, multi-year biosecurity forecasting tournament was developed in partnership with pandemic preparedness non-profit Guarding Against Pandemics and the research and advocacy think tank the Institute for Progress. The Biosecurity Tournament has three aims: to map out the future biosecurity risk landscape, to help direct resource allocation to the highest-impact interventions, and to inform the distribution of attention and efforts — all with a focus on global catastrophic biological risks (GCBRs).
The Metaculus Community’s forecasts will be shared with public health experts, informing their decision making and helping to mitigate future COVID-like risks. Guarding Against Pandemics has generously provided a $25,000 prize pool to support forecasting efforts.
Round 1 of this tournament is open for forecasting, with new questions arriving on 2022-6-16, 2022-6-23, 2022-6-30. We appreciate the forecasting community’s insights and expertise in furthering this critical initiative.
Read on for additional detail about how this tournament is structured, how prizes are awarded, and how forecasts are scored.
Tournament Prize Structure
The Biosecurity Tournament consists of a short-term 2-year phase and a long-term 10-year phase, with $15,000 in prizes allocated for the short-term and $10,000 allocated for the long-term. In total, across both phases, the tournament will feature between 30 and 80 questions. Both short-term and long-term questions will close in December 2024, but long-term questions will not resolve until 2032. Prizes for the long-term phase will be entirely determined by the long-term questions.
After the current round concludes, there will be at least one additional round of questions, which is currently set to launch in fall 2022. The second round may use conditional forecasting to assess the expected impact of interventions and investments in biosecurity.
Tournament Scoring
For most questions there will be a hidden period of roughly two months, during which the community median will not be visible. For the leaderboard, half of the coverage will be determined by participation during this hidden period and the other half will be based on participation during the visible period. Tournament scoring aims to balance rewarding independent forecasting skill (during the hidden period) and providing latecomers who miss the hidden period a realistic chance to finish in first place.
This tournament will be the first to calculate the leaderboard based on an effective number of questions parameter, in order to give more forecasters an opportunity to win prizes. All leaderboards to date have calculated a tournament score by adding forecasters’ question scores and calculating a take. In practice, this has resulted in some concentrated prize distributions to the top forecasters, especially in the case of the Real-Time Pandemic Decision Making Tournament.
To reduce the leaderboard concentration, forecasters’ tournament scores will be divided by a normalization factor. This factor equals the number of questions (expected to be between 30 and 80) divided by an effective number of questions of 20. If this tournament ultimately features 80 questions, then the usual tournament scores will be divided by a normalization factor of 4 (=80/20). If there are 40 questions, then the normalization factor will be 2 (=40/20).
This normalization reduces the spread in tournament scores, which reduces the spread in the take and prize distribution. Said another way, the prize distribution will be similar to that in a 20 question tournament, which is expected to have a more even prize distribution.