> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/Metaculus/metaculus/llms.txt
> Use this file to discover all available pages before exploring further.
# Forecasting
> How to make predictions, update forecasts, and understand forecast aggregation
## Overview
Forecasting is the core activity on Metaculus. Forecasters make predictions about future events by submitting probability distributions that represent their beliefs about different outcomes.
## Making Predictions
Forecasts vary by question type, but all share common characteristics:
Submit a single probability between 0% and 100% representing your confidence the event will occur.
**Data Structure** (from `questions/models.py:639-642`):
```python theme={null}
probability_yes: float = models.FloatField(null=True, blank=True)
```
**Example**: 75% probability that SpaceX launches Starship to orbit in 2025
Distribute probability across available options. Probabilities must sum to 100%.
**Data Structure** (from `questions/models.py:643-648`):
```python theme={null}
probability_yes_per_category: list[float | None] = ArrayField(
models.FloatField(null=True),
null=True,
blank=True,
)
```
Options not yet available at forecast time are stored as `None`.
Provide a cumulative distribution function (CDF) over the range of possible values.
**Data Structure** (from `questions/models.py:630-637`):
```python theme={null}
continuous_cdf: list[float] = ArrayField(
models.FloatField(),
null=True,
max_length=DEFAULT_INBOUND_OUTCOME_COUNT + 1,
blank=True,
)
```
The CDF is evaluated at evenly-spaced quantiles from 0.0 to 1.0 (default: 201 points).
## Forecast Model
Each forecast is a snapshot of a user's prediction at a specific time.
### Core Fields
From `questions/models.py:609-663`:
```python theme={null}
class Forecast(models.Model):
# Time range when this forecast is active
start_time = models.DateTimeField(db_index=True)
end_time = models.DateTimeField(null=True, db_index=True, blank=True)
# Prediction data (one of these will be set based on question type)
probability_yes: float # Binary questions
probability_yes_per_category: list[float | None] # Multiple choice
continuous_cdf: list[float] # Numeric/Date/Discrete
# Metadata
author = models.ForeignKey(User, models.CASCADE)
question = models.ForeignKey(Question, models.CASCADE)
post = models.ForeignKey("posts.Post", models.CASCADE)
source = models.CharField(max_length=30, choices=SourceChoices.choices)
distribution_input = models.JSONField(null=True, blank=True)
```
### Forecast Sources
Forecasts can originate from different sources (from `questions/models.py:664-669`):
* **`api`**: Made via the API
* **`ui`**: Made through the web interface
* **`automatic`**: Automatically assigned (e.g., when forecasts are split)
## Updating Forecasts
When you update your prediction, Metaculus creates a new forecast entry:
1. The previous forecast's `end_time` is set to the current timestamp
2. A new forecast is created with `start_time` = current timestamp
3. The new forecast has `end_time = None` (indicating it's currently active)
This time-series approach allows Metaculus to track how predictions evolve and score forecasters based on their entire prediction history.
## Active Forecasts
A forecast is considered "active" when (from `questions/models.py:574-601`):
```python theme={null}
def active(self):
now = timezone.now()
# Forecast timing conditions
forecast_started = Q(start_time__lte=now)
forecast_not_ended = Q(end_time__isnull=True) | Q(end_time__gt=now)
# Question status conditions
question_not_closed = Q(question__actual_close_time__isnull=True)
question_still_accepting_forecasts = Q(question__scheduled_close_time__gt=now)
question_opened = Q(question__open_time__lte=now)
return self.filter(
forecast_started
& forecast_not_ended
& question_not_closed
& question_still_accepting_forecasts
& question_opened
)
```
## Forecast Aggregation
Metaculus combines individual forecasts into a **Community Prediction (CP)** that represents the consensus view.
### Aggregation Methods
From `questions/types.py:18-22`, Metaculus supports four aggregation methods:
Recent forecasts are weighted more heavily than older forecasts. This is the **default method** for most questions.
**Best for**: Standard questions with extended forecasting periods
**Algorithm**: Uses time-decaying weights to give recent predictions more influence
All forecasts are weighted equally regardless of when they were made.
**Best for**: Very short-term questions or live forecasting events
**Algorithm**: Simple geometric mean of all active forecasts
Aggregation calculated at a single point in time.
**Best for**: Snapshot aggregations or historical analysis
Proprietary Metaculus algorithm that incorporates additional signals.
**Best for**: High-stakes questions requiring sophisticated aggregation
### Geometric Mean Aggregation
The core aggregation algorithm uses geometric mean (from `scoring/score_math.py:28-53`):
```python theme={null}
def get_geometric_means(
forecasts: Sequence[Forecast | AggregateForecast],
) -> list[AggregationEntry]:
geometric_means = []
timesteps: set[float] = set()
# Collect all forecast start and end times
for forecast in forecasts:
timesteps.add(forecast.start_time.timestamp())
if forecast.end_time:
timesteps.add(forecast.end_time.timestamp())
# Calculate geometric mean at each timestep
for timestep in sorted(timesteps):
prediction_values = [
f.get_pmf()
for f in forecasts
if f.start_time.timestamp() <= timestep
and (f.end_time is None or f.end_time.timestamp() > timestep)
]
if not prediction_values:
continue
geometric_mean = gmean(prediction_values, axis=0)
predictors = len(prediction_values)
geometric_means.append(
AggregationEntry(geometric_mean, predictors, timestep)
)
return geometric_means
```
Geometric mean is used instead of arithmetic mean because it better handles extreme probabilities and prevents a single forecaster from dominating the aggregate.
## Aggregate Forecast Model
Aggregated forecasts are stored separately from individual forecasts (from `questions/models.py:760-833`):
```python theme={null}
class AggregateForecast(models.Model):
question = models.ForeignKey(Question, models.CASCADE)
method = models.CharField(max_length=200, choices=AggregationMethod.choices)
# Time range for this aggregate
start_time = models.DateTimeField(db_index=True)
end_time = models.DateTimeField(null=True, db_index=True)
# Aggregated prediction values
forecast_values: list[float | None] = ArrayField(
models.FloatField(null=True),
max_length=DEFAULT_INBOUND_OUTCOME_COUNT + 1
)
# Metadata
forecaster_count: int | None = models.IntegerField(null=True)
# Uncertainty quantification (for continuous distributions)
interval_lower_bounds = ArrayField(models.FloatField(), null=True)
centers = ArrayField(models.FloatField(), null=True)
interval_upper_bounds = ArrayField(models.FloatField(), null=True)
means = ArrayField(models.FloatField(), null=True)
histogram = ArrayField(models.FloatField(), null=True, size=100)
```
### Forecaster Count
The `forecaster_count` field tracks how many unique forecasters contributed to the aggregate at each timestep. This is important for:
* Determining minimum participation thresholds
* Calculating peer scores (which require at least 2 forecasters)
* Displaying confidence in the community prediction
## Forecast Constraints
Metaculus enforces several constraints on forecasts:
### Time Constraints
From `questions/models.py:691-697`:
```python theme={null}
constraints = [
# end_time must be after start_time
models.CheckConstraint(
check=Q(end_time__isnull=True) | Q(end_time__gt=F("start_time")),
name="end_time_after_start_time",
),
]
```
### Question Period Filter
Forecasts are filtered to only count those made during the question's active period (from `questions/models.py:539-562`):
```python theme={null}
def filter_within_question_period(self):
return self.filter(
# Has no end time or an end time after question open time
(Q(end_time__isnull=True) | Q(end_time__gt=F("question__open_time")))
# AND has a start time earlier than the questions close time
& (
(Q(question__actual_close_time__isnull=False)
& Q(start_time__lt=F("question__actual_close_time")))
| (Q(question__actual_close_time__isnull=True)
& Q(start_time__lt=F("question__scheduled_close_time")))
),
)
```
## Bot Forecasts
Metaculus includes bot forecasters that provide baseline predictions.
### Bot Types
From the user model:
* **`is_bot`**: Marks automated forecasters
* **`is_primary_bot`**: Distinguishes official Metaculus bots from third-party bots
### Including Bots in Aggregates
Questions can control whether bots are included (from `questions/models.py:89`):
```python theme={null}
include_bots_in_aggregates = models.BooleanField(default=False)
```
Bot forecasts can be excluded from queries using (from `questions/models.py:569-572`):
```python theme={null}
def exclude_non_primary_bots(self):
return self.filter(
Q(author__is_bot=False) | Q(author__is_primary_bot=True),
)
```
## Distribution Input
The `distribution_input` field stores the raw input format provided by the user (from `questions/models.py:681-684`):
```python theme={null}
distribution_input = models.JSONField(null=True, blank=True)
```
This preserves the original format (e.g., parameters of a normal distribution) even though the forecast is stored as a CDF.
## Best Practices
Frequent updates as you gather new information lead to better scores
Avoid overconfidence - extreme predictions (0%, 100%) are rarely justified
Start with historical frequencies before adjusting for specifics
Comment on your forecasts to track your thinking and help others
## API Reference
Explore the full Forecasts API documentation
## Related Topics
Understand question types and structure
Learn how forecasts are evaluated
Track your performance against others