> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/Metaculus/metaculus/llms.txt
> Use this file to discover all available pages before exploring further.

# Leaderboards

> Understanding ranking systems, score aggregation, and competitive standings

## Overview

Leaderboards rank forecasters based on their prediction accuracy across multiple questions. Metaculus uses sophisticated leaderboard systems to recognize top performers, distribute prizes, and track forecasting skill over time.

## Leaderboard Types

From `scoring/constants.py:17-28`, Metaculus supports multiple leaderboard score types:

```python theme={null}
class LeaderboardScoreTypes(models.TextChoices):
    PEER_TOURNAMENT = "peer_tournament"
    DEFAULT = "default"
    SPOT_PEER_TOURNAMENT = "spot_peer_tournament"
    SPOT_BASELINE_TOURNAMENT = "spot_baseline_tournament"
    RELATIVE_LEGACY_TOURNAMENT = "relative_legacy_tournament"
    BASELINE_GLOBAL = "baseline_global"
    PEER_GLOBAL = "peer_global"
    PEER_GLOBAL_LEGACY = "peer_global_legacy"
    COMMENT_INSIGHT = "comment_insight"
    QUESTION_WRITING = "question_writing"
    MANUAL = "manual"
```

<CardGroup cols={3}>
  <Card title="Tournament" icon="medal">
    Sum of scores in competitions
  </Card>

  <Card title="Global" icon="globe">
    Platform-wide rankings
  </Card>

  <Card title="Specialized" icon="star">
    Question writing, comments, manual
  </Card>
</CardGroup>

## Leaderboard Model

From `scoring/models.py:97-223`, the core leaderboard structure:

```python theme={null}
class Leaderboard(TimeStampedModel):
    name = models.CharField(max_length=200, null=True, blank=True)
    project = models.ForeignKey(
        Project,
        null=True,
        blank=True,
        on_delete=models.CASCADE,
        related_name="leaderboards",
    )
    
    score_type = models.CharField(
        max_length=200,
        choices=LeaderboardScoreTypes.choices,
    )
    
    # Time boundaries
    start_time = models.DateTimeField(null=True, blank=True)
    end_time = models.DateTimeField(null=True, blank=True)
    finalize_time = models.DateTimeField(null=True, blank=True)
    finalized = models.BooleanField(default=False)
    
    # Prize configuration
    prize_pool = models.DecimalField(
        decimal_places=2,
        max_digits=15,
        null=True,
        blank=True,
    )
    minimum_prize_amount = models.DecimalField(
        default=50.00,
        decimal_places=2,
        max_digits=15,
    )
    
    # User filtering
    user_list = models.ManyToManyField(User, blank=True)
    bot_status = models.CharField(
        max_length=32,
        choices=Project.BotLeaderboardStatus.choices,
        null=True,
        blank=True,
    )
    
    # Display configuration
    display_config = models.JSONField(null=True, blank=True)
```

## Score Types Explained

### Tournament Score Types

<Tabs>
  <Tab title="Peer Tournament">
    **Most common tournament scoring**

    Sums peer scores across all tournament questions. This is the **default** for new tournaments.

    From `scoring/constants.py:30-42`:

    ```python theme={null}
    case cls.PEER_TOURNAMENT:
        return ScoreTypes.PEER
    ```

    **Formula**: `Total = Σ(peer_score × question_weight)`

    **Best for**: Standard forecasting competitions
  </Tab>

  <Tab title="Default">
    Uses each question's default score type (which can vary per question).

    From `questions/models.py:91-102`:

    ```python theme={null}
    default_score_type = models.CharField(
        max_length=20,
        choices=ScoreTypes.choices,
        default=ScoreTypes.PEER,
    )
    ```

    **Best for**: Mixed competitions with different scoring rules per question
  </Tab>

  <Tab title="Spot Tournaments">
    Use spot peer or spot baseline scores (evaluated at a single point in time).

    **Best for**: Questions with hidden community predictions or rapid resolution
  </Tab>
</Tabs>

### Global Score Types

<Tabs>
  <Tab title="Peer Global">
    Coverage-weighted average of peer scores across all qualifying questions.

    **Formula**: `Score = Σ(peer_score × coverage × weight) / Σ(coverage × weight)`

    **Best for**: Platform-wide rankings that reward both accuracy and participation

    **Note**: Requires many questions (50+) to be meaningful
  </Tab>

  <Tab title="Baseline Global">
    Sum of baseline scores across all qualifying questions.

    **Formula**: `Total = Σ(baseline_score × question_weight)`

    **Best for**: Measuring absolute forecasting accuracy at global scale
  </Tab>

  <Tab title="Peer Global Legacy">
    Simple average of peer scores (older algorithm).

    **Status**: Used for historical leaderboards pre-2024
  </Tab>
</Tabs>

### Specialized Score Types

<AccordionGroup>
  <Accordion title="Comment Insight" icon="message">
    H-index of upvotes for comments on questions.

    **Calculation**: Your comment insight score is N if you have N comments with at least N upvotes each.

    **Use**: Rewards helpful commentary and explanation
  </Accordion>

  <Accordion title="Question Writing" icon="pen">
    H-index of (number of forecasters / 10) on authored questions.

    **Calculation**: Your question writing score is N if you wrote N questions with at least N×10 forecasters each.

    **Use**: Rewards writing popular, engaging questions
  </Accordion>

  <Accordion title="Manual" icon="hand">
    Does not automatically update. Entries must be manually set.

    **Use**: Special awards, judged competitions, or custom scoring systems
  </Accordion>
</AccordionGroup>

## Question Filtering

Leaderboards filter which questions contribute to rankings.

### Tournament Leaderboards

From `scoring/models.py:224-238`:

```python theme={null}
def get_questions(self) -> QuerySet[Question]:
    questions = Question.objects.filter(
        post__curation_status=Post.CurationStatus.APPROVED
    )
    
    if not (self.project and self.project.type == Project.ProjectTypes.SITE_MAIN):
        # Normal project leaderboard
        if self.project:
            questions = questions.filter(
                Q(post__projects=self.project) | Q(post__default_project=self.project)
            )
        return questions.distinct("id")
```

Tournament leaderboards include questions where:

* Post is approved (curated)
* Post's default project OR tagged projects include the tournament

### Global Leaderboards

From `scoring/models.py:240-297`, global leaderboards have strict time filtering:

```python theme={null}
# Questions must:
# 1. Be public
questions = questions.filter_public().filter(
    post__in=Post.objects.filter_for_main_feed()
)

# 2. Have time boundaries within leaderboard period
questions = questions.filter(
    open_time__gte=self.start_time,
    open_time__lt=self.end_time,
    scheduled_close_time__lte=self.end_time + close_grace_period,
)

# 3. Exclude questions that belong to other overlapping leaderboards
# (Questions only count for the shortest matching time window)
```

<Info>
  Global leaderboards ensure questions only count toward ONE global leaderboard period, preventing double-counting.
</Info>

## Time Boundaries

Leaderboards have three key timestamps:

### Start Time and End Time

From `scoring/models.py:154-170`:

```python theme={null}
start_time = models.DateTimeField(
    null=True,
    blank=True,
    help_text="""Optional (required for global leaderboards).
    Global Leaderboards: filters for questions that have an open time after this.
    Non-Global Leaderboards: has no effect on question filtering.
    Filtering MedalExclusionRecords: MedalExclusionRecords that have no end_time
    or an end_time greater than this will be triggered.
    """,
)

end_time = models.DateTimeField(
    null=True,
    blank=True,
    help_text="""Optional (required for global leaderboards).
    Global Leaderboards: filters for questions that have a scheduled_close_time
    before this (plus a grace period).
    """,
)
```

### Finalize Time

From `scoring/models.py:172-183`:

```python theme={null}
finalize_time = models.DateTimeField(
    null=True,
    blank=True,
    help_text="""Optional. If not set, the Project's close_date will be used instead.
    For all Leaderboards: used to filter out questions that have a
    resolution_set_time after this (as they were resolved after this
    Leaderboard was finalized).
    """,
)

finalized = models.BooleanField(
    default=False,
    help_text="If true, this Leaderboard's entries cannot be updated except by manual action.",
)
```

<Steps>
  <Step title="Active Period">
    Leaderboard tracks questions between `start_time` and `end_time`
  </Step>

  <Step title="Resolution Period">
    Questions resolve and scores are calculated (can extend past `end_time`)
  </Step>

  <Step title="Finalization">
    At `finalize_time`, leaderboard locks and becomes immutable
  </Step>
</Steps>

## Prize Distribution

### Prize Pool

From `scoring/models.py:184-201`:

```python theme={null}
prize_pool = models.DecimalField(
    default=None,
    decimal_places=2,
    max_digits=15,
    null=True,
    blank=True,
    help_text="""Optional. If not set, the Project's prize_pool will be used.
    If the Project has a prize pool, but this leaderboard has none, set this to 0.
    """,
)

minimum_prize_amount = models.DecimalField(
    default=50.00,
    decimal_places=2,
    max_digits=15,
    help_text="""The minimum amount a user can win in this leaderboard.
    Any remaining money is redistributed. Tournaments that close before June 2025
    will have a value of 0.00.
    """,
)
```

### Prize Calculation

Typical prize distribution:

1. Rank users by total score (descending)
2. Apply medal exclusions (see below)
3. Distribute prize pool according to curve (e.g., top 10% of prize pool to 1st place)
4. Apply minimum prize floor - redistribute amounts below minimum

<Tip>
  The `minimum_prize_amount` prevents many small payouts. Set to $50 means prizes below $50 are redistributed to higher performers.
</Tip>

## Bot Handling

Leaderboards control how bots are ranked and displayed.

### Bot Status

From `scoring/models.py:202-209` and `projects/models.py:248-264`:

```python theme={null}
bot_status = models.CharField(
    max_length=32,
    choices=Project.BotLeaderboardStatus.choices,
    null=True,
    blank=True,
    help_text="Optional. If not set, the Project's bot_leaderboard_status will be used.",
)

class BotLeaderboardStatus(models.TextChoices):
    EXCLUDE_AND_HIDE = "exclude_and_hide"  # Bots hidden completely
    EXCLUDE_AND_SHOW = "exclude_and_show"  # Bots shown but no prizes (default)
    INCLUDE = "include"                    # Bots compete for prizes
    BOTS_ONLY = "bots_only"               # Only bots compete
```

<Warning>
  Most leaderboards use `EXCLUDE_AND_SHOW` - bots appear for benchmarking but don't win prizes or count in official rankings.
</Warning>

## User Filtering

Leaderboards can restrict participation to specific users.

### User List

From `scoring/models.py:210-217`:

```python theme={null}
user_list = models.ManyToManyField(
    User,
    blank=True,
    help_text="""Optional. If not set, all users with scores will be included.
    If set, only users in this list will be included.
    Exclusion Records still apply independent of this list.
    """,
)
```

**Use cases**:

* Invite-only competitions
* Organization-specific leaderboards
* Qualifying tournaments with pre-approved participants

## Display Configuration

Leaderboards can customize their presentation.

### Display Config JSON

From `scoring/models.py:112-128`:

```python theme={null}
display_config = models.JSONField(
    null=True,
    blank=True,
    help_text=(
        "Optional JSON configuration for displaying this leaderboard."
        "If not set, default display settings will be used."
        "Example display_config:"
        """{
            "display_name": "My Custom Leaderboard",
            "column_renames": {
                "Questions": "Question Links"
            },
            "display_order": 1,
            "display_on_project": true
        }"""
    ),
)
```

**Configuration options**:

* `display_name`: Override default leaderboard name
* `column_renames`: Custom column headers
* `display_order`: Sort order when project has multiple leaderboards
* `display_on_project`: Whether to show on project page

## Global Leaderboard Periods

Metaculus maintains time-based global leaderboard periods.

### Period Generation

From `scoring/models.py:300+`, global leaderboards follow patterns:

```python theme={null}
GLOBAL_LEADERBOARD_STRING = "Leaderboard"
GLOBAL_LEADERBOARD_SLUG = "leaderboard"

def global_leaderboard_dates() -> list[tuple[datetime, datetime]]:
    # Returns list of (start_date, end_date) tuples for each period
    # Typically: quarterly periods (Q1 2024, Q2 2024, etc.)
```

### Period Resolution

From `questions/models.py:385-419`, questions are matched to the **shortest** overlapping period:

```python theme={null}
def get_global_leaderboard_dates(self) -> tuple[datetime, datetime] | None:
    forecast_horizon_start = self.open_time
    forecast_horizon_end = self.scheduled_close_time
    
    # Find shortest window that contains this question's forecast period
    shortest_window = None
    for gl_start, gl_end in gl_dates[::-1]:  # Reverse order (shortest first)
        if forecast_horizon_start < gl_start:
            continue
        if forecast_horizon_end > gl_end + timedelta(days=3):
            continue
        if shortest_window is None or (gl_end - gl_start) < window_size:
            shortest_window = (gl_start, gl_end)
```

<Info>
  Questions only contribute to ONE global leaderboard - the shortest period that fully contains their forecasting window.
</Info>

## Leaderboard Entries

Leaderboard rankings are stored as entries (from `scoring/models.py:97`):

```python theme={null}
class Leaderboard(TimeStampedModel):
    entries: QuerySet["LeaderboardEntry"]  # Related model
```

Each entry represents one user's standing:

* User reference
* Total score
* Rank
* Prize amount (if applicable)
* Number of questions forecasted
* Coverage statistics

## Best Practices

<CardGroup cols={2}>
  <Card title="Forecast Broadly" icon="arrows-maximize">
    Global leaderboards reward breadth - forecast on many questions across topics
  </Card>

  <Card title="Maintain Coverage" icon="clock">
    Keep forecasts active throughout question periods to maximize coverage weights
  </Card>

  <Card title="Start Early" icon="flag">
    Tournament leaderboards often reward early participation with higher coverage
  </Card>

  <Card title="Track Your Rank" icon="chart-line">
    Monitor leaderboard position to understand your standing and adjust strategy
  </Card>

  <Card title="Read Leaderboard Rules" icon="book">
    Each leaderboard may use different scoring - understand which score type applies
  </Card>

  <Card title="Focus Quality" icon="star">
    One well-researched forecast beats many hasty predictions
  </Card>
</CardGroup>

## API Reference

<Card title="Leaderboards API" icon="code" href="/api/leaderboards">
  Explore the full Leaderboards API documentation
</Card>

## Related Topics

<CardGroup cols={2}>
  <Card title="Scoring" icon="trophy" href="/features/scoring">
    Understand how individual scores are calculated
  </Card>

  <Card title="Tournaments" icon="medal" href="/features/tournaments">
    Learn about competitive forecasting events
  </Card>

  <Card title="Projects" icon="folder" href="/features/projects">
    Understand project-leaderboard relationships
  </Card>

  <Card title="Questions" icon="circle-question" href="/features/questions">
    Learn which questions contribute to leaderboards
  </Card>
</CardGroup>