Skip to content

Configuration Guide

This guide explains how to configure the GA-Optimized Decision Trees framework using YAML configuration files.

Overview

All hyperparameters are externalized to YAML files for:

  • Reproducibility: Same config = same results
  • Experimentation: Easy parameter comparison
  • Version control: Track configuration changes
  • Sharing: Share configurations with collaborators

Configuration Structure

A complete configuration file has four main sections:

ga:           # Genetic algorithm parameters
tree:         # Tree constraints
fitness:      # Fitness function weights
experiment:   # Experiment settings

Available Configurations

The framework includes several standardized configuration files:

Config File Purpose Best For Key Features
paper.yaml ⭐ Larger research budget Longer runs pop 80 × 40 gens
fast.yaml Quick experiments Development, testing 3-5× faster
balanced.yaml Equal objectives General exploration 50/50 balance
accuracy_focused.yaml Max performance Production systems 85% accuracy weight
interpretability_focused.yaml Max transparency Medical, legal 60% interp weight
default.yaml Standard settings First-time users Good defaults

📄 Paper Configuration

paper.yaml carries the largest search budget of the shipped configs. There is no published research paper behind it — the values were fixed by hand, not tuned:

# configs/paper.yaml - largest search budget
ga:
  population_size: 80
  n_generations: 40
  crossover_prob: 0.72
  mutation_prob: 0.18
  tournament_size: 4
  elitism_ratio: 0.12

fitness:
  weights:
    accuracy: 0.68
    interpretability: 0.32

  interpretability_weights:
    node_complexity: 0.50
    feature_coherence: 0.10
    tree_balance: 0.10
    semantic_coherence: 0.30

tree:
  max_depth: 6
  min_samples_split: 8
  min_samples_leaf: 3
  growth_stop_prob: 0.3

Note: semantic_coherence is weighted 0.30 here. It's a search-guidance term, not a measure of interpretability.

Quick Start Examples

# Use paper configuration (recommended)
python scripts/experiment.py --config configs/paper.yaml

# Quick test
python scripts/train.py --config configs/fast.yaml --dataset iris

# Balance accuracy and interpretability
python scripts/experiment.py --config configs/balanced.yaml

# Maximum interpretability
python scripts/train.py --config configs/interpretability_focused.yaml --dataset breast_cancer

Configuration Details

GA Parameters

ga:
  population_size: 80          # Number of trees per generation
  n_generations: 40            # Number of evolution iterations
  crossover_prob: 0.72         # Probability of crossover (0-1)
  mutation_prob: 0.18          # Probability of mutation (0-1)
  tournament_size: 4           # Selection pressure (2-7)
  elitism_ratio: 0.12          # Top % preserved (0-0.3)

  mutation_types:              # Must sum to 1.0
    threshold_perturbation: 0.45
    feature_replacement: 0.25
    prune_subtree: 0.25
    expand_leaf: 0.05

Tree Constraints

tree:
  max_depth: 6                 # Maximum tree depth (3-10)
  min_samples_split: 8         # Min samples to split (2-20)
  min_samples_leaf: 3          # Min samples in leaf (1-10)
  growth_stop_prob: 0.3        # P(stop growing) per node when seeding (0-1)

Fitness Weights

fitness:
  mode: weighted_sum           # 'weighted_sum' or 'pareto'
  validation_fraction: 0.2     # Held out from the training data to score fitness on (0-1)
  classification_metric: accuracy  # 'accuracy', 'f1_macro', 'f1_weighted', 'balanced_accuracy'
  regression_metric: neg_mse       # 'neg_mse', 'r2'
  weights:                     # Must sum to 1.0
    accuracy: 0.68
    interpretability: 0.32

  interpretability_weights:    # Must sum to 1.0
    node_complexity: 0.50      # Penalize tree size
    feature_coherence: 0.10    # Reward feature reuse
    tree_balance: 0.10         # Prefer balanced trees
    semantic_coherence: 0.30   # Feature depth consistency

Tip: For imbalanced datasets, use classification_metric: f1_weighted or balanced_accuracy to prevent the GA from evolving majority-class-only trees.

validation_fraction controls what fitness is scored on. The leaves are fitted on the rest of the training data and the tree is scored on the held-out share, so the search can't win by memorising its own training rows. Once evolution ends, train.py and experiment.py refit the winning tree's leaves on all the training data. 0.0 scores fitness on the rows the leaves were fitted on. train.py --validation-fraction overrides it.

Choosing the Right Configuration

By Use Case

Use Case Configuration Why
Longest search runs paper.yaml Largest budget
Quick testing fast.yaml Faster iterations
Production deployment accuracy_focused.yaml Max performance
Medical diagnosis interpretability_focused.yaml Transparency required
General use balanced.yaml Good starting point

By Priority

Accuracy is most important (Competitions, production):

python scripts/train.py --config configs/accuracy_focused.yaml

Interpretability is most important (Healthcare, legal):

python scripts/train.py --config configs/interpretability_focused.yaml

Equal priority (Exploration):

python scripts/train.py --config configs/balanced.yaml

Creating Custom Configurations

Method 1: Copy and Modify

# Copy paper config as starting point
cp configs/paper.yaml configs/my_custom.yaml

# Edit my_custom.yaml with your preferences
nano configs/my_custom.yaml

# Use your config
python scripts/train.py --config configs/my_custom.yaml --dataset wine

Method 2: Create from Scratch

# configs/my_custom.yaml

# GA settings
ga:
  population_size: 100        # Your choice
  n_generations: 50
  crossover_prob: 0.75
  mutation_prob: 0.20
  tournament_size: 3
  elitism_ratio: 0.15

  mutation_types:
    threshold_perturbation: 0.45
    feature_replacement: 0.25
    prune_subtree: 0.25
    expand_leaf: 0.05

# Tree constraints
tree:
  max_depth: 7                # Deeper trees
  min_samples_split: 10
  min_samples_leaf: 4

# Fitness function
fitness:
  mode: weighted_sum
  weights:
    accuracy: 0.75            # More accuracy focus
    interpretability: 0.25

  interpretability_weights:
    node_complexity: 0.60
    feature_coherence: 0.20
    tree_balance: 0.10
    semantic_coherence: 0.10

# Experiment settings
experiment:
  datasets:
    - iris
    - wine
  cv_folds: 10
  random_state: 42

Parameter Guidelines

Population Size

  • Small (30-50): Fast, may miss optimal solutions
  • Medium (50-100): Good balance ✓ (paper uses 80)
  • Large (100-200): Better exploration, slower

Generations

  • Few (20-30): Quick experiments
  • Medium (30-50): Standard ✓ (paper uses 40)
  • Many (50-100): Thorough optimization

Accuracy Weight

  • 0.85-0.95: Maximum accuracy (production)
  • 0.68-0.75: Balanced ✓ (paper uses 0.68)
  • 0.50-0.60: High interpretability (medical)

Max Depth

  • 3-4: Very interpretable
  • 5-7: Good balance ✓ (paper uses 6)
  • 8-10: More complex, less interpretable

Configuration Presets Summary

# Paper configuration (largest search budget)
configs/paper.yaml
  - 80 population, 40 generations
  - Accuracy 0.68 / interpretability 0.32
  - No benchmark results are claimed for it

# Fast configuration (development)
configs/fast.yaml
  - 50 population, 30 generations
  - 3-5× faster than paper config
  - Good for quick iteration

# Balanced configuration (exploration)
configs/balanced.yaml
  - 50/50 accuracy-interpretability
  - 100 population, 40 generations
  - Good starting point

# Accuracy-focused (production)
configs/accuracy_focused.yaml
  - 85% accuracy weight
  - Deeper trees allowed (depth 8)
  - Maximum performance

# Interpretability-focused (high-stakes)
configs/interpretability_focused.yaml
  - 60% interpretability weight
  - Aggressive pruning
  - Very small trees

# Default configuration (beginners)
configs/default.yaml
  - Simple, reasonable defaults
  - 70/30 accuracy-interpretability
  - Easy to understand

Command-Line Overrides

Override config parameters from command line:

# Use paper config but change generations
python scripts/train.py --config configs/paper.yaml \
    --generations 60 \
    --dataset breast_cancer

# Override multiple parameters
python scripts/train.py --config configs/paper.yaml \
    --population 100 \
    --generations 50 \
    --max-depth 7 \
    --accuracy-weight 0.75

Configuration Best Practices

1. Start with Paper Config

# Always start here for research
python scripts/experiment.py --config configs/paper.yaml

2. Use Fast Config for Development

# Quick iteration during development
python scripts/train.py --config configs/fast.yaml --dataset iris

3. Version Control Your Configs

git add configs/my_experiment.yaml
git commit -m "Add config for medical diagnosis experiment"

4. Document Your Changes

# configs/my_experiment.yaml
# Author: Your Name
# Date: 2025-01-15
# Purpose: Optimize for medical diagnosis
# Changes from paper.yaml:
#   - Increased interpretability_weight to 0.45
#   - Reduced max_depth to 5
#   - Increased prune_subtree mutation to 0.35

ga:
  # ... your settings

Troubleshooting

Problem: Trees too large Solution: Use interpretability_focused.yaml or increase interpretability weight

Problem: Accuracy too low Solution: Use accuracy_focused.yaml or increase accuracy weight

Problem: Training too slow Solution: Use fast.yaml or reduce population/generations

Problem: Results not reproducible Solution: Ensure random_state: 42 in experiment section

Next Steps

Quick Reference

# Research (paper replication)
python scripts/experiment.py --config configs/paper.yaml

# Development (fast iteration)
python scripts/train.py --config configs/fast.yaml --dataset iris

# Production (max accuracy)
python scripts/train.py --config configs/accuracy_focused.yaml

# Medical/Legal (max interpretability)
python scripts/train.py --config configs/interpretability_focused.yaml

# General (balanced)
python scripts/experiment.py --config configs/balanced.yaml