Gen 5 — RangeNet (State of the Art)

# Gen 5 — RangeNet (State of the Art)

> **Declared 2026-07-31.** Gen 5 = G48 formula + RangeNet (neural opponent range prediction). Pure NN (alpha=1.0), no heuristic blending. Config: `cash_nl_g50`.

## What Changed from Gen 4

Gen 4 (G48, `cash_nl_g48`) uses **heuristic ranges** built from observer stats (VPIP/PFR/AF → hand range narrowing). Gen 5 replaces that with a **neural network** (RangeNet) that predicts the opponent's hand distribution directly from observable features.

The formula (G47 preflop + G48 postflop) and MC equity engine are unchanged. Only the range source changes:

```
Gen 4:  observer stats → heuristic range builder → MC equity → formula decision
Gen 5:  observable features → RangeNet NN → 169-dim softmax → MC equity → formula decision
```

## RangeNet Architecture

```
Input(81) → Linear(512) → ReLU → LayerNorm
         → Linear(512) → ReLU → LayerNorm + residual
         → Linear(512) → ReLU → LayerNorm + residual
         → Linear(256) → ReLU
         → Linear(169) → softmax
```

Output: 169-dim softmax over hand types (13 pocket pairs, 78 suited, 78 offsuit).
Each type probability is divided among its combos (2-pass: count available combos per type after dead-card exclusion, then assign `type_prob / count` per combo).

## Feature Vector (81 dimensions = 49 base + 32 action history)

### Base Features (49)
| Category | Dims | Fields |
|---|---|---|
| Player EMA stats | 11 | vpip, pfr, af, fold_to_bet, bet/raise/call/cbet/threebet/wtsd/call_raise_freq |
| Confidence | 1 | hands_observed / 50 |
| Table stats | 2 | table_vpip, table_pfr |
| Position | 3 | normalized pos, is_sb, is_bb |
| Game context | 5 | ln(pot_bb), ln(stack_bb), num_active, num_seats, street |
| Board texture | 4 | high_rank, paired, monotone, connected |
| Board cards | 10 | 5 ranks + 5 suits |
| Max sizing per street | 4 | pot-fraction capped at 3× |
| In-hand action aggregates | 9 | aggressive/passive counts per street + total_wagered_bb |

### Rich Action History (32 = 8 per street × 4 streets)
| Feature | Description |
|---|---|
| num_raises | 0-3 |
| num_calls | 0-3 |
| total_wagered_bb | log normalized |
| max_bet_frac | largest bet as pot fraction |
| faced_bet | did player face aggression before acting? |
| first_action | action code |
| last_action | action code |
| aggression_ratio | raises / (raises + calls) |

Shared function `compute_rich_action_history()` in `range_recorder.rs` — used by both training and inference to guarantee consistency.

## Pure NN Decision (alpha=1.0)

Alpha blending: `final_range = alpha × NN_range + (1-alpha) × heuristic_range`

- **Gen 5 (production)**: `alpha = 1.0` (pure NN, heuristic output discarded)
- Configurable via TOML: `range_alpha_min`, `range_alpha_max` in `Gen48Config`
- `blend_alpha()` in `range_predictor.rs` interpolates by hands observed (min at 10 hands, max at 50)

### Why Pure NN?
- Sweep: pure NN (+138 BB/100 seed 42) > blend 0.3-0.7 (+80.5) > heuristic only (-0.5)
- NN is better even at cold start (0 hands): default stats + game context + action history is richer input than heuristic VPIP/PFR defaults
- Combo-count fix was critical: `probs_to_hand_range()` was inflating offsuit 3× vs suited. Fixed → turned blend from -16.4 to +15.1 BB/100

## Range Model History

| Version | Dims | Samples | Top-1 | Top-10 | Notes |
|---------|------|---------|-------|--------|-------|
| v10-v13 | 57 | 2-4M | ~3% | ~19% | Hit ceiling with coarse 8-dim action history |
| v14_b1 | 81 | 1M | 2.86% | 19.0% | Accuracy unchanged, but combo-count fix improved downstream equity |

**Accuracy ceiling**: ~19% top-10 across all versions. Bottleneck is inherent poker unpredictability, not features or data quantity.

## Training Pipeline

1. **Collection**: `equity_v3/` — 8 tables, formula-guided play, range-only recording (no transitions). ~6K samples/min.
2. **Rolling training**: `range_rolling_train.sh` auto-trains at each 1M-sample milestone.
3. **Trainer CLI**: `train_gen5_range range_merged.jsonl 30 512 1e-4 models/range_v14_bX.safetensors`

### Current State
- `models/range_v14_b1.safetensors`: 81-dim, 1M samples
- `equity_v3/`: ~1.7M / 4M target samples
- Batch 2 trains at 2M, batch 3 at 3M, final at 4M

## Validation Status (2026-07-31)

| Gate | Result |
|---|---|
| Harrington | 17/18 pass (Harr 4-5: raises turn with 99, book says check — judgment call) |
| Self-play audit | ✓ passed |
| Gen2 | ✓ +0.5 BB/100 |
| Gen3 | ✗ -0.1 BB/100 (marginal) |
| G48 | running |
| TAG/Nit | pending |

## Configs

| Config | Purpose |
|---|---|
| `cash_nl_g50.toml` | Gen 5 sim config (alpha=1.0, 10K MC budget) |
| `cash_nl_g50_live.toml` | Gen 5 live config (40K MC budget, profiles) |
| `gen5_equity_collect.toml` | Range data collection (formula-only teacher) |

## Key Files

| File | Content |
|---|---|
| `holdem_bots/src/gen5/range_net.rs` | RangeNet network + `RangeTrainer` |
| `holdem_bots/src/gen5/range_recorder.rs` | Feature extraction (81 dims), `compute_rich_action_history()` |
| `holdem_bots/src/gen4/range_predictor.rs` | `RangePredictor`, `probs_to_hand_range()` (combo-count fix), `blend_alpha()` |
| `holdem_bots/src/gen4/formula_postflop.rs` | G48 strategy + `Gen48Config` (alpha params, model path) |
| `holdem_bots/src/gen4/mod.rs` | `build_ranges_and_equity()` — blends NN + heuristic (alpha=1.0) |
| `holdem_bots/src/gen5/rl_strategy.rs` | RL strategy (teacher mode, range recording) |
| `holdem_bots/src/bin/train_gen5_range.rs` | Range model training CLI |
| `scripts/gen5_equity_collect.sh` | 8-table range collection |
| `scripts/range_rolling_train.sh` | Rolling training at 1M milestones |

## Dropped Gen 5 Approaches (historical)

These were attempted and abandoned before settling on RangeNet:

- **Direct-action DQN** (imitation/Q-reg on 290-dim features): 71.5% accuracy, catastrophic mistakes (All-In with weak draws). Action selection without opponent range info is fundamentally blind.
- **Counterfactual rollout training**: CF rollouts gave per-action EV targets for dueling DQN. Fold model was biased → AllIn 43.9% of time → -849 BB/100.
- **Equity NN** (`model_infer_equity`): Pure value betting works in 6-max (+121 BB/100) but fails in 9-max (-148 BB/100). Redundant with G48's MC computation.
- **Combined inference** (`model_infer_combined`): Single-street EV formula too aggressive. -157 to -2140 BB/100.
- **FoldEquityNet**: Overfit, insufficient data.

## Where This Goes Next

- **Gen 5 tuning**: faster NN inference (sim bottleneck), better training data, formula parameter sweep with NN ranges
- **Gen 6 (future)**: RL self-play for continuous improvement — use Gen 5 formula as teacher, RL fine-tunes against self-play data

id: c7a39e16770542a08004fd73e971ca1f
parent_id: 5a06903f7db44bfcb4c8c8a9cf0d2326
created_time: 2026-06-28T05:17:26.889Z
updated_time: 2026-07-31T18:40:59.603Z
is_conflict: 0
latitude: 0.00000000
longitude: 0.00000000
altitude: 0.0000
author: 
source_url: 
is_todo: 0
todo_due: 0
todo_completed: 0
source: joplin-desktop
source_application: net.cozic.joplin-desktop
application_data: 
order: 1782623846889
user_created_time: 2026-06-28T05:17:26.889Z
user_updated_time: 2026-07-31T18:40:59.603Z
encryption_cipher_text: 
encryption_applied: 0
markup_language: 1
is_shared: 0
share_id: 
conflict_original_id: 
master_key_id: 
user_data: 
deleted_time: 0
is_locked: 0
extracted_resource_ids: 
type_: 1