id: 4e2ce58716bf4588ba05a9918b205e3d
parent_id: b76fcf8c4cbc433e8f09277e3971b4a9
item_type: 1
item_id: 16ee20ba74264eb2a2bb0a20a9cea103
item_updated_time: 1785443669028
title_diff: "[]"
body_diff: "[{\"diffs\":[[0,\"er\\\n\\\n\"],[-1,\"Track BB/100 (candidate avg) across iterations. 9-max, 4 CAND + 5 OPP, 75 BB.\\\n\\\n## ⚠️ CRITICAL: All results before 2026-07-30 12:25 are INVALID\\\n\\\nThe `holdem_bots` binary was compiled WITHOUT `gen5_nn`/`gen5_cuda`. The `ModelWeights` \\\"stub\\\" is literally an empty placeholder (`_placeholder: ()`). Without `gen5_nn`, `model_infer()` always returns `None`, so ALL decisions fell back to the G48 teacher formula. The NN never fired once.\\\n\\\n**Symptom**: Identical self-play results for different model configs (both fall back to same teacher).\\\n\\\n**Root cause**: `cargo test --release -p hand_replay` (Gate 1) recompiles holdem_bots as a dependency without `gen5_cuda`, corrupting shared build artifacts.\\\n\\\n**Fix**: Sanity check script now verifies binary has \\\"Using CUDA\\\" string before running, rebuilds if missing.\\\n\\\n## Iteration History (INVALID — all G48 teacher, not NN)\\\n\\\n| Model | Date | vs Gen2 | vs Gen3 | vs G48 | vs TAG | Notes |\\\n|-------|------|---------|---------|--------|--------|-------|\\\n| v8/v12 (pre-fix) | 07-29 | -14.4 | -12.6 | +4.0 | — | G48 teacher, not NN |\\\n| v8/v12 (post-fix) | 07-30 | -0.1 | 0.0 | +37.2 | +37.3 | G48 teacher, not NN |\\\n| v6/v10 (post-fix) | 07-30 | -0.1 | 0.0 | +37.3 | +37.1 | G48 teacher, not NN |\\\n\\\n## Pre-bug History (from 07-22, OLD test setup — 5-bot tables)\\\n\\\n| Model | vs Gen2 | vs Gen3 | Notes |\\\n|-------|---------|---------|-------|\\\n| CF v6 (buggy) | — | -321 | Buggy EVs |\\\n| CF v7 + mask | — | -20.5 | First working aggressive masking |\\\n| Q-reg v1 | — | -47.2 | G48 teacher data only |\\\n| Q-reg v2 (=v6 prod) | +23.7 | +12.1 | Warm-start + combined data |\\\n\\\n## Current Models\\\n\\\n| Model | Corr | Status |\\\n|-------|------|--------|\\\n| postflop v6 | 0.533 | Production (best in play historically) |\\\n| postflop v8 | 0.581 | From scratch, Q-reg |\\\n| postflop v9 | 0.594 | Warm-start from v6, combined_v2 data |\\\n| preflop v10 | 0.904 | Production |\\\n| preflop v12 | 0.438 | From scratch, Q-reg |\\\n| preflop v13 | 0.925 | Warm-start from v10, combined_v2 data |\\\n| range_v10 | 3.4% top-1 / 20.9% top-10 | Best range model |\\\n\\\n## Next Steps\\\n\\\n1. **Re-run sanity checks** with correct binary (gen5_nn + CUDA)\\\n2. **Wait for selfplay_v2** clean collection (now running with real NN)\\\n3. **Train range_v12** from clean range data\\\n4. **Train postflop v10 + preflop v14**: warm-start from v9/v13, clean combined data, --range-model range_v12\\\n5. **Sanity check new models**\\\n6. **Record REAL results here**\\\n\\\n## Key Lessons\\\n\\\n1. **Binary verification is critical**: Always check `strings holdem_bots | grep \\\"Using CUDA\\\"` before running\\\n2. **Warm-start is critical**: Every from-scratch model was terrible\\\n3. **Combined data beats pure self-play**: Teacher + self-play outperforms either alone\\\n4. **min_raise bug fixed**: 556 forced-call conversions → 0. Preflop mask now respects dynamic min_raise.\\\n5. **Correlation metric is unreliable**: Higher corr ≠ better play\\\n6. **Train-inference mismatch**: All bug-era models trained on data where raises were silently converted to calls\"],[1,\"## Current: v10/v14/range_v12 (Q-reg, clean data)\\\n- **Status: FAILED** (2026-07-30)\\\n- Gate 1 (Harrington): 52/53 (Q9 G47 failure only)\\\n- Gate 2 (Self-play): ✅ Clean\\\n- Gate 3: Gen2 -17.0, Gen3 -18.0, G48 -5.5, TAG -4.7, Nit -12.9 — ALL NEGATIVE\\\n- Gate 4 (Mixed): FAILED\\\n- **Conclusion**: Q-regression on hand_reward is fundamentally flawed (no credit assignment, noisy)\\\n\\\n## Equity-Based Training Approach (NEW - 2026-07-30)\\\nInstead of training on noisy chip delta (hand_reward), train on **deterministic equity targets**:\\\n- Compute real equity from revealed opponent cards at hand end\\\n- Per-action Q targets from formula:\\\n  - Fold: 0\\\n  - Check: equity × pot_bb\\\n  - Call: equity × (pot + to_call) - to_call\\\n  - Bet/Raise: fold_equity × pot + (1-fold_equity) × (equity × (pot+bet) - bet)\\\n- Also tracks fold_equity_observed (did opponents fold to bet?)\\\n\\\n### Implementation Status\\\n- ✅ Added real_equity, fold_equity_observed, pot_bb, invested_bb, to_call_bb to Transition\\\n- ✅ Unified rl_pending buffer (all transitions go through cards_revealed_event)\\\n- ✅ Fixed event ordering: game_over_event fires BEFORE cards_revealed_event\\\n  - on_hand_end does NOT drain (deferred to cards_revealed_event)\\\n  - Safety net moved to on_hand_start\\\n- ✅ Added --equity mode to train_gen5.rs (computes targets from formula)\\\n- ✅ Verified: postflop transitions get valid equity (0.22-0.96 range)\\\n- 🔄 Collecting equity_v1 data (500K hands, 8 tables, teacher-guided)\\\n- ⬜ Train postflop EquityNet with --equity --postflop --qreg\\\n- ⬜ Evaluate vs Gen2/Gen3\\\n\\\n### Config\\\n- Collection: `gen5_equity_collect.toml` (teacher-guided, range_v12, v9/v13 models)\\\n- Training: `scripts/train_equity_postflop.sh` (warm-start from v9, range_v12)\\\n- Eval: gen5_v7_eval.toml (postflop → eq1 model)\\\n\\\n## Previous Results (all INVALID - binary lacked gen5_nn)\\\n| Model | vs Gen3 | Notes |\\\n|-------|---------|-------|\\\n| v9/v13 | -34.7 BB/100 | 2000 hands |\\\n| v6/v10 | -36.3 BB/100 | 2000 hands |\\\n\"]],\"start1\":37,\"start2\":37,\"length1\":2995,\"length2\":1948}]"
metadata_diff: {"new":{},"deleted":[]}
encryption_cipher_text: 
encryption_applied: 0
updated_time: 2026-07-30T20:39:31.598Z
created_time: 2026-07-30T20:39:31.598Z
is_locked: 0
type_: 13