id: d4c6a4e614d84802a4dc103afff359fb
parent_id: bfc4f723ff2c475a8bf7c4814a69b28f
item_type: 1
item_id: 47031623e2c6451fb36730ffe93380ea
item_updated_time: 1785502461192
title_diff: "[]"
body_diff: "[{\"diffs\":[[0,\"-07-\"],[1,\"3\"],[0,\"1\"],[-1,\"7\"],[0,\" (rev \"],[-1,\"5\"],[1,\"6\"],[0,\").\\\n\\\n\"]],\"start1\":25,\"start2\":25,\"length1\":17,\"length2\":17},{\"diffs\":[[0,\"tion\"],[-1,\", **22 registered strategies**\"],[0,\"\\\n- G\"]],\"start1\":191,\"start2\":191,\"length1\":38,\"length2\":8},{\"diffs\":[[0,\"lds \"],[-1,\"— **80+ PostflopThresholds fields** with TOML overrides\\\n- Experiment GUI (`holdem_gui`) — web-based parameter sweep tool (84 params)\"],[1,\"with TOML overrides\"],[0,\"\\\n- `\"]],\"start1\":252,\"start2\":252,\"length1\":140,\"length2\":27},{\"diffs\":[[0,\"ssion\\\n- \"],[-1,\"**\"],[0,\"PEM2 pre\"]],\"start1\":345,\"start2\":345,\"length1\":18,\"length2\":16},{\"diffs\":[[0,\"y matrix\"],[-1,\"**\"],[0,\" — 169×1\"]],\"start1\":371,\"start2\":371,\"length1\":18,\"length2\":16},{\"diffs\":[[0,\"trix\"],[-1,\" (10 metrics × 8 bins)\"],[0,\"\\\n\\\n--\"]],\"start1\":405,\"start2\":405,\"length1\":30,\"length2\":8},{\"diffs\":[[0,\" Gen 2-4\"],[-1,\".8\"],[0,\" Strateg\"]],\"start1\":427,\"start2\":427,\"length1\":18,\"length2\":16},{\"diffs\":[[0,\"LETE\"],[-1,\" (in production)\"],[0,\"\\\n\\\n-\"],[-1,\" ✅\"],[0,\" Gen\"]],\"start1\":463,\"start2\":463,\"length1\":29,\"length2\":11},{\"diffs\":[[0,\"gine\"],[-1,\" + KEHU Nash HU + NG preflop\\\n- ✅\"],[1,\"\\\n-\"],[0,\" Gen\"]],\"start1\":503,\"start2\":503,\"length1\":40,\"length2\":10},{\"diffs\":[[0,\"ty (\"],[-1,\"single predicted opponent\"],[1,\"heuristic\"],[0,\" range\"],[1,\"s\"],[0,\")\\\n-\"],[-1,\" ✅\"],[0,\" Gen\"]],\"start1\":533,\"start2\":533,\"length1\":44,\"length2\":27},{\"diffs\":[[0,\"ng (\"],[-1,\"EMA observer, three-tier blend, range narrowing)\\\n- ✅ Gen 4.5: Playability-enhanced preflop + adaptive postflop\\\n- ✅ Gen 4.7: Matrix-based equity preflop (PEM2)\\\n- ✅ **Gen 4.8: F\"],[1,\"G45/G47/G48) — f\"],[0,\"ormu\"]],\"start1\":589,\"start2\":589,\"length1\":183,\"length2\":24},{\"diffs\":[[0,\"a-based \"],[-1,\"postflop \"],[0,\"with obs\"]],\"start1\":614,\"start2\":614,\"length1\":25,\"length2\":16},{\"diffs\":[[0,\"ver \"],[-1,\"ranges — CURRENT PRODUCTION POSTFLOP**\\\n  - Unified score: `value_score + ppot_w×ppot - npot_w×npot`\\\n  - `value_score = hs×(1 - npot_w×npot) + nutredraw_w×nutpot×(1-hs) - behind_penalty_w×(1-hs)×(1-ppot)`\\\n  - call_threshold with multiway pot-odds cap: `0.28/√opp` (non-river), `0.15/√opp` (river)\\\n  - Bluff gating on weak opponents, stack-off AllIn guard, position-aware sizing\\\n  - Observer-based range equity (shared with G45 via `build_ranges_and_equity()`)\\\n  - Cold-start priors: VPIP 0.30, PFR 0.15, AF 1.0, fold_to_bet 0.50\\\n  - MC budget: 40,000 samples (divided by opponent count, floor 8K)\\\n  - **Config: `cash_nl_g48_live.toml`**\\\n\\\n### G47 Preflop — CURRENT PRODUCTION PREFLOP\\\n-\"],[1,\"EMA stats\\\n\\\n**Current production formula: G47 preflop + G48 postflop**\\\n- G47:\"],[0,\" PEM\"]],\"start1\":632,\"start2\":632,\"length1\":691,\"length2\":84},{\"diffs\":[[0,\"uity\"],[1,\",\"],[0,\" \"],[-1,\"(\"],[0,\"zero MC\"],[-1,\" — pure lookup)\\\n- Unified score: `equity + ppot_w×ppot + nutpot_w×nutpot - npot_w×npot`\\\n- Position-based thresholds, stack-depth sigmoid\\\n- Per-opponent PFR/VPIP for reraise/call paths (with table-level fallback)\\\n- **Config: `cash_nl_g47_live.toml`**\\\n\\\n### Recent Bug Fixes (2026-07-16)\\\n- ✅ Fixed `remaining_stack_share` double-counting bug (formula_postflop:887)\\\n- ✅ Relaxed multiway call margin 0.35→0.28 for non-river streets\\\n- ✅ Postflop fold logging upgraded to info level\\\n- ✅ PlayerId seat mismatch fix (bot_id tracking through seat changes)\\\n- ✅ Call amount reconciliation (downward + upward current_bet correction)\\\n- ✅ Blinds synthesis for missing blind-posting events\\\n- ✅ G47 call-path sizing narrowing (`call_range` narrows by bet sizing)\\\n- ✅ G48 KK overplay on Ace-high boards — board-hit boost (narrow_board_hit_boost=3.0)\\\n- ✅ Per-player looseness (`aggressor_vpip()` keys off bettor's individual VPIP)\\\n- ✅ Per-opponent PFR in G47 reraise path (was table-level)\\\n- ✅ RaiseBelowMinimum fix (BB rounding clamp in raise sizing)\\\n- ✅ Draw-heavy board trap guard (`board_is_drawy()` in gen3 + formula_postflop)\\\n- ✅ Deep-stack overbet allin fix (raise to opponent effective stack, not full shove)\\\n- ✅ Behind-penalty in value_score (behind_penalty_weight=0.15)\\\n\\\n### Open Tuning Items\\\n- [x] ~~Postflop equity matrix~~ — DEFERRED (MC rollouts are fine for live play)\\\n- [x] All 2026-07-16 tuning items complete (see above)\\\n- [x] Chump_bot RaiseBelowMinimum — DONE (BB rounding clamp)\\\n\\\n---\\\n\\\n## Phase 3: Live Integration (Torn) ✅ FUNCTIONAL\\\n\\\n### Completed\\\n- Event-stream protocol (`poker_protocol::TableEvent`)\\\n- `live_server` (Axum gateway) — per-table state machine, bot auto-detection\\\n- Torn TamperMonkey scraper v0.1.11 — DOM parsing (seats, stacks, cards, actions, blinds)\\\n- Profile persistence (player ID keyed, survives restarts)\\\n- Graceful shutdown/restart endpoints\\\n- **Default bot: G48 live (primary) + G47 live (fallback)**\\\n- Per-hand event/decision logging in `/tmp/super_marvin_hands/`\\\n- ✅ **DOM snapshot mode** — `dom_snapshot` on `waiting_for_action` + `raw_text` on `player_action` events. Userscript captures table HTML (gated by `window.__marvinDomDebug = true`). Server logs snapshots to per-hand decision logs.\\\n- ✅ Raise amount regex fix (k/M suffix handling in parser.ts)\\\n- ✅ Corrupt blinds false positive fix (silent when sb/bb non-zero but not yet seen)\\\n- ✅ Raise tolerance fix (1% → 5% for Torn chip denomination rounding)\\\n\\\n### Open Issues\\\n- [ ] Multi-table support\\\n- [ ] Native desktop scrapers (deferred)\\\n\\\n---\\\n\\\n## Phase 4: Gen 5 Reinforcement Learning 🔄 IN PROGRESS\\\n\\\n> Gen5 data collection RESUMED with G48 teacher (2026-07-17).\\\n\\\n### Implemented\\\n- **\"],[1,\", position-based thresholds\\\n- G48: Unified score formula with observer-based range equity, MC budget 10K-40K\\\n\\\n---\\\n\\\n## Phase 3: Live Integration (Torn) ✅ FUNCTIONAL\\\n\\\n- `live_server` (Axum gateway), Torn TamperMonkey scraper\\\n- Default bot: G49 (G48 formula + RangeNet)\\\n- Profile persistence, graceful shutdown/restart\\\n\\\n---\\\n\\\n## Phase 4: Gen 5 — Range Prediction NN ✅ DEFINED\\\n\\\n**Gen 5 = G48 formula + RangeNet (neural opponent range prediction)**\\\n\\\nOne NN per generation:\\\n- Gen 5: RangeNet (1 NN) — predicts opponent hand distribution (169-dim softmax)\\\n- Gen 6 (future): Gen 5 + additional NN\\\n\\\n### Dropped Approaches\\\n- **Direct-action Gen5** (imitation/Q-reg on \"],[0,\"290-dim\"],[-1,\"**\"],[0,\" feature\"],[-1,\" extraction (equity, board, betting, opponents, actions, range-weighted + multiway)\\\n- 11-discrete-action space with legality masking\\\n- JSONL transition recorder (thread-safe, bounded)\\\n- Candle MLP policy network (290→512→LN→512→LN→256→3+5+1 multi-head)\\\n- Training CLI with GPU/CUDA, class-weighted loss, focal loss, gradient clipping\\\n- Multi-head policy (action + sizing heads, class weights)\\\n- DAgger strategy (teacher fallback + \"],[1,\"s): 71.5% accuracy, catastrophic mistakes. Action selection without opponent range is fundamentally blind.\\\n- **Equity NN** (`model_infer_equity`): Works in 6-max (+121 BB/100) but fails in 9-max (-148 BB/100). Redundant with G48's MC.\\\n- **Combined inference** (`\"],[0,\"model\"],[-1,\" \"],[1,\"_\"],[0,\"infer\"],[-1,\"ence)\\\n- PPO self-play loop (v11 = best deployed model, +27.9 avg chips/hand)\\\n- ✅ **G48 teacher wiring** — `teacher_postflop` config field: `\\\"formula\\\"` → `FormulaPostflopGen48`, `\\\"adaptive\\\"` → `AdaptivePostflopGen4`\\\n\\\n### Data Collection (2026-07-17)\\\n- **Short-handed batch (3-6 handed)**: COMPLETE — 24.1M transitions from 10M hands (39GB)\\\n- **Full-ring batch (7-8 handed)**: RUNNING — 8 tables × 1.25M hands, ETA ~23h\\\n- Combined target: ~44M transitions covering full Torn seat distribution\\\n- Teacher: G47 preflop (PEM2) + G48 postflop (formula, 10K MC budget for collection speed)\\\n- Config: `gen5_collect_g48.toml` (mirrors `cash_nl_g48_live.toml`)\\\n- Scripts: `gen5_collect_g48.sh` (short-handed), `gen5_collect_g48_fullring.sh` (full-ring)\\\n\\\n### Models\\\n| Version | Training | Eval Acc | Statu\"],[1,\"_combined`): Single-street EV too aggressive (-157 to -2140 BB/100).\\\n\\\n### RangeNet Architecture\\\n```\\\nInput(81) → Linear(512) → ReLU → LayerNorm\\\n         → Linear(512) → ReLU → LayerNorm + residual\\\n         → Linear(512) → ReLU → LayerNorm + residual\\\n         → Linear(256) → ReLU\\\n         → Linear(169) → softmax\\\n```\\\n\\\n81 features = 49 base + 32 rich action history (8 per street × 4 streets).\\\n\\\n### Range Model History\\\n| Version | Dims | Top-1 | Top-10 | Note\"],[0,\"s |\\\n\"]],\"start1\":727,\"start2\":727,\"length1\":3941,\"length2\":1419},{\"diffs\":[[0,\"-----|------\"],[-1,\"----|---\"],[1,\"|\"],[0,\"-------|----\"]],\"start1\":2151,\"start2\":2151,\"length1\":32,\"length2\":25},{\"diffs\":[[0,\"---|\"],[1,\"-------|\"],[0,\"\\\n| v1\"],[1,\"0\"],[0,\"-v\"],[-1,\"4 | Various | Degenerate | Fixed (LayerNorm, action mapping, features) |\\\n| v5-v10 | Imitation (G4.5 teacher) | 45-46% | Production baseline |\\\n| v11 | 3.15M transitions, 290-dim, GPU | 45.2% | Previous best (+27.9 chips/hand) |\\\n| v12-v13 | G47 teacher, class weights | 97.6% action, 76.6% sizing | Plateaued — teacher quality ceiling |\\\n| **v14** | **G48 teacher, ~44M transitions** | **TBD** | **Next: train after collection** |\\\n\\\n### Next Steps\\\n1. ✅ Wire G48 as teacher (`teacher_postflop = \\\"formula\\\"`)\\\n2. ✅ Collect short-handed data (24.1M transitions)\\\n3. ⏳ Collect full-ring data (~20M transitions,\"],[1,\"13 | 57 | ~3% | ~19% | Hit ceiling with coarse 8-dim action history |\\\n| v14 (target) | 81 | TBD | 30%+ target | Rich 32-dim action history |\\\n\\\n### Current Pipeline\\\n1. Collect range data with 81-dim features (`equity_v3/`, 8 tables\"],[0,\" run\"]],\"start1\":2177,\"start2\":2177,\"length1\":614,\"length2\":253},{\"diffs\":[[0,\"ng)\\\n\"],[-1,\"4. ⬜ Train v14 on merged G48 teacher transitions\\\n5. ⬜ Evaluate v14 vs G48 in simulation\\\n6. ⬜ PPO fine-tuning (if imitation plateaus again)\"],[1,\"2. Merge range samples → train range_v14\\\n3. Update G49 config with range_v14\\\n4. Alpha sweep (NN vs heuristic blend ratio)\\\n5. Sanity check before live promotion\"],[0,\"\\\n\\\n--\"]],\"start1\":2432,\"start2\":2432,\"length1\":146,\"length2\":167},{\"diffs\":[[0,\"-07-\"],[1,\"3\"],[0,\"1\"],[-1,\"7\"],[0,\")\\\n\\\n|\"]],\"start1\":2622,\"start2\":2622,\"length1\":10,\"length2\":10},{\"diffs\":[[0,\"--|\\\n\"],[-1,\"| Registered strategies | 22 |\\\n\"],[0,\"| Pr\"]],\"start1\":2665,\"start2\":2665,\"length1\":39,\"length2\":8},{\"diffs\":[[0,\"** (\"],[-1,\"`g47_equity_preflop` — \"],[0,\"PEM2\"]],\"start1\":2697,\"start2\":2697,\"length1\":31,\"length2\":8},{\"diffs\":[[0,\"2 matrix\"],[1,\" equity\"],[0,\") |\\\n| Pr\"]],\"start1\":2704,\"start2\":2704,\"length1\":16,\"length2\":23},{\"diffs\":[[0,\"** (\"],[-1,\"`g48_formula_postflop` — \"],[0,\"form\"]],\"start1\":2752,\"start2\":2752,\"length1\":33,\"length2\":8},{\"diffs\":[[0,\"|\\\n| \"],[-1,\"Live integration | Torn — functional, userscript v0.1.11 |\\\n| MC budget (live) | 40,000 (divided by opponents, floor 8K) |\\\n| PEM2 matrix | 8.7 MB, 10 metrics × 8 bins × 169² pairs |\\\n| Regression tests | 49 hand replay tests + 478 lib tests (all pass) |\\\n| Gen 5 transitions | 24.1M (short-handed done) + ~20M (full-ring in progress) |\\\n| Live sessions | 2026-07-08 (~42.5M winnings), 2026-07-16 (clean, no issues) |\\\n\\\n---\\\n\\\n## Immediate Next Session Priorities\\\n\\\n1. **Wait for full-ring collection** (~23h, auto-running) → merge both batches\\\n2. **Train v14** — `train_gen5_multi <merged> 30 1024 1e-4 models/postflop_v14.safetensors --street postflop` (needs `--features gen5_nn`)\\\n3. **Evaluate v14** vs G48 in simulation — target: beat G48 or match with lower latency\\\n4. **PPO fine-tuning** if v14 imitation plateaus (REINFORCE/PPO on RL-sampled transitions)\\\n5. **Live monitoring** — keep watching for reconciliation warnings, test DOM snapshot mode\\\n6. **Update `gen5_rl.toml`** deploy config to point at v14 multi-head model + norm.json\\\n\"],[1,\"Production range | **RangeNet** (Gen 5) |\\\n| Configs | `cash_nl_g49.toml` (sim), `cash_nl_g49_live.toml` (live) |\\\n| Harrington tests | 18/18 pass |\\\n| Sanity check | Gen2 -0.1, Gen3 +0.1, G48 +1.2 BB/100 (TAG/Nit pending) |\\\n| Range features | 81 dims (49 base + 32 action history) |\\\n| Action space | 11 actions (fold/check/call + 5 bet sizes + 2 raise sizes + allin) |\"]],\"start1\":2783,\"start2\":2783,\"length1\":1037,\"length2\":370}]"
metadata_diff: {"new":{},"deleted":[]}
encryption_cipher_text: 
encryption_applied: 0
updated_time: 2026-07-31T12:59:40.345Z
created_time: 2026-07-31T12:59:40.345Z
is_locked: 0
type_: 13