ecw-charge-validation/docs/superpowers/specs/2026-09-15-payer-group-categorization-design.md

# Payer → Group Name categorization — design

Status: approved by user (Vasanth), 2026-09-15. Not yet implemented.

## Background

This is the next phase of the Salud eCW Charge Entry bot (see `README.md`),
which already automates login, 2FA, claim lookup, and exporting the
"Medcor Ready" claim queue to `Input\Claim Report_<date>.xlsx`.

The next manual step in the real process (shown in
`Salud-Charge Entry_Medcor Ready.mp4`, reviewed 2:45–10:00) is: an operator
renames a copy of that export to "Medcor ready", opens it in a dated network
folder (`...\Charge\Allocation\<Month YYYY>\<MMDDYYYY>\`, alongside sibling
workbooks `Patient`, `Pending with Error`, and `Program`), and manually
categorizes every row by filtering on the `PAYER` column and bulk-filling a
`Group Name` column with one of a small set of values (observed in the video:
`Commercial`, `Dentaquest`, `Medicaid`, `Medicare`, `Program`, plus at least
one more category for payers like `BH COLORADO ACCESS`).

This spec covers automating **only** that categorization step — not the
downstream split into the three sibling workbooks, which stays manual for now
and may be scoped separately later.

## Scope decisions (from user)

- **Scope**: categorize only. Do not build the split into
  Patient/Pending with Error/Program workbooks.
- **Mapping source**: derived from the actual current export
  (`Input\Claim Report_09-11-2026.xlsx`, 729 rows, 39 distinct `PAYER`
  strings), not invented from video frames. Only high-confidence entries are
  seeded; everything else is left for a human to fill in (see "Seed mapping
  table" below) — this is intentional, not a gap to silently paper over.
- **Unknown payer handling**: flag, never guess. A `PAYER` value with no
  entry in the mapping table gets a blank `Group Name`, is logged, and is
  called out in the run summary. No fuzzy/substring matching — exact string
  match only, so a typo'd or renamed payer can never be silently bucketed
  into the wrong group.
- **Integration point**: inside the existing bot, not a standalone script.
  Runs automatically as part of `flows/export_medcor_ready.py`.
- **Output location**: a new file, `Output\Medcor Ready_<date>.xlsx`. The
  original `Input\Claim Report_<date>.xlsx` is never modified — `claim_file.py`
  and everything downstream of it depends on that file's exact column
  layout (per `README.md`'s "Reading the claim ids back" section), so this
  step must not touch it.

## Architecture

```
flows/export_medcor_ready.py
  ... existing steps (login → claim lookup → export → claim_file.claim_ids()) ...
  → categorize_by_payer()                     NEW — reads Input\, writes Output\
      core/payer_groups.py                    NEW — the lookup table + classify()
        payer_groups.json                     NEW — editable config, exact PAYER → Group Name
      core/spreadsheet.py                     EXTENDED — write_xlsx(), stdlib only
```

`categorize_by_payer()` runs after the existing claim-id read-back (the point
where the report is already saved and validated — see
`README.md`'s "Reading the claim ids back"). It:

1. Reads `Input\Claim Report_<date>.xlsx` with the existing `spreadsheet.read_rows()`.
2. For each row, looks up `PAYER` in `payer_groups.classify()`.
3. Writes every original column plus a new `Group Name` column to
   `Output\Medcor Ready_<date>.xlsx`.
4. Logs and message-boxes a summary: rows categorized, rows left blank, and
   the distinct list of unmapped `PAYER` strings — mirroring the existing
   `NOTIFY_AFTER_EXPORT` / `SHOW_CLAIM_IDS` pattern in the same file.

Never fatal: an unmapped payer does not stop the run, the same philosophy
`export_claims_report()` already uses for other soft failures.

## Components

### `src/ecw_bot/core/payer_groups.py`

```python
def classify(payer: str) -> str | None: ...   # exact match against the config, or None
def load_mapping() -> dict[str, str]: ...      # reads payer_groups.json once
```

Exact-string lookup only. No normalization beyond what's needed to match
eCW's own values (the config keys are the literal `PAYER` strings as eCW
exports them).

### `src/ecw_bot/payer_groups.json`

A flat `{"<exact PAYER string>": "<Group Name>"}` map, hand-editable by
anyone on the team without touching code — same reasoning as `.env` for
settings and `selectors\` for DOM selectors: this data changes for a
different reason (a new payer shows up) than the code around it does.

### `core/spreadsheet.py` — new `write_xlsx()`

The project deliberately has no `openpyxl` dependency (see README: "every
added package is an approval"). `write_xlsx()` follows the same approach
already used for reading — a minimal single-sheet xlsx as a zip of XML, cells
as inline strings (no `sharedStrings.xml`, matching what eCW's own exports
already do, per the existing reader's comments) — built with `zipfile` and
string templates only.

## Data flow

```
Input\Claim Report_<date>.xlsx
  → read_rows()                     existing
  → for each row: classify(PAYER)   new
  → write_xlsx(rows + Group Name)   new
  → Output\Medcor Ready_<date>.xlsx
```

## Error handling

- Unmapped payer: blank `Group Name`, logged, summarized — not fatal.
- `Input\` file missing/unreadable: this step is skipped with a clear log
  line; it never blocks the rest of the run, since the export already
  succeeded by the time this step runs.
- `write_xlsx()` failure (disk full, permission): logged and raised — same
  severity as any other unexpected I/O failure in this flow, since silently
  losing the categorized output would defeat the point of the step.

## Testing

`tests/test_payer_groups.py`, hand-rolled in the existing style (no pytest):

- Exact-match lookups against a small fixture mapping.
- Unmapped payer → `None`, never a guess.
- Round-trip test for `write_xlsx()`: write a small fixture table, read it
  back with the existing `read_rows()`, assert the data matches — the same
  "never touched eCW" local-fixture philosophy as `test_claim_form.py`.

## Seed mapping table

Built from the real, current payer list (`Input\Claim Report_09-11-2026.xlsx`,
729 rows). Only high-confidence entries are seeded — this covers roughly 95%
of row volume in that file.

**Mapped (seeded):**

| Payer (exact eCW string) | Group |
|---|---|
| Medicaid | Medicaid |
| DentaQuest | Dentaquest |
| CHP+ DENTAQUEST | Dentaquest |
| AARP Medicare Advantage | Medicare |
| Aetna Medicare ADV | Medicare |
| OUTREACH PROGRAM | Program |
| Anthem Blue Cross Blue Shield | Commercial |
| Cigna HealthCare Of Colorado | Commercial |
| Cigna Healthcare of CO | Commercial |
| Cigna Salud Employee | Commercial |
| Cigna | Commercial |
| Cypress Benefit Admin / Cigna PPO | Commercial |
| United Healthcare | Commercial |
| United Health Care | Commercial |
| United healthcare | Commercial |
| Unitedhealth One | Commercial |
| Oxford Health Insurance UHC | Commercial |
| UMR | Commercial |
| Aetna | Commercial |
| Aetna Alaska Care | Commercial |
| AETNA HEALTH AND LIFE INSURANCE | Commercial |
| BIND BENEFITS INC | Commercial |
| Select Health | Commercial |

**Left unmapped — needs a human decision before first real run (or will be
flagged on first run and filled in afterward):**

BH COLORADO ACCESS (15 rows), Delta Dental of Colorado (3), Devoted (2),
Clinic Sliding Fee Scale (2), Patient (1), Gravie Admin Services (1),
Tricare (1), Rocky Mountain Health Plans (1), Denver Elevate Health Plan (1),
The Kempton Group Administrators Inc (1), Optum Care Network (1), GPA (1),
Northeast Health Partners (1), Delta Dental of Pennsylvania (1),
Delta Dental of Wisconsin (1), Delta Dental (1), Delta Dental of Arkansas (1).

## Out of scope (explicitly)

- Splitting rows into the Patient / Pending with Error / Program sibling
  workbooks.
- Writing to the network `Charge\Allocation\...` folder — output stays local
  to `Output\`.
- Any change to `Input\Claim Report_<date>.xlsx` or anything that reads it.