ecw-charge-validation/README.md

# Salud eCW Charge Entry Bot

Single-client eCW automation for Salud. Login is done; the charge-entry phases
get built on top of the session it hands back.

Structurally a trimmed rework of the NAHMS Combined Reports bot — same portal,
same eCW V12 login markup, so the proven login handling is reused. What is
different: one client instead of a spreadsheet loop, credentials in `.env`
instead of `Input\input.xlsx`, and the second factor read from the mailbox
because this account has no authenticator secret.

The second factor is **not** a 6-digit code. eCW mails this account a
"Push Notification" containing an approval link
(`…/login/authenticate/openLoginConfirmation?token=<JWT>`, valid 10 minutes)
and the login page waits until it is confirmed. The bot reads the mail over
IMAP, opens the link in a side tab, clicks "Yes, it's me", and closes it.
`auth/mailbox.py` still handles a mailed 6-digit code and a TOTP secret, so a different
account needs no code change.

## Install

```bash
pip install -r requirements.txt
playwright install chromium
```

## Configure

`.env` at the project root holds every credential and is gitignored.
`.env.example` is the template.

## Run

```bash
python run.py
```

`run.py` is the only supported entry point. It puts `src\` on the import path
and calls the flow — which is why it exists: the old layout worked *only*
because Python happens to put a script's own directory on `sys.path`, so
`python src\main.py` resolved `import config` and nothing else did.
`python -m ecw_bot` works too, with `PYTHONPATH=src`.

Network preflight — run this first if anything about login or 2FA misbehaves:

```bash
python tools\check_net.py
```

Mailbox probe — run it if the second factor starts failing, or when pointing
the bot at a different account:

```bash
python tools\check_imap.py
```

It tries each plausible IMAP host, logs in, and prints recent subjects. As of
2026-09-01 the configured `cpanel1.methodhosting.net` passes.

Tests. No pytest — the hand-rolled `.env` parser exists for the same reason,
that every added dependency on these machines is one more approval:

```bash
python tests\test_settings.py
```

```bash
python tests\test_guard.py
```

```bash
python tests\test_dead_end.py
```

```bash
python tests\test_claim_file.py
```

```bash
python tests\test_claim_form.py
```

```bash
python tests\test_progress_note.py
```

```bash
python tests\test_input_archive.py
```

`test_claim_form.py` launches a **local headless chromium** against
`tests\fixtures\ub_claim.html`. It never touches eCW — no proxy, no
login, no 2FA mail, no attempt against the guard — so it costs nothing and can
be run as often as you like. That matters: the only other way to exercise that
code is a three-minute login and one of four hourly attempts.

## Layout

```
run.py                          the entry point
src\ecw_bot\
  settings.py                   .env values, and nothing else
  claim_file.py                 reads the saved report; finds CLAIMS#
  selectors\                    every CSS selector, and nothing else
    login.py  shell.py  claim_lookup.py  claim_form.py
  core\                         no eCW knowledge — reusable as-is
    browser.py                  chromium.launch(proxy=…) — see below
    page_utils.py               shot / visible / text_of / is_disabled / goto
    spreadsheet.py              xlsx / HTML / CSV reader, stdlib only
    msgbox.py                   operator prompts; every one times out
    guard.py                    attempt cooldown and burst limit
    runlog.py                   tees the run to Logs\run_<stamp>.log
  auth\
    login.py                    the login itself
    captcha.py                  two tries, incognito, one last try
    twofactor.py                the mailed approval link
    mailbox.py                  imaplib reader over a raw socket
    restricted.py               eCW's refusal page + the exception
  pages\                        one module per eCW screen
    leftnav.py                  left panel → Billing → Claims
    claim_filters.py            status, date type, date range, Balance
    claim_results.py            Lookup, and waiting for the grid honestly
    claim_report.py             Billing → View Claims Report → EXCEL
    claim_form.py               the UB Claim modal: open, read, close
    progress_note.py            the note: find it, read prisma-sections
  flows\
    export_medcor_ready.py      the step order, and the exit codes
tools\                          diagnostics you run by hand
  check_net.py  check_imap.py
tests\
  test_settings.py  test_guard.py
  test_dead_end.py  test_claim_file.py  test_claim_form.py
  test_progress_note.py  test_input_archive.py
  fixtures\ub_claim.html  fixtures\progress_note.html
  fixtures\progress_note_host.html
Logs\  Screenshots\  Output\            all gitignored
Input\  Claim Report_<date>.xlsx    the ONE current report
       Archive\                    every earlier one
```

Nothing imports a layer above it, so the graph is acyclic and any module can be
read without chasing a cycle. `tests\test_settings.py` asserts that, along with
`settings.ROOT` resolving to the project root and no selector leaking back into
`settings`.

**Why it is split this way.** It started as eleven flat modules in `src\`,
3,253 lines, with `claims.py` alone at 952 — nearly a third of the project in
one file — and `config.py` mixing 43 DOM selectors in with the date range and
the proxy. These things change for entirely different reasons: a selector
changes when eCW ships a new build, a setting when the process changes, a flow
when the work changes. In one file, editing a date range and editing a selector
were edits to the same place.

## Login flow

```
goto ECW_URL                    (retried — the proxy kills the first goto intermittently)
  → #doctorID       → #nextStep
  → #err check                  ("User is Inactive" fails fast)
  → security-image page         → message box, operator continues
  → #passwordField  → #Login
  → CAPTCHA         → 2 manual attempts → incognito relaunch → final manual attempt
  → 2FA             → mailbox poll, 10min → Resend OTP → 10min
                        (patient on purpose: the mail can take 10 minutes)
                        link  → open in side tab → "Yes, it's me" → page proceeds
                        code  → #OTPCode → #Submit
  → #jellybean-panelLink4       (home confirmed)
  → goto ECW_LANDING_URL        (eCW drops the fragment during the login redirect)
```

Screenshots of each notable step land in `Screenshots\<MM-DD-YYYY>\`.

## Navigation

```
#jellybean-panelLink4           (left panel — only if it is not already open)
  → #Billing_menu > a           (waited for: injected late)
  → #CLAIMS_BILLING span.padl10
  → claimLookup.jsp             (URL check + #routeLoader hidden + screenshot)
```

`pages/leftnav.py` is written defensively for three reasons, all of them properties of
eCW's own markup rather than caution for its own sake:

- **The hamburger is a toggle, not an "open".** `launchLeftPanel()` flips a
  `visible` class, so clicking it on an open panel closes it. State is checked
  before every click.
- **`#jellybean-panelLink4` matches two elements.** eCW renders
  `#JellyBeanCountCntrl` twice, and Playwright's strict mode refuses a plain
  `click()` on that. Every locator is `.first`.
- **The menu does not exist when login is confirmed.** `getleftnavhtml()`
  injects it on `DOMContentLoaded`, so `#Billing_menu` is waited for. A timeout
  there means the left nav never rendered — a different bug from a moved item,
  and the message says so.

Claims is clicked on its label span, not the anchor: the anchor also holds
`#favorite_16`, whose own `ng-click` stars the item instead of navigating.

If any of that fails the run does not stop — it navigates straight to
`CLAIMS_HASH`, the same hash the menu item's `openURL()` would have used, and
logs `[NAV] … falling back to the hash route`. Worth fixing the selector when
you see that line, rather than leaving the bot on the fallback permanently.

No selector on the claims screen itself is asserted — `claim_filters.py` waits for the
form's own controls, so nav only has to get the route right.

## Claim lookup

```
#claimStatusCodeId ← 108ME "Medcor Ready"        FIRST — everything hangs off it
span.toggledropdown[ng-bind='selectedDateType']   (reads "Service Dt(s)")
  → #claimLookupUl2 li a span:text-is("Claim Dt(s)")
  → toggle reads back "Claim Dt(s)"               (the confirmation)
#fromdate  ← CLAIM_FROM_DATE (06/01/2020)         Escape closes the datepicker
#todate    ← today, computed at run time
Balance row cleared — all five controls, when eCW has them enabled
  → already empty? left untouched, nothing to clear
  → filters logged + claims_before_lookup.png    (no box: CONFIRM_BEFORE_LOOKUP=0)
#btnclaimlookup                                   (NOT #btnclaimlookupClear)
  → poll "Total Counts" until it changes             (NOT networkidle)
#claimLookupBtn17  "Billing"                      toolbar, not the left nav
  → #claimLookupBtn71 "View Claims Report"        hover, submenu
  → button[ng-click*="EXCEL"]                     NOT #claimLookupBtn1002
  → archive every earlier report to Input\Archive\
  → Input\Claim Report_<MM-DD-YYYY>.xlsx     the ONE current report
  → message box: rows, file, size, folder
read the file back → the CLAIMS# column
  → message box: count, first 15, last 5   (full list to the log)
```

### Waiting for the results grid

`run_lookup` used to call `wait_for_load_state("networkidle")` after pressing
Lookup. **That never returns on this page.** eCW's Angular shell polls in the
background, so the network is never quiet for the 500 ms Playwright wants, and
the call printed nothing at all for its full timeout. Every run so far was
killed at exactly that line — which is why the export below had never once run,
and why it looked like the export was broken when it was simply never reached.

It now polls the grid's own **Total Counts** every `LOOKUP_POLL_SECONDS`, up to
`LOOKUP_TIMEOUT_MS`, printing progress every 15 s.

**One changed reading is not enough.** eCW blanks Total Counts to 0 the instant
Lookup is pressed. The first version accepted any figure that differed from the
pre-click one, so it saw `0 != 894` on the very first poll, logged
`Results in after 0s` and recorded a row count of **0** for a search that had
returned 894 — a wrong number in the log and in the message box, for a run that
otherwise succeeded. A figure now has to hold across `_STABLE_POLLS`
consecutive reads with `#routeLoader` hidden, and a `0` — the one value that is
both a legitimate answer and the transient — has to hold for twice as long.

### Exporting the report

Three things about that submenu:

- **CSV and EXCEL share an id.** Both are `#claimLookupBtn1002`. Clicking it
  with `.first` exports **CSV** — a real file downloads, the log looks clean,
  and the format is wrong. The format is matched on `ng-click` instead, which is
  unique per format (`mnuViewReport_Click('EXCEL');`).
- **"View Claims Report" is a `dropdown-submenu` with no `ng-click`.** It opens
  on hover, so `claim_report.py` hovers first and only clicks as a fallback — on a
  submenu a click can close the parent dropdown as easily as open the child.
- **"Billing" here is not the left-nav Billing.** `#claimLookupBtn17` is on the
  toolbar above the results grid; `pages/leftnav.py`'s Billing is the sidebar menu.

**The page is left scrolled.** Playwright scrolls an element into view before
clicking it, so reaching the toolbar's Billing menu scrolls the whole filter
form off the top. Nothing scrolled it back, so a successful run ended on a
screen showing six grid rows, an eCW spacer and a lot of white — reported as
"the portal looks broken" when the form was simply above the viewport.
`_restore_view()` closes the dropdown (Escape, then toggling
`#claimLookupBtn17` if Escape did not take), scrolls the form back, and takes
`claims_report_done`. Never fatal: the file is already saved by then.

The first version of it used `window.scrollTo(0, 0)` and **that did nothing** —
`claims_report_done` came back with the dropdown correctly closed and the page
still scrolled, which is what isolated the failing half. On this SPA the
document does not scroll; an inner container does, so scrolling the window is a
no-op on an element that was never offset. It now calls
`scroll_into_view_if_needed()` on `#claimStatusCodeId` and lets Playwright walk
up to whatever actually scrolls, then sweeps any element with `scrollTop > 0`
and **logs which ones they were** — so nobody has to rediscover that the window
is not the scroller.

**The report goes to `Input\`, not `Output\`.** It reads like output — the bot
downloaded it — but it is the **input to the charge-entry phase**, which walks
its claim ids one at a time. Calling it output would invite someone to treat it
as a finished deliverable and clear it out between runs, stranding the phase
that consumes it. Saved under a fixed name, `Claim Report_<MM-DD-YYYY>.xlsx`,
because the next phase has to find it without being told where it is — eCW's
own name is a millisecond timestamp (`CwReport_1788783056306.xlsx`), which is
useless for that.

The extension follows what eCW actually sent, not the format that was asked
for: its "EXCEL" export has been a real xlsx (a PK zip) and, on other builds,
an HTML table — and calling HTML `.xlsx` makes Excel refuse to open it.

A second run on the same day does **not** overwrite the first. The earlier copy
is moved to `Claim Report_<date> (2).xlsx` and kept: it is patient data, and
the next phase may already be partway through it. Same-day re-runs are the norm
here, not the exception.

### Searching one claim at a time

```
#claimLookupIpt10  ← claim id            ng-model="_InvId"  (fill(), not evaluate)
  → #btnclaimlookup
  → wait for that claim id to appear in the grid
  → message box: found / not found, Total Counts, seconds
```

`fill("")` before every id. The box keeps the previous claim's number, and
appending instead of replacing would search for `55151455509935` — zero rows,
with the screen looking almost right.

**The check is the claim id coming back in the grid, not the count.** Two
reasons, and the first is the one that matters:

- The Claim No box's meaning depends on a type selector beside it. eCW switches
  the date-type dropdown's `pointer-events` off while that reads
  `Rule Engine ID#` — the only observable sign that this box is not searching
  claim numbers at all. No selector for it has been captured, so rather than
  invent one, `search_claim` checks the *outcome*: if the id it typed comes back
  in the grid, the box meant what was assumed.
- A count is a poor signal. 541 rows → 1 is a change the settle-wait can see,
  but the next claim goes 1 → 1 and looks identical to a grid that never
  refreshed.

A claim that is not found does **not** stop the loop — the report was exported
minutes earlier and a claim can leave the queue in between. It is logged, a
screenshot is taken, and the run moves to the next id.

`CLAIM_PROCESS_LIMIT` bounds it, **default 1**. About 540 ids at one
server-side lookup each is a run measured in hours, which is not something to
discover by accident while testing a new step. `SHOW_EACH_CLAIM=0` when you
raise the limit, or it is one dialog per claim; the summary box and the log
appear either way. The Claim No box is cleared at the end, since a leftover id
would narrow the *next* whole-queue lookup to a single claim.

### The UB Claim form

```
click the claim number in the grid
  → #UBClaimDiv                    the modal-dialog, NOT the backdrop
  → a[ng-click*="cptTabUpdate('cptHcpcs')"]      the CPT/HCPCS tab
  → extract every field in one page.evaluate
  → #UBClaimProgressNoteBtn        Prog. Notes
      → the note, in an iframe       read by prisma-section
      → closeProgressNoteViwer()     eCW's own typo
  → Output\claims\<claim>.json
  → #UBClaimBtn17  Cancel          NEVER #UBClaimBtn18
```

**One rule governs this screen: never use an id with a number in it.** eCW
bakes a per-open timestamp into most of them:

```
<div id="UBClaimDiv1788790287089">        the backdrop
<select id="claimStatusSel1788790287089">  claim status
<table id="claimCPTCodesTable1788790287089">
```

That number is `Date.now()` at the moment the modal opened — different for
every claim, and different for the same claim opened twice. A selector holding
it works exactly once and then silently matches nothing, which is the worst
failure shape available: the run looks clean and the data comes back empty.
So everything is matched on an id with no digits (`#BillType`,
`#UBClaimProgressNoteBtn`), or on an `ng-model` / `ng-bind` / `ng-click`.
`tests\test_claim_form.py` asserts that no selector value in
`selectors\claim_form.py` contains a long run of digits, and its fixture uses
a *different* timestamp so a baked-in one fails the other assertions too.

Two more traps, both of which produce a clean-looking run with empty data:

- **Values live in inputs, not text.** The ICD code is
  `<input value="R69">` while its description beside it is plain text; the
  claim status is a `<select>`; every CPT column is an input. Reading
  `textContent` returns blanks for half the form. The table reader takes an
  input's `value`, then a select's selected label, then the cell's text.
- **Every grid is padded with blank rows.** eCW appends empty `ng-repeat`
  rows to hold a fixed height, plus a `*` entry row at the bottom. Counting
  those reports five CPT lines on a claim that has two.

**`#UBClaimBtn18` is OK and it saves** (`saveAllData(true)`). This phase only
reads, so the form is always closed with Cancel (`#UBClaimBtn17`) — closing
with OK would turn every read into an unintended write.

**Prog. Notes is discovered, not assumed.** Its destination has never been
captured, so the code looks at what actually changed — a new tab or a new
in-page modal — screenshots it, and captures its text wholesale. The button
also carries `ng-disabled="!FormData.EnableProgressNote"`, so it can be
perfectly visible and do nothing; that is checked and reported as a fact about
the claim rather than as an error.

The extracted claim is returned as one dict **and** written to
`Output\claims\<claim_no>.json` — a dict in a variable is gone when the run
ends, and this is the record of what a claim said at the moment the bot looked
at it. In `Output\`, not `Input\`: this is output, and `Input\` is what the run
consumes.

### The Progress Note

**eCW tags every section of the note with `prisma-section`**, which is the
best thing about this screen:

```
<tr prisma-section="Assessments">
<tr prisma-section="Procedure Codes">
<tr prisma-section="History of Present Illness">
```

So the note is not scraped as a wall of text — it comes back as a dict keyed
by eCW's own labels. Those labels drive its export/PRISMA feature, which makes
them considerably more stable than the table markup around them.

Two sections carry what charge entry actually needs, so they are parsed
individually rather than left as prose:

```
Assessments      1. Illness, unspecified - R69 (Primary)
                 → {code: "R69", primary: true}
Procedure Codes  H0023 Behavioral Health Outreach Service (TH). Modifiers: GT
                 → {code: "H0023", modifiers: ["GT"]}
```

Those are the **same ICD and CPT the UB Claim form shows**. Having both sides
as structure is what will later make "does the claim match the note?" a
comparison instead of a person reading two screens; for now both appear in the
same message box, side by side.

Four traps, every one of which produces a clean-looking run with nothing in it:

- **The note is a whole document.** It has its own `<html>`, so it arrives in
  an **iframe** (or a separate tab), never inline. Reading the hosting modal's
  `innerText` finds none of it — iframe content is not part of it — which is
  exactly what an earlier version of this code did. `progress_note.find()`
  walks the tabs and then every frame, and reports where it landed.
- **Close lives on the parent, not in the note.** And its handler is
  `closeProgressNoteViwer()` — **Viwer**, eCW's own typo. Spelling it
  correctly matches nothing. Closing matters: the viewer sits on top of the UB
  Claim form and blocks the Cancel button underneath, so a note left open
  would make the next claim open its form behind a stale one.
- **`prisma-section="Footer"` repeats four times.** Building the dict by
  overwriting keeps only the last — the facility block — and silently loses
  the signature and the sign-off status. Repeats are joined.
- **`#chartLockedId` is `display:none`.** `innerText` returns `""` for a
  hidden element, so an innerText-only reader reports the chart as *unlocked*
  — the opposite of the truth, on a field that decides whether a claim should
  be touched at all. Read with `textContent`.

`tests\test_progress_note.py` covers all four against a fixture that serves the
note in an iframe with Close on the host page, in a local headless chromium.

### One report in Input, always the current one

Reported symptom: run the bot a second time and it takes the **old** file's
claims. Three days of testing had left six reports sitting in `Input\`:

```
Claim Report_09-07-2026 (2).xlsx
Claim Report_09-07-2026 (3).xlsx
Claim Report_09-07-2026.xlsx
Claim Report_09-08-2026 (2).xlsx
Claim Report_09-08-2026.xlsx
Claim Report_09-09-2026.xlsx
```

Once more than one is there, "which report is current?" stops being obvious
— to a person opening the folder, and to any code that has to find it
without being told the name. And the wrong answer is silent: the run looks
clean while working a stale queue.

So the export now **archives every earlier report before saving the new one**.
`Input\` ends up holding exactly one file, which makes the answer structural
instead of a convention someone has to remember:

```
Input\
  Claim Report_09-09-2026.xlsx                   the current one
  Archive\
    Claim Report_09-07-2026__20260907_200644.xlsx
    Claim Report_09-08-2026__20260908_211100.xlsx
```

Moved, never deleted: patient data, and each one is the evidence of what an
earlier run actually worked from. The archived name carries the file's own
modification time, so two runs on the same day cannot collide.

`latest_report()` picks by **modification time, never by name**. Sorting names
is the trap here: `Claim Report_09-09-2026 (2).xlsx` sorts *before*
`Claim Report_09-09-2026.xlsx`, because a space (0x20) is lower than a dot
(0x2e) — so the alphabetically first file can be the older one. It also warns
if it finds more than one report, rather than quietly choosing; with archiving
on, more than one means something was left behind and the pick has become a
guess.

`ARCHIVE_OLD_INPUTS=0` leaves them all in `Input\` side by side.

### Reading the claim ids back

The last step of the export opens the file it just saved and pulls the claim ids
out of it, then shows them in a message box (`SHOW_CLAIM_IDS`) — count, first
15, last 5, and any duplicates or blank rows skipped. The **full list always
goes to the log**, which is searchable in a way a MessageBox is not, and is what
makes a run auditable: which claims were in scope that day, in the order they
would be worked.

It reads the **file on disk**, not the grid it came from. That is the point: the
file is what the charge-entry phase consumes, so this checks the thing that
actually matters — that it parses, that the claim-id column is where it is
expected, and that the count matches the lookup. A grid read would prove none of
that. It is never fatal; the report is already saved.

Two things came out of reading the real 391 KB file eCW produced on 2026-09-07,
neither of which could have been guessed:

- **The header is `CLAIMS#`,** not `CLAIM #` as the on-screen grid labels the
  same column. Taking the name off the screen would have found nothing.
- **There is no `sharedStrings.xml`.** Every cell is an inline `t="str"`, which
  makes the parser much simpler. The shared-string path is implemented anyway —
  "this build does not use it" is not "no build will".

The claim id is in column C with a blank column A before it, so nothing reads
positionally: cells are addressed by their column letter, because eCW can omit
an empty cell entirely and reading in order would shift every column after it.

`core\spreadsheet.py` handles xlsx, HTML-as-.xls and CSV, sniffed from the
file's own first bytes rather than its extension — the same reason the export
does not assume `.xlsx`. **No openpyxl:** an xlsx is a zip of XML and `zipfile`
plus `xml.etree` cover it, which matters on machines where every added package
is an approval. A legacy binary `.xls` (OLE2) is the one thing it refuses, and
it says so with the CSV alternative rather than returning nothing.

**Orphaned downloads.** Chromium writes a download to
`Output\_pw_downloads\<guid>` with no extension, and Playwright only gives it a
real name when `download.save_as()` runs. A run killed between those two points
leaves a complete, correct report on disk looking like nothing at all — a 552 KB
`.xlsx` sat there unnoticed while the export was being reported as broken.
`_rescue_orphans()` sweeps that folder into `Output\` at the start of the export
step and again if the download event is missed, sniffing the file header
(`PK` → `.xlsx`, `<` → HTML-table `.xls`) rather than assuming a format.

The step ends with a message box (`NOTIFY_AFTER_EXPORT`) reporting the row
count, file name, size and folder. It is raised **after** the file is on disk,
so nothing about it can lose the export — if nobody clicks OK it times out and
the run continues. It also fires on the no-download path, so a failed export is
not something you have to go looking in the log for.

`export_claims_report()` returns the saved `Path`, or `None` if eCW answered
with something other than a download — in which case it logs any new tab's URL
rather than raising, because the search has already succeeded and that URL is
what the next step needs. An empty file is called out too: it usually means the
filter matched nothing.

**Claim Status runs first, and it is the load-bearing step.** It decides whether
a claim-based date type can match anything (rows with no claim have no claim
date) and whether the Balance row is editable at all — its
`ng-change="enableInsurance()"` is what sets `enableBalanceFlag`. Get the order
wrong and both later steps misbehave for reasons that look unrelated.

It is a plain `<select>`, chosen **by option value with the label verified
afterwards**. Both halves earn their place: `108ME` is the only unambiguous
handle, since "Medcor Ready" sits one line from "Medcor Testing" (`113ME`) and a
loose text match could take either; and the label check catches eCW reassigning
a code, which would otherwise search the wrong queue and still look like a clean
run. `#claimStatusCodeId` is also `ng-show="ClaimStatusFlag"` — absent means the
mode selector beside it has moved off "Claim Status", so the list is hidden
rather than gone, and the error says which.

Options in the date-type dropdown are matched by **label, never by id**. eCW numbers them
`claimLookupLink2ngR0…ngR10` from the ngRepeat index, so `ngR1` means "the
second option" rather than "Claim Dt(s)" — reorder `cboDateType` upstream and an
id-based click silently selects the wrong filter, which is the worst kind of
bug here because the search still runs and still returns claims.

Two other properties of that control, both load-bearing:

- **Its text is the current value** (`ng-bind="selectedDateType"`), so the
  selection is verified by reading the trigger back, not by trusting the click.
  It also makes the step idempotent — already-correct is a no-op.
- **It can be switched off.** The `ng-style` sets `pointer-events: none` while
  the claim-number type is `Rule Engine ID#`. Playwright reports that as a
  generic timeout that reads like a dead selector, so it is checked for and
  named explicitly.

A missing label fails with the list of options eCW actually offered, so a rename
is a one-line fix rather than another login attempt spent looking.

The date inputs carry jQuery UI's `hasDatepicker`, so focusing one opens a
calendar that can cover the next control — Escape closes it before the next
click. Values go in with `fill()`, which fires the `input` event `ng-model`
listens for; setting `.value` through `evaluate` would update the DOM and leave
Angular's scope stale. Each one is read back to confirm eCW did not reformat it.

**The date type has to match the Claim Status.** Claim Status
"Encounters Without Claims" returns rows that have no claim — so they have no
claim date, no submitted date and no payment date. Pairing it with anything but
`Service Dt(s)` returns nothing. Measured 2026-09-02 on the same 06/01/2020 →
today range:

| Date type | Total Counts |
|---|---|
| `Service Dt(s)` | 936 (47 pages) |
| `Claim Dt(s)` | 0 |

It is a nasty failure because it does not look like one: the grid keeps
rendering the *previous* result set while Total Counts silently drops to 0. So
`run_lookup()` logs that number and `set_date_type()` warns on the contradictory
pair before spending a lookup.

`CLAIM_DATE_TYPE` is `Claim Dt(s)` — the process asks for it, and the
expectation is that a Claim Status step lands before this one and makes the pair
sensible. Until it does, the warning fires and the lookup returns 0. Set
`CLAIM_DATE_TYPE=Service Dt(s)` in `.env` to search the encounters-without-claims
queue instead, with no code change.

**The Balance row is usually disabled, and disabled does not mean inactive.**
All five controls carry `ng-disabled="!enableBalanceFlag"`, which eCW drives off
the Claim Status selection. With "Encounters Without Claims" chosen they are
greyed out *while still holding `> 0.0`* — and eCW builds its query from the
scope values, not from the inputs. So the filter narrows every search while
being impossible to click or type into.

`clear_balance()` therefore has two paths. Enabled: blank the controls the way a
user would. Disabled: `_force_clear_balance()` sets the five `ng-model`s
(`_condition1`, `_conditionVal1`, `andOrCondition`, `_condition2`,
`_conditionVal2`) to `''` inside `$apply`, which is the only thing that actually
clears the filter.

Inside `$apply` matters. Angular re-renders the inputs from the new model, so
the form the operator confirms is the form eCW will search on — and the cleared
value is then verified by reading the input back. Writing `el.value` directly
would do the reverse: change the display and leave the query untouched.

This leans on `angular.element(el).scope()`, available only because eCW ships
with debug info on — the `ng-scope` / `ng-binding` classes in the DOM are the
evidence. If that ever changes the helper reports why it failed instead of
claiming a clear that did not happen. `FORCE_CLEAR_BALANCE=0` opts out.

### Categorizing by payer

Right after the claim ids are read back, every row of the just-saved report
gets a `Group Name` (`Commercial`, `Medicaid`, `Medicare`, `Dentaquest`,
`Program`, ...) based on its `PAYER`, and the result is saved to
`Output\Medcor Ready_<MM-DD-YYYY>.xlsx`. The original `Input\` file is never
touched — `claim_file.py` and the charge-entry phase depend on its exact
column layout.

The mapping is `src\ecw_bot\payer_groups.json` — a flat, hand-editable
`{"exact PAYER string": "Group Name"}` table. **Exact match only, no fuzzy
matching**: eCW's own PAYER values already differ by case and spacing for
what is really the same carrier (`United Healthcare` / `United Health Care`
/ `United healthcare` are three separate real values), so a PAYER string
with no entry gets a **blank** Group Name rather than a guess. Unmapped rows
are logged and, unless `NOTIFY_UNMAPPED_PAYERS=0`, shown in a message box —
adding one is a one-line edit to the JSON file, no code change.

Never fatal: a missing PAYER column, an unreadable report, or a write
failure is logged and the run continues — the export already succeeded by
the time this step runs.

## Known gaps

- **The 6-digit code path is still untested.** The whole login has now run end
  to end (2026-09-02 16:16 — mailbox poll, approval link, "Yes, it's me",
  confirmed, landed) but only via the *link* branch, which is what this account
  actually gets. `Challenge("code", …)` and `#OTPCode` have never fired.
- **eCW's push mail is delayed by minutes, and that looks like a failure.**
  Proven by delivery order on 2026-09-02: the message eCW stamped `11:19:10`
  arrived in the mailbox *after* the one stamped `11:26:10` — about seven
  minutes late, and it has been seen taking ten. An earlier diagnosis here
  claimed eCW simply wasn't sending; that was wrong, and the "Resend OTP" work
  built on it initially made things worse by shortening the first poll to 60s.
  The schedule is now **10 min → Resend OTP → 10 min**: the full observed delay
  is waited out before resending, because a second mail only joins the same slow
  queue while the first is still in flight. The poll logs the lag from
  IMAP's `INTERNALDATE`, so this stays visible instead of being guessed at.
  A challenge past `OTP_LINK_AGE_WARN_SECONDS` is **warned about and still
  used** — with a ten-minute mail delay against a ten-minute JWT a link can
  arrive nearly expired, and refusing to click it is a guaranteed failure while
  clicking it costs nothing already unspent.
- **Twenty minutes of waiting may outlive eCW's own login session.** If the page
  falls back to the username box, `_challenge_with_resend` stops with that as
  the stated reason rather than resending into a dead session — but it only
  checks *between* windows, so a session that dies at minute 4 is not noticed
  until minute 10. Whether eCW times out that fast is still unknown. The fix if
  it does is a *lower* `OTP_RESEND_WAIT_SECONDS`, not a higher one.
- **CAPTCHA needs a human.** Two attempts, then incognito, then one more —
  all via message box. Unattended scheduling is not possible until this is
  either solved automatically or confirmed not to appear for this account.

## The proxy / webmail deadlock

Doing this login by hand is a deadlock, and it is worth knowing why the bot is
not stuck in it. Measured 2026-09-01:

| | without proxy | through proxy |
|---|---|---|
| eCW portal | HTTP 400 | **HTTP 200** |
| webmail (cpanel HTTP) | **HTTP 200** | 403, refused by the proxy |
| mailbox (IMAP :993) | **connects** | n/a — never proxied |

So a human has to flip the system proxy on to reach eCW, off to read the
approval mail, and on again to confirm the link.

The bot avoids it by never opening the webmail *website*. The proxy is set
**per-browser** at `chromium.launch(proxy=…)`, not system-wide, and the mail is
read with `imaplib` — a raw TCP socket that ignores WinINET and the
`HTTP(S)_PROXY` environment variables because it never speaks HTTP. The browser
sits behind the proxy and Python talks to the mailbox directly, at the same
time.

**Leave the Windows system proxy OFF** (`ProxyEnable = 0`). Turning it on is
what breaks the mailbox side; `tools\check_net.py` fails loudly if it is on.

## Rate limiting

eCW restricts an account that is retried too hard. On 2026-09-01 seven attempts
inside forty-five minutes produced:

> Access to the application has been restricted.

The nasty part is what happens next: the portal goes on rendering the ordinary
"eClinicalWorks sent a notification to your email" screen, but sends no mail at
all. A run then polls an empty mailbox for two minutes and blames the mailbox.

Three things guard against repeating that:

* `auth/restricted.py` is checked at every step at every step and raises
  `Restricted` immediately — exit code **4**, distinct from an ordinary
  failure.
* `core/guard.py` records each attempt in `Logsttempts.json` and refuses one that
  is too frequent (`LOGIN_MAX_PER_HOUR`, default 4) or that follows the
  restriction page — exit code **3**. That lockout is 5 minutes
  (`RESTRICTED_LOCKOUT`), kept short because the usual cause is a lost proxy,
  which waiting does not fix.
* A **plain cooldown is waited out, not refused.** `wait_if_cooling()` sleeps
  the remainder of `LOGIN_COOLDOWN_SECONDS` (up to
  `COOLDOWN_MAX_WAIT_SECONDS`) and then proceeds. It used to exit 3, and five
  of six consecutive runs died that way — the last of them with **3 seconds**
  left. Re-running by hand until the clock clears is precisely the pattern the
  cooldown exists to discourage, so sleeping honours it just as well and
  finishes the job. The burst limit and the restriction lockout are never
  waited out: those mean stop, not "not yet".
* Every message box times out (`MSGBOX_TIMEOUT_SECONDS`), so an unattended run
  can no longer sit on a modal holding a browser open.

`ECW_FORCE=1` overrides the guard when you genuinely need to retry now.

## Exit codes

| Code | Meaning |
|---|---|
| 0 | logged in, and landed on the claims screen |
| 1 | login failed |
| 2 | config incomplete |
| 3 | refused by the attempt guard — wait, or `ECW_FORCE=1` |
| 4 | eCW has restricted the account — stop retrying |
| 5 | logged in, but navigation to Claims failed — kept separate because the login is the expensive part and should not read as broken |