# Salud eCW Charge Entry Bot Single-client eCW automation for Salud. Login is done; the charge-entry phases get built on top of the session it hands back. Structurally a trimmed rework of the NAHMS Combined Reports bot — same portal, same eCW V12 login markup, so the proven login handling is reused. What is different: one client instead of a spreadsheet loop, credentials in `.env` instead of `Input\input.xlsx`, and the second factor read from the mailbox because this account has no authenticator secret. The second factor is **not** a 6-digit code. eCW mails this account a "Push Notification" containing an approval link (`…/login/authenticate/openLoginConfirmation?token=`, valid 10 minutes) and the login page waits until it is confirmed. The bot reads the mail over IMAP, opens the link in a side tab, clicks "Yes, it's me", and closes it. `auth/mailbox.py` still handles a mailed 6-digit code and a TOTP secret, so a different account needs no code change. ## Install ```bash pip install -r requirements.txt playwright install chromium ``` ## Configure `.env` at the project root holds every credential and is gitignored. `.env.example` is the template. ## Run ```bash python run.py ``` `run.py` is the only supported entry point. It puts `src\` on the import path and calls the flow — which is why it exists: the old layout worked *only* because Python happens to put a script's own directory on `sys.path`, so `python src\main.py` resolved `import config` and nothing else did. `python -m ecw_bot` works too, with `PYTHONPATH=src`. Network preflight — run this first if anything about login or 2FA misbehaves: ```bash python tools\check_net.py ``` Mailbox probe — run it if the second factor starts failing, or when pointing the bot at a different account: ```bash python tools\check_imap.py ``` It tries each plausible IMAP host, logs in, and prints recent subjects. As of 2026-09-01 the configured `cpanel1.methodhosting.net` passes. Tests. No pytest — the hand-rolled `.env` parser exists for the same reason, that every added dependency on these machines is one more approval: ```bash python tests\test_settings.py ``` ```bash python tests\test_guard.py ``` ```bash python tests\test_dead_end.py ``` ```bash python tests\test_claim_file.py ``` ```bash python tests\test_claim_form.py ``` ```bash python tests\test_progress_note.py ``` ```bash python tests\test_input_archive.py ``` `test_claim_form.py` launches a **local headless chromium** against `tests\fixtures\ub_claim.html`. It never touches eCW — no proxy, no login, no 2FA mail, no attempt against the guard — so it costs nothing and can be run as often as you like. That matters: the only other way to exercise that code is a three-minute login and one of four hourly attempts. ## Layout ``` run.py the entry point src\ecw_bot\ settings.py .env values, and nothing else claim_file.py reads the saved report; finds CLAIMS# selectors\ every CSS selector, and nothing else login.py shell.py claim_lookup.py claim_form.py core\ no eCW knowledge — reusable as-is browser.py chromium.launch(proxy=…) — see below page_utils.py shot / visible / text_of / is_disabled / goto spreadsheet.py xlsx / HTML / CSV reader, stdlib only msgbox.py operator prompts; every one times out guard.py attempt cooldown and burst limit runlog.py tees the run to Logs\run_.log auth\ login.py the login itself captcha.py two tries, incognito, one last try twofactor.py the mailed approval link mailbox.py imaplib reader over a raw socket restricted.py eCW's refusal page + the exception pages\ one module per eCW screen leftnav.py left panel → Billing → Claims claim_filters.py status, date type, date range, Balance claim_results.py Lookup, and waiting for the grid honestly claim_report.py Billing → View Claims Report → EXCEL claim_form.py the UB Claim modal: open, read, close progress_note.py the note: find it, read prisma-sections flows\ export_medcor_ready.py the step order, and the exit codes tools\ diagnostics you run by hand check_net.py check_imap.py tests\ test_settings.py test_guard.py test_dead_end.py test_claim_file.py test_claim_form.py test_progress_note.py test_input_archive.py fixtures\ub_claim.html fixtures\progress_note.html fixtures\progress_note_host.html Logs\ Screenshots\ Output\ all gitignored Input\ Claim Report_.xlsx the ONE current report Archive\ every earlier one ``` Nothing imports a layer above it, so the graph is acyclic and any module can be read without chasing a cycle. `tests\test_settings.py` asserts that, along with `settings.ROOT` resolving to the project root and no selector leaking back into `settings`. **Why it is split this way.** It started as eleven flat modules in `src\`, 3,253 lines, with `claims.py` alone at 952 — nearly a third of the project in one file — and `config.py` mixing 43 DOM selectors in with the date range and the proxy. These things change for entirely different reasons: a selector changes when eCW ships a new build, a setting when the process changes, a flow when the work changes. In one file, editing a date range and editing a selector were edits to the same place. ## Login flow ``` goto ECW_URL (retried — the proxy kills the first goto intermittently) → #doctorID → #nextStep → #err check ("User is Inactive" fails fast) → security-image page → message box, operator continues → #passwordField → #Login → CAPTCHA → 2 manual attempts → incognito relaunch → final manual attempt → 2FA → mailbox poll, 10min → Resend OTP → 10min (patient on purpose: the mail can take 10 minutes) link → open in side tab → "Yes, it's me" → page proceeds code → #OTPCode → #Submit → #jellybean-panelLink4 (home confirmed) → goto ECW_LANDING_URL (eCW drops the fragment during the login redirect) ``` Screenshots of each notable step land in `Screenshots\\`. ## Navigation ``` #jellybean-panelLink4 (left panel — only if it is not already open) → #Billing_menu > a (waited for: injected late) → #CLAIMS_BILLING span.padl10 → claimLookup.jsp (URL check + #routeLoader hidden + screenshot) ``` `pages/leftnav.py` is written defensively for three reasons, all of them properties of eCW's own markup rather than caution for its own sake: - **The hamburger is a toggle, not an "open".** `launchLeftPanel()` flips a `visible` class, so clicking it on an open panel closes it. State is checked before every click. - **`#jellybean-panelLink4` matches two elements.** eCW renders `#JellyBeanCountCntrl` twice, and Playwright's strict mode refuses a plain `click()` on that. Every locator is `.first`. - **The menu does not exist when login is confirmed.** `getleftnavhtml()` injects it on `DOMContentLoaded`, so `#Billing_menu` is waited for. A timeout there means the left nav never rendered — a different bug from a moved item, and the message says so. Claims is clicked on its label span, not the anchor: the anchor also holds `#favorite_16`, whose own `ng-click` stars the item instead of navigating. If any of that fails the run does not stop — it navigates straight to `CLAIMS_HASH`, the same hash the menu item's `openURL()` would have used, and logs `[NAV] … falling back to the hash route`. Worth fixing the selector when you see that line, rather than leaving the bot on the fallback permanently. No selector on the claims screen itself is asserted — `claim_filters.py` waits for the form's own controls, so nav only has to get the route right. ## Claim lookup ``` #claimStatusCodeId ← 108ME "Medcor Ready" FIRST — everything hangs off it span.toggledropdown[ng-bind='selectedDateType'] (reads "Service Dt(s)") → #claimLookupUl2 li a span:text-is("Claim Dt(s)") → toggle reads back "Claim Dt(s)" (the confirmation) #fromdate ← CLAIM_FROM_DATE (06/01/2020) Escape closes the datepicker #todate ← today, computed at run time Balance row cleared — all five controls, when eCW has them enabled → already empty? left untouched, nothing to clear → filters logged + claims_before_lookup.png (no box: CONFIRM_BEFORE_LOOKUP=0) #btnclaimlookup (NOT #btnclaimlookupClear) → poll "Total Counts" until it changes (NOT networkidle) #claimLookupBtn17 "Billing" toolbar, not the left nav → #claimLookupBtn71 "View Claims Report" hover, submenu → button[ng-click*="EXCEL"] NOT #claimLookupBtn1002 → archive every earlier report to Input\Archive\ → Input\Claim Report_.xlsx the ONE current report → message box: rows, file, size, folder read the file back → the CLAIMS# column → message box: count, first 15, last 5 (full list to the log) ``` ### Waiting for the results grid `run_lookup` used to call `wait_for_load_state("networkidle")` after pressing Lookup. **That never returns on this page.** eCW's Angular shell polls in the background, so the network is never quiet for the 500 ms Playwright wants, and the call printed nothing at all for its full timeout. Every run so far was killed at exactly that line — which is why the export below had never once run, and why it looked like the export was broken when it was simply never reached. It now polls the grid's own **Total Counts** every `LOOKUP_POLL_SECONDS`, up to `LOOKUP_TIMEOUT_MS`, printing progress every 15 s. **One changed reading is not enough.** eCW blanks Total Counts to 0 the instant Lookup is pressed. The first version accepted any figure that differed from the pre-click one, so it saw `0 != 894` on the very first poll, logged `Results in after 0s` and recorded a row count of **0** for a search that had returned 894 — a wrong number in the log and in the message box, for a run that otherwise succeeded. A figure now has to hold across `_STABLE_POLLS` consecutive reads with `#routeLoader` hidden, and a `0` — the one value that is both a legitimate answer and the transient — has to hold for twice as long. ### Exporting the report Three things about that submenu: - **CSV and EXCEL share an id.** Both are `#claimLookupBtn1002`. Clicking it with `.first` exports **CSV** — a real file downloads, the log looks clean, and the format is wrong. The format is matched on `ng-click` instead, which is unique per format (`mnuViewReport_Click('EXCEL');`). - **"View Claims Report" is a `dropdown-submenu` with no `ng-click`.** It opens on hover, so `claim_report.py` hovers first and only clicks as a fallback — on a submenu a click can close the parent dropdown as easily as open the child. - **"Billing" here is not the left-nav Billing.** `#claimLookupBtn17` is on the toolbar above the results grid; `pages/leftnav.py`'s Billing is the sidebar menu. **The page is left scrolled.** Playwright scrolls an element into view before clicking it, so reaching the toolbar's Billing menu scrolls the whole filter form off the top. Nothing scrolled it back, so a successful run ended on a screen showing six grid rows, an eCW spacer and a lot of white — reported as "the portal looks broken" when the form was simply above the viewport. `_restore_view()` closes the dropdown (Escape, then toggling `#claimLookupBtn17` if Escape did not take), scrolls the form back, and takes `claims_report_done`. Never fatal: the file is already saved by then. The first version of it used `window.scrollTo(0, 0)` and **that did nothing** — `claims_report_done` came back with the dropdown correctly closed and the page still scrolled, which is what isolated the failing half. On this SPA the document does not scroll; an inner container does, so scrolling the window is a no-op on an element that was never offset. It now calls `scroll_into_view_if_needed()` on `#claimStatusCodeId` and lets Playwright walk up to whatever actually scrolls, then sweeps any element with `scrollTop > 0` and **logs which ones they were** — so nobody has to rediscover that the window is not the scroller. **The report goes to `Input\`, not `Output\`.** It reads like output — the bot downloaded it — but it is the **input to the charge-entry phase**, which walks its claim ids one at a time. Calling it output would invite someone to treat it as a finished deliverable and clear it out between runs, stranding the phase that consumes it. Saved under a fixed name, `Claim Report_.xlsx`, because the next phase has to find it without being told where it is — eCW's own name is a millisecond timestamp (`CwReport_1788783056306.xlsx`), which is useless for that. The extension follows what eCW actually sent, not the format that was asked for: its "EXCEL" export has been a real xlsx (a PK zip) and, on other builds, an HTML table — and calling HTML `.xlsx` makes Excel refuse to open it. A second run on the same day does **not** overwrite the first. The earlier copy is moved to `Claim Report_ (2).xlsx` and kept: it is patient data, and the next phase may already be partway through it. Same-day re-runs are the norm here, not the exception. ### Searching one claim at a time ``` #claimLookupIpt10 ← claim id ng-model="_InvId" (fill(), not evaluate) → #btnclaimlookup → wait for that claim id to appear in the grid → message box: found / not found, Total Counts, seconds ``` `fill("")` before every id. The box keeps the previous claim's number, and appending instead of replacing would search for `55151455509935` — zero rows, with the screen looking almost right. **The check is the claim id coming back in the grid, not the count.** Two reasons, and the first is the one that matters: - The Claim No box's meaning depends on a type selector beside it. eCW switches the date-type dropdown's `pointer-events` off while that reads `Rule Engine ID#` — the only observable sign that this box is not searching claim numbers at all. No selector for it has been captured, so rather than invent one, `search_claim` checks the *outcome*: if the id it typed comes back in the grid, the box meant what was assumed. - A count is a poor signal. 541 rows → 1 is a change the settle-wait can see, but the next claim goes 1 → 1 and looks identical to a grid that never refreshed. A claim that is not found does **not** stop the loop — the report was exported minutes earlier and a claim can leave the queue in between. It is logged, a screenshot is taken, and the run moves to the next id. `CLAIM_PROCESS_LIMIT` bounds it, **default 1**. About 540 ids at one server-side lookup each is a run measured in hours, which is not something to discover by accident while testing a new step. `SHOW_EACH_CLAIM=0` when you raise the limit, or it is one dialog per claim; the summary box and the log appear either way. The Claim No box is cleared at the end, since a leftover id would narrow the *next* whole-queue lookup to a single claim. ### The UB Claim form ``` click the claim number in the grid → #UBClaimDiv the modal-dialog, NOT the backdrop → a[ng-click*="cptTabUpdate('cptHcpcs')"] the CPT/HCPCS tab → extract every field in one page.evaluate → #UBClaimProgressNoteBtn Prog. Notes → the note, in an iframe read by prisma-section → closeProgressNoteViwer() eCW's own typo → Output\claims\.json → #UBClaimBtn17 Cancel NEVER #UBClaimBtn18 ``` **One rule governs this screen: never use an id with a number in it.** eCW bakes a per-open timestamp into most of them: ```
the backdrop ` while its description beside it is plain text; the claim status is a ``, chosen **by option value with the label verified afterwards**. Both halves earn their place: `108ME` is the only unambiguous handle, since "Medcor Ready" sits one line from "Medcor Testing" (`113ME`) and a loose text match could take either; and the label check catches eCW reassigning a code, which would otherwise search the wrong queue and still look like a clean run. `#claimStatusCodeId` is also `ng-show="ClaimStatusFlag"` — absent means the mode selector beside it has moved off "Claim Status", so the list is hidden rather than gone, and the error says which. Options in the date-type dropdown are matched by **label, never by id**. eCW numbers them `claimLookupLink2ngR0…ngR10` from the ngRepeat index, so `ngR1` means "the second option" rather than "Claim Dt(s)" — reorder `cboDateType` upstream and an id-based click silently selects the wrong filter, which is the worst kind of bug here because the search still runs and still returns claims. Two other properties of that control, both load-bearing: - **Its text is the current value** (`ng-bind="selectedDateType"`), so the selection is verified by reading the trigger back, not by trusting the click. It also makes the step idempotent — already-correct is a no-op. - **It can be switched off.** The `ng-style` sets `pointer-events: none` while the claim-number type is `Rule Engine ID#`. Playwright reports that as a generic timeout that reads like a dead selector, so it is checked for and named explicitly. A missing label fails with the list of options eCW actually offered, so a rename is a one-line fix rather than another login attempt spent looking. The date inputs carry jQuery UI's `hasDatepicker`, so focusing one opens a calendar that can cover the next control — Escape closes it before the next click. Values go in with `fill()`, which fires the `input` event `ng-model` listens for; setting `.value` through `evaluate` would update the DOM and leave Angular's scope stale. Each one is read back to confirm eCW did not reformat it. **The date type has to match the Claim Status.** Claim Status "Encounters Without Claims" returns rows that have no claim — so they have no claim date, no submitted date and no payment date. Pairing it with anything but `Service Dt(s)` returns nothing. Measured 2026-09-02 on the same 06/01/2020 → today range: | Date type | Total Counts | |---|---| | `Service Dt(s)` | 936 (47 pages) | | `Claim Dt(s)` | 0 | It is a nasty failure because it does not look like one: the grid keeps rendering the *previous* result set while Total Counts silently drops to 0. So `run_lookup()` logs that number and `set_date_type()` warns on the contradictory pair before spending a lookup. `CLAIM_DATE_TYPE` is `Claim Dt(s)` — the process asks for it, and the expectation is that a Claim Status step lands before this one and makes the pair sensible. Until it does, the warning fires and the lookup returns 0. Set `CLAIM_DATE_TYPE=Service Dt(s)` in `.env` to search the encounters-without-claims queue instead, with no code change. **The Balance row is usually disabled, and disabled does not mean inactive.** All five controls carry `ng-disabled="!enableBalanceFlag"`, which eCW drives off the Claim Status selection. With "Encounters Without Claims" chosen they are greyed out *while still holding `> 0.0`* — and eCW builds its query from the scope values, not from the inputs. So the filter narrows every search while being impossible to click or type into. `clear_balance()` therefore has two paths. Enabled: blank the controls the way a user would. Disabled: `_force_clear_balance()` sets the five `ng-model`s (`_condition1`, `_conditionVal1`, `andOrCondition`, `_condition2`, `_conditionVal2`) to `''` inside `$apply`, which is the only thing that actually clears the filter. Inside `$apply` matters. Angular re-renders the inputs from the new model, so the form the operator confirms is the form eCW will search on — and the cleared value is then verified by reading the input back. Writing `el.value` directly would do the reverse: change the display and leave the query untouched. This leans on `angular.element(el).scope()`, available only because eCW ships with debug info on — the `ng-scope` / `ng-binding` classes in the DOM are the evidence. If that ever changes the helper reports why it failed instead of claiming a clear that did not happen. `FORCE_CLEAR_BALANCE=0` opts out. ### Categorizing by payer Right after the claim ids are read back, every row of the just-saved report gets a `Group Name` (`Commercial`, `Medicaid`, `Medicare`, `Dentaquest`, `Program`, ...) based on its `PAYER`, and the result is saved to `Output\Medcor Ready_.xlsx`. The original `Input\` file is never touched — `claim_file.py` and the charge-entry phase depend on its exact column layout. The mapping is `src\ecw_bot\payer_groups.json` — a flat, hand-editable `{"exact PAYER string": "Group Name"}` table. **Exact match only, no fuzzy matching**: eCW's own PAYER values already differ by case and spacing for what is really the same carrier (`United Healthcare` / `United Health Care` / `United healthcare` are three separate real values), so a PAYER string with no entry gets a **blank** Group Name rather than a guess. Unmapped rows are logged and, unless `NOTIFY_UNMAPPED_PAYERS=0`, shown in a message box — adding one is a one-line edit to the JSON file, no code change. Never fatal: a missing PAYER column, an unreadable report, or a write failure is logged and the run continues — the export already succeeded by the time this step runs. ## Known gaps - **The 6-digit code path is still untested.** The whole login has now run end to end (2026-09-02 16:16 — mailbox poll, approval link, "Yes, it's me", confirmed, landed) but only via the *link* branch, which is what this account actually gets. `Challenge("code", …)` and `#OTPCode` have never fired. - **eCW's push mail is delayed by minutes, and that looks like a failure.** Proven by delivery order on 2026-09-02: the message eCW stamped `11:19:10` arrived in the mailbox *after* the one stamped `11:26:10` — about seven minutes late, and it has been seen taking ten. An earlier diagnosis here claimed eCW simply wasn't sending; that was wrong, and the "Resend OTP" work built on it initially made things worse by shortening the first poll to 60s. The schedule is now **10 min → Resend OTP → 10 min**: the full observed delay is waited out before resending, because a second mail only joins the same slow queue while the first is still in flight. The poll logs the lag from IMAP's `INTERNALDATE`, so this stays visible instead of being guessed at. A challenge past `OTP_LINK_AGE_WARN_SECONDS` is **warned about and still used** — with a ten-minute mail delay against a ten-minute JWT a link can arrive nearly expired, and refusing to click it is a guaranteed failure while clicking it costs nothing already unspent. - **Twenty minutes of waiting may outlive eCW's own login session.** If the page falls back to the username box, `_challenge_with_resend` stops with that as the stated reason rather than resending into a dead session — but it only checks *between* windows, so a session that dies at minute 4 is not noticed until minute 10. Whether eCW times out that fast is still unknown. The fix if it does is a *lower* `OTP_RESEND_WAIT_SECONDS`, not a higher one. - **CAPTCHA needs a human.** Two attempts, then incognito, then one more — all via message box. Unattended scheduling is not possible until this is either solved automatically or confirmed not to appear for this account. ## The proxy / webmail deadlock Doing this login by hand is a deadlock, and it is worth knowing why the bot is not stuck in it. Measured 2026-09-01: | | without proxy | through proxy | |---|---|---| | eCW portal | HTTP 400 | **HTTP 200** | | webmail (cpanel HTTP) | **HTTP 200** | 403, refused by the proxy | | mailbox (IMAP :993) | **connects** | n/a — never proxied | So a human has to flip the system proxy on to reach eCW, off to read the approval mail, and on again to confirm the link. The bot avoids it by never opening the webmail *website*. The proxy is set **per-browser** at `chromium.launch(proxy=…)`, not system-wide, and the mail is read with `imaplib` — a raw TCP socket that ignores WinINET and the `HTTP(S)_PROXY` environment variables because it never speaks HTTP. The browser sits behind the proxy and Python talks to the mailbox directly, at the same time. **Leave the Windows system proxy OFF** (`ProxyEnable = 0`). Turning it on is what breaks the mailbox side; `tools\check_net.py` fails loudly if it is on. ## Rate limiting eCW restricts an account that is retried too hard. On 2026-09-01 seven attempts inside forty-five minutes produced: > Access to the application has been restricted. The nasty part is what happens next: the portal goes on rendering the ordinary "eClinicalWorks sent a notification to your email" screen, but sends no mail at all. A run then polls an empty mailbox for two minutes and blames the mailbox. Three things guard against repeating that: * `auth/restricted.py` is checked at every step at every step and raises `Restricted` immediately — exit code **4**, distinct from an ordinary failure. * `core/guard.py` records each attempt in `Logsttempts.json` and refuses one that is too frequent (`LOGIN_MAX_PER_HOUR`, default 4) or that follows the restriction page — exit code **3**. That lockout is 5 minutes (`RESTRICTED_LOCKOUT`), kept short because the usual cause is a lost proxy, which waiting does not fix. * A **plain cooldown is waited out, not refused.** `wait_if_cooling()` sleeps the remainder of `LOGIN_COOLDOWN_SECONDS` (up to `COOLDOWN_MAX_WAIT_SECONDS`) and then proceeds. It used to exit 3, and five of six consecutive runs died that way — the last of them with **3 seconds** left. Re-running by hand until the clock clears is precisely the pattern the cooldown exists to discourage, so sleeping honours it just as well and finishes the job. The burst limit and the restriction lockout are never waited out: those mean stop, not "not yet". * Every message box times out (`MSGBOX_TIMEOUT_SECONDS`), so an unattended run can no longer sit on a modal holding a browser open. `ECW_FORCE=1` overrides the guard when you genuinely need to retry now. ## Exit codes | Code | Meaning | |---|---| | 0 | logged in, and landed on the claims screen | | 1 | login failed | | 2 | config incomplete | | 3 | refused by the attempt guard — wait, or `ECW_FORCE=1` | | 4 | eCW has restricted the account — stop retrying | | 5 | logged in, but navigation to Claims failed — kept separate because the login is the expensive part and should not read as broken |