Rpa_Skills/tests/evals/results/handoff/h-web-portal-screens-r5.md

# Payer portal downloader - ops handover

Version: not versioned (the project has no version string, see section 13) · Written 2026-10-23 · Escalation: raise a ticket in the **RPA Support** queue

## 1. What the bot does

Every day at 07:00 (Windows Task Scheduler) the bot opens Chrome, signs in to the payer web portal and downloads the remittance report (a CSV file) for each provider in the input Excel. The input is `data/input/providers.xlsx` with two sheets: **Login** (portal username and password) and **Providers** (one provider per row). The output is one Excel report per day, `data/output/Download_Report_<yyyymmdd>.xlsx`, with a Status and a Remarks column per provider, plus the downloaded CSV files in `data/downloads/` and screenshots of pages that failed to load in `data/screens/`. The bot only reads and downloads; it never submits anything in the portal. It drives one app: **Google Chrome** (a Chrome window that says "Chrome is being controlled by automated test software").

## 2. How it works

```
Windows Task Scheduler (07:00 daily)
   └─ starts → Bot (no window of its own) ──drives──> Chrome → payer web portal
                    │                                  (do not touch this Chrome window)
                    ├─ reads  → data/input/providers.xlsx
                    ├─ saves  → data/downloads/  (one CSV per provider)
                    ├─ saves  → data/screens/<NPI>.png  (only when the report page did not load)
                    ├─ writes → data/output/Download_Report_<yyyymmdd>.xlsx  (at the very end)
                    └─ writes → logs/bot.log  (developer log, ops has no access)
```

- **Task Scheduler** starts the bot every morning; nobody has to click anything.
- **The bot** has no screen of its own. You only see the Chrome window it drives.
- **Chrome** is opened and closed by the bot. Closing it yourself ends the run.
- **The report** is written once, when the run ends. During the run it does not exist yet.

**One run, step by step:**
1. Read the Login and Providers sheets.
2. Open Chrome and sign in to the portal. If sign-in fails, every provider gets **Login Failed** and the run ends.
3. For each provider, in input order:
   1. Provider marked `N` in the Active column → **Skipped - inactive provider**.
   2. Open that provider's remittance page. If the portal sent the bot back to the sign-in page, sign in again and retry (up to 2 re-sign-ins).
   3. Portal shows a maintenance banner → **Downloaded** with remark "portal maintenance - no file" (no file was downloaded, see E07).
   4. Wait up to 30 s for the report grid. Not there → screenshot + **Error**.
   5. Grid says no results → **No Data**.
   6. Click Export CSV and wait up to 60 s for a new CSV in `data/downloads/` → **Downloaded**, or **Error** if none appears.
4. Close Chrome and save the day's report.

## 3. Before a run

- [ ] Google Chrome is installed on the bot machine and the machine has internet access.
- [ ] `config.yaml` is in the bot folder next to the bot program. Without it the bot stops at once and **no report is written**.
- [ ] These folders exist inside the bot folder: `logs`, `data/output`, `data/downloads`, `data/screens`. The bot does not create them. A missing `logs` or `data/output` folder means no report; a missing `data/screens` folder means screenshots are silently not saved.
- [ ] The Task Scheduler task for 07:00 is enabled.
- [ ] Input file `data/input/providers.xlsx`:
  - **Login** sheet: username in cell A2, password in cell B2. When the portal password changes, update B2 the same day, or every provider shows Login Failed.
  - **Providers** sheet, from row 2: column A = NPI, column B = provider name, column C = Active.
  - **Every provider row needs an NPI in column A.** A row with an empty column A is skipped silently and does not appear in the report.
    Bad: NPI blank, name filled. Good: NPI `1xxxxxxxxx`, name filled.
  - **To skip a provider, put a capital `N` in column C.** Anything else (blank, `Y`, lowercase `n`, `No`) means "download".
    Bad: `n` or `No` (provider is still downloaded). Good: `N`.
- [ ] If you need to re-run the same day, **first copy today's report** out of `data/output`. A second run on the same date overwrites it.

## 4. During a run

- Don't click, type in, or close the Chrome window marked "Chrome is being controlled by automated test software". Closing it ends the run; providers not yet reached get no row in the report.
- Don't open, move or delete files in `data/downloads/` while the bot runs. The bot decides a download worked by spotting a *new* CSV there; extra files confuse it.
- Don't keep `data/output/Download_Report_<today>.xlsx` open in Excel at the end of a run. (Excel locks the file; the bot may fail to save the report.)
- **To stop the bot:** Task Scheduler → the bot's task → **End**. No report is written for that run; re-run the whole input later. Closing Chrome is not a clean stop.
- **A run looks stuck** when the Chrome window stays on the same page for more than 10 minutes. The bot's own waits are at most 30 s (page) and 60 s (download) per provider, plus up to 2 re-sign-ins. **The bot has no watchdog** and never gives up on a hung page by itself: after 10 minutes on one page, end the task and raise a ticket.

## 5. Where to look when something goes wrong

| Question | Where to look |
|---|---|
| Did the run finish? | `data/output/` has `Download_Report_<today's date>.xlsx`, and it has one row per provider in the input (minus rows with blank NPI). Fewer rows = the run stopped early (E03). No file = the run never finished (E02). |
| Which providers failed? | Today's report → Status column: red cells (**Error**) and **Login Failed** (not coloured, so filter by text). Also check green rows with remark "portal maintenance - no file" (E07). |
| Why did one provider fail? | Today's report → Remarks column, then the catalogue below. For "Report grid did not load", open `data/screens/<NPI>.png` to see what the portal showed. |
| Was the file really downloaded? | `data/downloads/` → a CSV with today's date/time. The file keeps the portal's own name, not the provider's name. |
| Did the bot already retry? | Only for sign-in timeouts: remark **"Session expired"** means it signed in again twice and gave up. No other error is retried. |
| Which version is running? | Not visible anywhere (section 10). |

## 6. Statuses

The Status column of today's report. Every value the bot writes is listed.

| Status | Remark | Colour | Meaning | What ops does |
|---|---|---|---|---|
| Downloaded | Report saved | green | A new CSV arrived in `data/downloads/` | nothing |
| Downloaded | portal maintenance - no file | green | **Needs follow-up. Nothing was downloaded**: the portal showed a maintenance banner | Re-run this provider after the maintenance (E07) |
| No Data | No remittances in the date range | green | The portal has no remittances for this provider | nothing |
| Skipped - inactive provider | Marked N in the input | green | Column C was `N`; the bot did not open this provider | nothing |
| Login Failed | Check the portal username/password | **no colour** (white) | Sign-in failed; **every** provider gets this row; nothing was downloaded | E01 |
| Error | Report grid did not load | red | The provider's report page did not load in 30 s | E04 |
| Error | Download did not finish | red | No new CSV appeared within 60 s of clicking Export | E05 |
| Error | Session expired | red | The portal kept logging the bot out | E06 |
| (no row) | - | - | Provider is in the input but not in the report | E03 or E09 |

## 7. Exception catalogue

| ID | What you see | How the bot identifies it | Where it is recorded | Type | Who acts | What to do | Escalate when |
|---|---|---|---|---|---|---|---|
| E01 | Every row **Login Failed** (cells not coloured) - "Check the portal username/password" | After clicking Sign in it waits 30 s for the portal dashboard; if it never appears, sign-in failed | Today's report, all rows | Business | Ops | Sign in to the portal by hand with the Login sheet's username/password. If it fails, fix A2/B2 (or reset the password) and re-run. If the portal is down, re-run later. | Manual sign-in works with the same details but the bot still gets Login Failed |
| E02 | No `Download_Report_<today>.xlsx` in `data/output/` | Nothing - the bot stopped before it could save (Chrome would not start, input file missing or open/locked, `config.yaml` or a folder missing, task ended by hand, or the save itself failed) | Nowhere ops can see | System | Ops, then Dev | Check the section 3 list (Chrome installed, `config.yaml`, folders, input file present and closed, Task Scheduler task ran). Fix and re-run. | Everything on the list is fine and still no report. Send the date/time of the run. |
| E03 | Report exists but has fewer rows than the input (sometimes only the header row) | Nothing - an unexpected page or error stopped the run (the sign-in page changed, Chrome was closed, re-sign-in failed during the run, the Export button was missing). The bot saves whatever rows it has and stops. | Today's report (rows missing); cause only in the developer log | System | Dev | Copy the report out of `data/output`, then re-run the whole input once (safe: the bot only downloads). | It happens again on the re-run. Send the date/time, the report, and the name of the first missing provider. |
| E04 | Red **Error** - "Report grid did not load" | Waits 30 s for the provider's report grid; it never appears | Today's report; screenshot `data/screens/<NPI>.png` | System | Ops | Open the screenshot. Portal slow or showing an error → re-run later. Page shows "not found"/no access → check the NPI in the input against the portal. | Same provider fails on 2 days in a row with a correct NPI. Send the screenshot and the date. |
| E05 | Red **Error** - "Download did not finish" | After clicking Export CSV it watches `data/downloads/` for a new CSV for 60 s; none appears | Today's report | System | Ops | **Check `data/downloads/` first**: the file may have arrived after 60 s. If a CSV from that minute is there, the download did work. Otherwise re-run (safe). | No file after a re-run. Send the date/time and provider row. |
| E06 | Red **Error** - "Session expired" | Opening the provider's page sent it back to the sign-in page 3 times, with 2 re-sign-ins in between | Today's report | System | Bot, then Ops | Nothing during the run (the bot already retried). Re-run later. | Several providers show it in the same run, or it repeats next day. Send the date/time. |
| E07 | Green **Downloaded** - "portal maintenance - no file" | The provider's page shows the portal's maintenance banner | Today's report | System | **Ops** | **No file was downloaded despite the green status.** Re-run after the maintenance window (copy today's report first). | The banner is shown for more than 1 day |
| E08 | Red **Error** on most providers right after one "Session expired" or slow period | Same checks as E04/E05/E06, the portal is slow or unstable | Today's report | System | Ops | Re-run later in the day (copy today's report first). | It happens on 2 runs in a row |
| E09 | A provider in the input has no row in the report, while the rows after it do | Rows with an empty NPI (column A) are skipped without a message | Nowhere (the row is simply missing) | Business | Ops | Fill in the NPI in column A and re-run. | NPI is filled and the row is still missing |
| E10 | A provider you meant to skip was downloaded | Only a capital `N` in column C means inactive | Today's report shows a status other than Skipped | Business | Ops | Change column C to a capital `N`. | `N` is set and it is still downloaded |
| E11 | A CSV in `data/downloads/` you can't match to a provider | The bot does not rename downloads; files keep the portal's name | `data/downloads/` | System | Ops | Match by time: report row order = download order. | Matching by time is not possible (send the date) |

## 8. What the bot recovers by itself

| Problem | What the bot does | Limit | After the limit |
|---|---|---|---|
| Portal logs the bot out (sent back to the sign-in page) | Signs in again and re-opens the same provider | 2 re-sign-ins per provider (3 tries) | Row = Error "Session expired", next provider. If a re-sign-in itself fails, the run stops (E03). |
| Sign-in at the start fails | No retry | 1 try, 30 s | All rows Login Failed, run ends |
| Report grid slow | No retry | 30 s | Row = Error + screenshot, next provider |
| Download slow | No retry | 60 s | Row = Error, next provider |
| A page hangs | **Nothing - no watchdog** | none | Ops ends the task after 10 minutes (section 4) |
| Any unexpected error | **Nothing** | - | Run stops; report saved with the rows so far (E03) or not at all (E02) |

There is no automatic re-run. A failed provider is only retried by tomorrow's 07:00 run or a manual re-run.

## 9. Known incidents

No incident history exists for this bot: no change history, bug log or release notes are kept with it.

| When / version | What ops saw | Cause | What to do now | Ops could handle? |
|---|---|---|---|---|
| 2026-09-02 07:00 run | One portal log-out during the run, then one provider showed Error | Not recorded: the developer log keeps the status but not the reason | Read the Remarks column for that row; follow E04-E06 | Partly |

## 10. Releases and version check

- How an update arrives: by hand - a developer replaces the bot files on the bot machine. There is no launcher or auto-update.
- How to confirm it applied: there is no version number anywhere. Ask the developer what changed and check that behaviour.
- If the behaviour did not change: raise a ticket; ops cannot check versions themselves.
- The bot does not write to anything shared on start, so an old copy on another machine cannot undo a newer one.

## 11. Escalating to dev

Raise a ticket in the **RPA Support** queue.

Send: the date and time of the run, the exception ID from section 7 (e.g. E04), today's report file from `data/output/`, the screenshot from `data/screens/` if there is one, and the provider row number in the input.
Never send: the portal password, the Login sheet, `config.yaml`, downloaded CSV files, or patient or provider data in chat.

## 12. Failure-class check

- F1 Missing prerequisite: covered in section 3 (Chrome, `config.yaml`, folders, input) and E02.
- F2 Wrong window / focus: covered in section 4 (don't touch or close the automated Chrome window). The bot drives Chrome through its own session, not by window title, so other windows are not grabbed.
- F3 Slower than expected: covered in E04, E05, E06, E08 and section 8 (30 s / 60 s limits, no retry). No screen-size dependency (web pages addressed by element, not position).
- F4 Input read differently: covered in section 3 input rules and E09, E10; fix item 6.
- F5 Done when nothing was done: covered in E07 (maintenance stored as Downloaded) and E03 (missing rows, no status); fix items 1, 2.
- F6 Action happened, not confirmed: covered in E05 (check `data/downloads/` before re-running). Re-running is safe: downloading twice posts nothing.
- F7 Business exception shown as success: covered in section 6 and E07; fix item 1.
- F8 Endless re-runs / watchdog: no auto-rerun loop (re-sign-in limited to 2 per provider, section 8). No watchdog at all: section 4 stuck rule; fix item 8.
- F9 Stopping the wrong way: covered in section 4 (End the task in Task Scheduler; closing Chrome is not a clean stop); fix item 8.
- F10 One stuck item blocks the queue: not applicable: there are no batches, locks or completion gates; each run is independent and each provider is tried in turn. (A crash stops the remaining providers: E03.)
- F11 Cause hidden from ops: covered in fix items 2, 3, 5 (crash cause and log only in the developer log, which ops cannot open).
- F12 Update didn't arrive: covered in section 10 (manual updates, no version shown); fix item 9.

## 13. Fix before handover (dev team)

| # | Gap | Why it hurts ops | Where (file:line) |
|---|---|---|---|
| 1 | Portal maintenance is stored as status **Downloaded** (green) with no file | Ops reads it as done; the provider's report is missed | `bot.py:54-55` |
| 2 | Only `SessionExpired` and the start-up login `RuntimeError` are caught. Any other error (`NoSuchElementException`, `WebDriverException`, a `RuntimeError` from the re-login at line 102) ends the run; remaining providers get no row and the traceback is not written to `bot.log` (goes to stderr, lost under Task Scheduler) | Report silently short (E03), cause invisible | `bot.py:85-109`, `bot.py:98-102` |
| 3 | `bot.log` logs status only, not the remark; the log is not reachable by ops | No way for ops to see causes beyond the Remarks column | `bot.py:105` |
| 4 | Report named by date only; a second run the same day overwrites the first | Ops loses the earlier run's results | `bot.py:110` |
| 5 | Downloads are not renamed per provider; success = "any new `*.csv` in the folder", so a stray or late CSV from a previous provider can be credited to the wrong one | CSVs can't be tied to providers (E11); possible false Downloaded | `bot.py:63-68` |
| 6 | Inactive flag is exact `"N"` only; blank NPI rows are dropped silently | Lowercase `n` / `No` downloads anyway (E10); missing rows (E09) | `bot.py:77-78` |
| 7 | Folders `logs`, `data/output`, `data/downloads`, `data/screens` are never created; `save_screenshot` fails silently if `data/screens` is missing | Crash with no report, or missing screenshots (E02, E04) | `bot.py:20-21`, `bot.py:30`, `bot.py:59`, `bot.py:110` |
| 8 | Workbook load and Chrome start are outside the `try/finally`; no watchdog / overall timeout; no clean stop path (ending the task skips the report save) | No report at all (E02); hung runs need a manual kill | `bot.py:75-81`, `bot.py:85-110` |
| 9 | No version string anywhere; `requirements.txt` unpinned | Ops can't confirm an update; a fresh install may pull a different Selenium | `bot.py` (none), `requirements.txt:1-3` |
| 10 | Download wait is a hard 60 s; a slow download shows Error although the file arrives later | False Error rows (E05) | `bot.py:23`, `bot.py:65-70` |
| 11 | `config.yaml` holds an `api_token` in plain text, committed with the code, and the code never uses it | Secret exposure; anyone with the folder has the token. Rotate it and move it to a protected `.env`, or delete it | `config.yaml:2` |
| 12 | Portal password is kept in plain text in the input Excel (Login sheet B2) | Everyone who edits the input sees the password | `bot.py:76` |
| 13 | Screenshot only for "grid did not load"; no screenshot for login failure, download failure or session expiry | Ops has nothing to look at for E01, E05, E06 | `bot.py:44-47`, `bot.py:65-70`, `bot.py:103-104` |
| 14 | "Login Failed" rows are written without the red fill (fill is only applied inside the provider loop); login failure is also not logged | A failed run looks uncoloured instead of red | `bot.py:88-91`, `bot.py:107` |