Rpa_Skills/tests/evals/results/handoff/h-web-portal-full-r5.md

# Payer portal downloader - ops handover

Version: not set (the bot has no version number, see section 10) · Written 2026-10-23 · Escalation: RPA dev team, Teams channel **#rpa-support**

## 1. What the bot does

Every morning at 07:00 Windows Task Scheduler starts the bot. It signs in to the payer web portal once. Then, for each provider in the input Excel `data/input/providers.xlsx`, it opens that provider's remittance page and downloads the remittance report as a CSV file. The input file has two sheets: **Login** (portal username and password) and **Providers** (NPI, name, active flag). The bot produces three things:
- a report Excel, `data/output/Download_Report_<YYYYMMDD>.xlsx`, with one row per provider and its Status and Remarks
- the CSV files, in `data/downloads/`
- a screenshot in `data/screens/` when a provider's report page does not load

The bot drives one app: a **Google Chrome window** that it opens itself.

## 2. How it works

```
Windows Task Scheduler (07:00 daily)
   └─ starts → bot (no window of its own) ──drives──> its own Chrome window → payer web portal
                    │                                    (do not touch while it runs)
                    ├─ reads  → data/input/providers.xlsx
                    ├─ writes → data/downloads/*.csv  (one CSV per downloaded provider)
                    ├─ writes → data/screens/<NPI>.png (only when a page does not load)
                    ├─ writes → data/output/Download_Report_<YYYYMMDD>.xlsx (saved at the very end)
                    └─ writes → logs/bot.log (one line per provider)
```

- **Task Scheduler** starts the bot. The bot has no screen or window of its own.
- **The bot** works in the background and controls a Chrome window that it opens. It closes that window when it finishes.
- **The payer web portal** is the website that Chrome shows.
- **The input Excel** tells the bot who to sign in as and which providers to download.
- **`data/downloads/`** gets one CSV per downloaded provider. The file keeps the name the portal gives it; it is **not** renamed per provider.
- **`data/screens/`** gets one picture per provider whose report page did not load. It is named by NPI and replaced every day.
- **The report Excel** is written only when the run ends. A run that is still going, or one that was killed, has no report.
- **`logs/bot.log`** gets one line per provider (`<NPI> <name> -> <status>`) and one warning line each time the bot has to sign in again.

**One run, step by step:**
1. Read the input Excel.
2. Open Chrome.
3. Sign in to the portal (wait up to 30 s for the dashboard).
4. For each provider:
   - inactive → skip
   - otherwise open its remittance page → wait up to 30 s for the report grid → click Export CSV → wait up to 60 s for a new CSV file
5. Close Chrome and save the report Excel.

## 3. Before a run

- [ ] Google Chrome is installed and up to date on the machine. Python and the bot's packages are installed. If Chrome cannot start, there is no report at all (E02).
- [ ] These folders exist inside the bot folder: `data/input`, `data/output`, `data/downloads`, `data/screens`, `logs`. **The bot does not create them.** Without `data/output` there is no report (E02). Without `data/screens` the screenshot is silently not saved.
- [ ] The settings file `config.yaml` is in the bot folder.
- [ ] Input file `data/input/providers.xlsx` has sheets named exactly **Login** and **Providers**.
- [ ] **Login** sheet: the username is in cell A2 and the password in B2. Both must be filled. If either is blank, the run stops before any provider (E02).
- [ ] **Providers** sheet: one provider per row, starting at row 2, with:
  - Column A = NPI. A row with a blank NPI is ignored and does **not** appear in the report.
  - Column B = provider name.
  - Column C = active flag. Only a capital **N** means inactive. Good: `N`. Bad: `n`, `No`, `inactive`. Those, and a blank cell, all count as *active*, so the provider is downloaded.
- [ ] Move or archive yesterday's CSVs from `data/downloads/` if you need to tell today's files apart. The bot never cleans that folder.

## 4. During a run

- Don't click, type in, close or resize the Chrome window the bot opened. It belongs to the bot. Closing it stops the whole run (E03).
- Don't put any CSV file into `data/downloads/` while the bot runs. The bot counts **any** new CSV there as the current provider's download (E10).
- Don't run the bot twice on the same day without a reason. A second run downloads every provider again and **overwrites** that day's report.
- **To stop the bot:**
  1. Open Task Scheduler, right-click the bot's task, and choose **End**.
  2. Then close any Chrome window the bot left open.
  3. A run stopped this way writes **no report**.
- **A run looks stuck** when the Chrome page has not changed for more than 5 minutes.
  - One provider normally takes under 2 minutes.
  - The bot has **no watchdog**: it never restarts itself.
  - If one portal page keeps loading, the run stops on its own with a crash after about 5 minutes (E03). Wait that long, then stop it as above.

## 5. Where to look when something goes wrong

| Question | Where to look |
|---|---|
| Did the run finish? | `data/output/` has `Download_Report_<today>.xlsx`, and its row count matches the providers in the input (minus rows with a blank NPI). No file means E02. Fewer rows means E03. |
| Which providers failed? | The report Excel, Status column: red cells, and white cells ("Login Failed"). **Also** check the Remarks column for "portal maintenance - no file". Those rows are green but nothing was downloaded (E07). |
| Why did one provider fail? | The Remarks column on that provider's row. For "Report grid did not load", open `data/screens/<NPI>.png`. |
| Did the bot already retry? | `logs/bot.log`: search for the NPI, then look just above it for lines `session expired, re-login 1/2/3`. Only an expired session is ever retried. Nothing else is. |
| Was a CSV really downloaded? | `data/downloads/`: compare the number of today's CSVs with the number of rows whose remark is "Report saved". The files are not named per provider. |
| Which version is running? | Not possible today (see section 10). |
| Why did the run crash? | Not recorded anywhere ops can open. Escalate (E02 / E03). |

## 6. Statuses

All statuses are in the report Excel, in the Status column with the Remarks column next to it.

| Status - Remarks | Colour | Meaning | What ops does |
|---|---|---|---|
| Downloaded - "Report saved" | green | A new CSV appeared in `data/downloads/` | nothing (optionally check the CSV count, E10) |
| Downloaded - "portal maintenance - no file" | green | **Needs follow-up.** The portal showed a maintenance banner. **Nothing was downloaded.** | Download it later (E07) |
| No Data - "No remittances in the date range" | green | The portal says there are no remittances for this provider | nothing, unless the provider is expected to have payments (E08) |
| Skipped - inactive provider - "Marked N in the input" | green | Column C was a capital N | nothing |
| Error - "Report grid did not load" | red | The provider's page did not load in 30 s. No retry. | E04 |
| Error - "Download did not finish" | red | Export was clicked but no CSV appeared in 60 s. No retry. | E05 |
| Error - "Session expired" | red | The portal logged the bot out 3 times on this provider | E06 |
| Login Failed - "Check the portal username/password" | **no colour (white)** | Sign-in failed. Every provider gets this row and nothing was downloaded. | E01 |
| *(provider missing from the report)* | - | The NPI was blank in the input, or the run crashed before reaching that provider | E03 / section 3 |

## 7. Exception catalogue

| ID | What you see | How the bot identifies it | Where it is recorded | Type | Who acts | What to do | Escalate when |
|---|---|---|---|---|---|---|---|
| E01 | Every row says "Login Failed - Check the portal username/password" (white) | After signing in, it waits 30 s for the portal dashboard. If the dashboard never appears, it gives up. | Report Excel only. Nothing in the log. | Business (wrong password) or System (portal slow/down) | Ops | Sign in to the portal by hand with the Login sheet's username and password. If that fails, fix the password in the Login sheet (B2) and re-run. If the portal itself is down, re-run later. | The manual sign-in works but the bot still fails. Send the date and the report file. |
| E02 | No `Download_Report_<today>.xlsx` at all, or a report with only the header row | The bot crashed before or during sign-in. Causes: Chrome could not start, a folder is missing, the input file or a sheet is missing, a blank username or password, or the login page looks different. | Nowhere ops can open. The crash message is lost. | System | Ops, then Dev | Work through the section 3 checklist: Chrome installed, all folders exist, input sheets named right, A2/B2 filled. Re-run by hand. | It still fails after the checklist. Send the date, the time, the input file's sheet names (not the password), and the last lines of `logs/bot.log`. |
| E03 | The report stops partway: later providers are missing (no row at all) | Something the bot does not handle stopped the whole run. Causes: the Export CSV button was missing, a page kept loading for 5 minutes, the bot could not sign in again after being logged out, or someone closed the Chrome window. | The report has rows up to the crash. `logs/bot.log`: the last provider line shows where it stopped. The cause is not recorded. | System | Ops, then Dev | Note the last provider in the report. Re-run once (it downloads everyone again). Or download the missing providers by hand. | It stops at the same provider twice. Send the date, the time, the last NPI in `logs/bot.log`, and the report file. |
| E04 | Error - "Report grid did not load" (red) | It opens the provider's remittance page and waits 30 s for the report grid. If the grid never appears, it saves a screenshot and moves on. | Report Excel. Screenshot `data/screens/<NPI>.png`. Log line `<NPI> ... -> Error`. | System | Ops | Look at the screenshot. Open that provider in the portal by hand. If the report is there, download it by hand or re-run later. | More than a few providers in one run, or the same provider every day. Send the date, the NPIs, and the screenshots. |
| E05 | Error - "Download did not finish" (red) | It clicks Export CSV and checks `data/downloads/` every second for 60 s for a new CSV. None appeared. | Report Excel. Log line `<NPI> ... -> Error`. | System | Ops | **Look in `data/downloads/` first.** The file may have arrived late. If it is not there, download it from the portal by hand. A late file can also be counted for the next provider (E10). | It happens on most providers in a run. Send the date, the time, and the NPIs. |
| E06 | Error - "Session expired" (red) | The portal sent the bot back to its login page. The bot signs in again and retries, 3 attempts in all. | Report Excel. Log lines `session expired, re-login 1/2/3` just before that provider. | System | Bot, then Ops | Download that provider by hand or re-run later. | Several providers in the same run, or every day. Send the date and the log lines. |
| E07 | Downloaded - "portal maintenance - no file" (**green**) | The portal page shows a maintenance banner. The bot records Downloaded and moves on without trying. | Report Excel, Remarks column only. The log says just "Downloaded". | System | Ops | **Filter the Remarks column for "maintenance" after every run.** Re-run after the maintenance window, or download by hand. | Never needed. The fix is in section 13. |
| E08 | No Data - "No remittances in the date range" (green) | The portal shows its "no results" message for that provider | Report Excel | Business | Ops (only if it is unexpected) | If the provider should have payments, check the portal by hand. | The portal shows remittances but the bot says No Data. Send the NPI and the date. |
| E09 | A provider you expected is missing, or "Skipped - inactive provider" for an active provider | A row with a blank NPI is ignored. Only a capital N in column C counts as inactive. | Report Excel (missing row, or the Skipped status) | Business | Ops | Fix the input: fill in the NPI, and use a capital N only for inactive providers. | - |
| E10 | The number of CSVs in `data/downloads/` doesn't match the number of "Report saved" rows, or a CSV holds the wrong provider's data | The bot counts **any** new CSV in the folder as the current provider's download. A late file from the previous provider, or a file someone else dropped there, makes a provider look Downloaded. | Not recorded. Only visible by counting the files and opening them. | System | Ops, then Dev | Open the CSVs and check which provider each belongs to. Download the missing providers by hand. | It happens again. Send the date and the file names. |

## 8. What the bot recovers by itself

| Problem | What the bot does | Limit | After the limit |
|---|---|---|---|
| The portal logs the bot out (session expired) | Signs in again and retries the same provider | 3 attempts per provider (each sign-in waits up to 30 s) | Row = Error "Session expired", next provider. If the new sign-in itself fails, the whole run stops (E03). |
| Report grid does not load | **No retry.** Screenshot, then the next provider. | 30 s, once | Row = Error |
| CSV does not arrive | **No retry** | 60 s, once | Row = Error |
| Sign-in fails at the start | **No retry** | 30 s, once | Every row = Login Failed |
| Run is stuck or crashes | **Nothing.** No watchdog, no restart, no resume. | - | Ops re-runs by hand. A re-run repeats **every** provider. |

The worst case for one provider is about 3 minutes (three sign-ins plus the waits). Anything longer is not normal.

## 9. Known incidents

There is no change history or bug log for this bot. The only record is `logs/bot.log`, which covers one run.

| When / version | What ops saw | Cause | What to do now | Ops could handle? |
|---|---|---|---|---|
| 2026-09-02 / no version | One provider showed Error, right after a "session expired, re-login" warning | Not recorded: the log doesn't keep the remark. Most likely a slow page right after signing in again. | Follow E04 / E05 / E06 by the remark | Partly |

## 10. Releases and version check

- **How an update arrives:** there is no updater. The developer replaces the bot files in its folder. The next 07:00 run uses the new files.
- **How to confirm it applied:** not possible today. The bot has no version number in its files, its log or its report (see section 13).
- **If you suspect an old build:** ask in #rpa-support for the expected file date of the bot, and compare it with the date modified of the bot's main file in the bot folder.

## 11. Escalating to dev

Post in the **#rpa-support** Teams channel and send:
- the run date and the time it failed
- the exception ID from section 7
- the report file `data/output/Download_Report_<date>.xlsx`
- the relevant lines of `logs/bot.log`
- for E04, the screenshot `data/screens/<NPI>.png`

Never send: the portal password, the Login sheet, `config.yaml`, or provider or patient data in chat. Attach files rather than pasting their rows.

## 12. Failure-class check

- **F1** (missing prerequisite) covered in section 3 (Chrome, folders, input sheets, Login cells) and E02.
- **F2** (wrong window or focus) covered in section 4: don't touch the bot's Chrome window. The bot does not match windows by title or take focus, because it opens its own Chrome.
- **F3** (target app slower or different) covered in E03, E04, E05 and section 8. Grid and download waits are not retried (section 13).
- **F4** (input read differently) covered in section 3 (capital N only, blank NPI ignored, exact sheet names) and E09.
- **F5** (marked done when nothing was done) covered in E07 and E10, and section 13 items 1 and 6.
- **F6** (action done but not confirmed) covered in E05: check `data/downloads/` before downloading again. A late file can be credited to the next provider (E10).
- **F7** (business exception shown as success) covered in section 6 ("portal maintenance" shows green), E07, and section 13 item 1.
- **F8** (endless re-runs or watchdog kill): there is no rerun loop, so it cannot loop. There is also no watchdog: covered in sections 4 and 8, and section 13 item 2.
- **F9** (stopping the wrong way) covered in section 4 (End the task in Task Scheduler, then close Chrome).
- **F10** not applicable: there is no batch, completion gate or upload lock; each provider is independent. The nearest risk, a crash stopping all later providers, is E03.
- **F11** (cause hidden from ops) covered in section 5 ("why did the run crash" is not recorded) and section 13 items 2, 3 and 4.
- **F12** (update didn't arrive) covered in section 10 (no version shown) and section 13 item 8. The bot rewrites no shared resources on start.

## 13. Fix before handover (dev team)

| # | Gap | Why it hurts ops | Where (file:line) |
|---|---|---|---|
| 1 | The maintenance banner returns "Downloaded" (green) with no file | Ops thinks the report was downloaded | `bot.py:54-55` |
| 2 | No per-provider exception handling, and no watchdog. A missing Export button, a page-load timeout (driver default about 5 min) or a failed re-login ends the whole run. | Later providers are silently missing (E03) | `bot.py:51`, `bot.py:64`, `bot.py:92-107`, `bot.py:102` |
| 3 | Crash messages go to stderr only, which is lost under Task Scheduler. No traceback reaches `bot.log`. | The cause of E02/E03 is unrecoverable | `bot.py:30`, `bot.py:113-114` |
| 4 | "Login Failed" is never logged and its row is never coloured (the colour map has red, but it isn't applied). The log line never includes the remark. | Ops can't spot it by colour. The log can't tell the Error types apart (section 9). | `bot.py:88-91`, `bot.py:105` |
| 5 | The bot never creates `data/output`, `data/downloads` or `data/screens`. If `data/output` is missing, the report save in `finally` fails and hides the original error. A screenshot to a missing folder fails silently. | No report at all, and the cause is masked | `bot.py:20-21`, `bot.py:59`, `bot.py:108-110` |
| 6 | The download check accepts any new `*.csv`. Files are not renamed per provider. | A wrong or late file is credited to the wrong provider (E10). CSVs can't be tied to providers. | `bot.py:63-68` |
| 7 | Grid-load and download failures are never retried | Avoidable red rows that ops has to handle by hand | `bot.py:56-60`, `bot.py:65-70` |
| 8 | No version string anywhere (report, log, file) | Ops can't confirm which build ran | `bot.py:1-2` |
| 9 | The portal password is stored in plain text in the input Excel. `config.yaml` holds an access token that the code never uses: remove it and rotate it. | Credential exposure | `bot.py:76`, `config.yaml` |
| 10 | NPI and provider name are written to `bot.log` on every line, and the log is never rotated | Sensitive data piles up in an ops-visible file | `bot.py:30`, `bot.py:105` |
| 11 | A same-day re-run overwrites the report. Screenshots are overwritten daily. Downloads are never cleaned. There is no resume. | A re-run after E03 loses the first report and re-downloads everyone | `bot.py:59`, `bot.py:110` |
| 12 | The active flag is case-sensitive: only "N" counts | Inactive providers marked "n" are downloaded | `bot.py:77` |
| 13 | After the 3rd session expiry the bot signs in once more before giving up. A failed re-login is not caught. | Extra 30 s wait, or a whole-run crash | `bot.py:96-104` |
| 14 | `requirements.txt` is unpinned. A Selenium or openpyxl upgrade can change behaviour between machines. | "Works on one machine only" | `requirements.txt` |