Lubna Fathima 291507c2f7 Add ECW/HTTP planner scripts, browser crash recovery, and Planner error-overlay handling
- Add ecw_download.py and onedrive_planner_http.py for ECW and HTTP-based Planner access
- Add build_exe.bat for packaging
- Restart Chrome automatically on crash mid-run in planner_download.py
- Detect and recover from Planner "something went wrong" error overlays
- Improve board-load waiting, sort-descending menu matching, and screenshot timestamp rendering in chrome.py/onedrive_planner.py
2026-07-09 08:41:28 -07:00

OneDrive / Planner download-only automation

Automates the MISF VOB face-sheet download workflow: sign in to OneDrive, open Planner, search each facility plan, download attachment PDFs, and mark tasks In progress.

Reference: docs/MISF-VOB-Planner-Workflow.docx

This repository contains only OneDrive / Planner download automation. There is no Noridian Medicare, Medi-Cal, PDF extraction, or eligibility verification code.


Source requirements

Doc step Action
1 Open OneDrive URL, enter username / password, complete MFA (Google Authenticator 6-digit code)
2–4 Open Planner (direct URL in config; default: Assigned to me grid)
5 For each facility: search/select plan → open tasks → download attachment PDFs → set task status In progress (board card, or Task details → Status)

Clarifications:

  • Process 54 facilities by typing each name into Planner’s search box, one at a time.
  • MFA via TOTP secret in Input/input.xlsx (automated with pyotp).

Not in this repo (upload path from the MISF guide):

  • Uploading completed work to OneDrive
  • Marking tasks Completed after eligibility verification
flowchart TD
    start[run_download_workflow] --> config[Read input.xlsx]
    config --> chrome[start_chrome_debug + attach_browser]
    chrome --> login[Microsoft login + TOTP]
    login --> planner[Open Planner / In my plans]
    planner --> loop{For each facility}
    loop --> search[Search facility in Planner]
    search --> dl[Download task attachments]
    dl --> status[Mark tasks In progress]
    status --> loop
    loop --> summary[Log summary + Excel report]

Repository layout

File Role
run_download.py Main entry point
src/planner_download.py Orchestrator: config → Chrome → login → facility loop → summary
src/onedrive_planner.py Selenium: Microsoft login, Planner, downloads, In-Process
src/config.py read_app_config(), read_facility_list()
src/totp_utils.py TOTP secret sanitization / validation
scripts/validate_config.py Pre-flight config check (no browser)
src/chrome.py Chrome debug startup, Selenium attach, proxy auth
scripts/bootstrap_input_xlsx.py Create config template
tests/test_download_config.py Config smoke tests
.
├── README.md
├── requirements.txt
├── run_download.py
├── docs/
├── Input/
│   ├── input.xlsx.example
│   ├── input.xlsx          # gitignored
│   └── PDFs/
├── Output/
├── scripts/
├── tests/
└── src/
    ├── planner_download.py
    ├── onedrive_planner.py
    ├── config.py
    └── chrome.py

Configuration (Input/input.xlsx)

copy Input\input.xlsx.example Input\input.xlsx
copy .env.example .env

Fill credentials in either place:

  • Input/input.xlsx → Sheet1, column B (OneDrive Username, OneDrive Password, Microsoft TOTP Secret), or
  • .env in the project root (ONEDRIVE_USERNAME, ONEDRIVE_PASSWORD, MICROSOFT_TOTP_SECRET)

Check config before running:

python scripts\validate_config.py

Sheet Sheet1

Key Purpose
OneDrive Username Microsoft login email
OneDrive Password Password
Microsoft TOTP Secret Base32 authenticator secret (pyotp)
OneDrive URL OneDrive landing URL
pdf_folder_path Download root (default: Input/PDFs)
Proxy IP / Proxy Port / Proxy Username / Proxy Password Optional

Sheet Facilities — put all 54 facility names here

Open Input/input.xlsx (not the .example file) and edit the Facilities sheet:

Column Purpose
Facility Name One row per facility (54 rows). Use the exact text you type in Planner search.

Example:

Facility Name
Amaya
Brighton Place San Diego
… (52 more rows)

The bot reads this sheet via read_facility_list() in src/config.py. For a first test, keep only 2 rows and run python run_download.py --limit 2.


Production .exe (double-click)

Build a Windows folder you can copy to any PC:

powershell -ExecutionPolicy Bypass -File scripts\build_production_exe.ps1

Output at project root: MISF_VOB_Download.exe and _internal\ (same folder as Input/, Output/, run_download.py).

Deploy: zip the project folder (omit .venv, build, src if you only ship the exe build). On the target machine:

  1. Unzip anywhere
  2. Copy Input\input.xlsx.example → Input\input.xlsx and configure credentials + facilities
  3. Double-click MISF_VOB_Download.exe

Config, Input/, and Output/ are read from the project root (folder containing the .exe). See README_PRODUCTION.txt after build.


Quick start

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
copy Input\input.xlsx.example Input\input.xlsx
# Edit Input\input.xlsx

python run_download.py --limit 2              # dry-run (first N facilities)
python run_download.py --facility "Amaya"     # single facility
python run_download.py --resume               # continue after last checkpoint
python run_download.py --keep-browser         # leave browser open when done
python run_download.py                        # full run

Outputs:

  • Input/PDFs/<FacilityName>/*.pdf
  • Output/<date>/Summary/Download_Summary_<date>.xlsx (includes run timestamp and Planner URL)
  • Output/<date>/logs/download.log
  • Output/<date>/checkpoint.json (while a run is in progress; cleared on success)

Set MISF_DEBUG=1 in .env to enable NDJSON debug traces (debug-fc545e.log).


Tests

python -m pytest
python scripts\validate_config.py

Troubleshooting

Issue What to check
Login / MFA fails TOTP secret; system clock (NTP)
Facility not found Facility Name matches Planner search
No PDFs downloaded Attachments exist; tune src/onedrive_planner.py selectors
Proxy errors Proxy IP/Port in input.xlsx; profile %USERPROFILE%\Automation\ChromiumProfile_MISF_Download
Browser not found Install Chromium or set CHROMIUM_PATH in .env (Google Chrome only with MISF_ALLOW_CHROME=1)
ChromeDriver only supports Chrome version 148 / attach fails Install Chromium matching driver version; driver auto-downloads to .drivers/

Browser

The automation uses Chromium only (minimized in the background by default). It looks for, in order:

  1. CHROMIUM_PATH or BROWSER_PATH environment variable
  2. chrome-win/chrome-win/chrome.exe (portable bundle at project root — default for production)
  3. Bundled chromedriver.exe in the same folder (no download needed)
  4. System Chromium under Program Files\Chromium, %LOCALAPPDATA%\Chromium, Scoop (MISF_USE_INSTALLED_CHROME=1 to prefer this over chrome-win)
  5. Google Chrome only when MISF_ALLOW_CHROME=1

Set MISF_BROWSER_FOREGROUND=1 to show/maximize the browser window. The production .exe runs without a console window; see Output/<date>/logs/download.log.

Chromedriver: webdriver-manager downloads a matching driver. If attach fails, install Chromium that matches your driver or set CHROMIUM_PATH explicitly.

User profile directory: %USERPROFILE%\Automation\ChromiumProfile_MISF_Download


UiPath migration

An equivalent UiPath Studio (Modern) project lives in uipath/MISF_Planner_Download/. Open project.json in UiPath Studio. Pre-flight check:

powershell -File scripts\validate_uipath_pilot.ps1
python -m pytest tests\test_uipath_pilot.py -v

Internal MISF / Access Psychology automation.

S
Description
No description provided
Readme
817 KiB
0 Stars 8 Watchers 0 Forks
Languages
Python 98.4%
PowerShell 1.2%
Batchfile 0.4%