MISF-Planner-Download/README.md

# OneDrive / Planner download-only automation

Automates the MISF VOB **face-sheet download** workflow: sign in to OneDrive, open Planner, search each facility plan, download attachment PDFs, and mark tasks **In progress**.

Reference: [`docs/MISF-VOB-Planner-Workflow.docx`](docs/MISF-VOB-Planner-Workflow.docx)

This repository contains **only** OneDrive / Planner download automation. There is no Noridian Medicare, Medi-Cal, PDF extraction, or eligibility verification code.

---

## Source requirements

| Doc step | Action |
|----------|--------|
| 1 | Open OneDrive URL, enter username / password, complete MFA (Google Authenticator 6-digit code) |
| 2–4 | Open **Planner** (direct URL in config; default: Assigned to me grid) |
| 5 | For each facility: search/select plan → open tasks → **download attachment PDFs** → set task status **In progress** (board card, or Task details → Status) |

**Clarifications:**

- Process **54 facilities** by typing each name into Planner’s **search box**, one at a time.
- MFA via **TOTP secret** in `Input/input.xlsx` (automated with `pyotp`).

**Not in this repo (upload path from the MISF guide):**

- Uploading completed work to OneDrive
- Marking tasks **Completed** after eligibility verification

```mermaid
flowchart TD
    start[run_download_workflow] --> config[Read input.xlsx]
    config --> chrome[start_chrome_debug + attach_browser]
    chrome --> login[Microsoft login + TOTP]
    login --> planner[Open Planner / In my plans]
    planner --> loop{For each facility}
    loop --> search[Search facility in Planner]
    search --> dl[Download task attachments]
    dl --> status[Mark tasks In progress]
    status --> loop
    loop --> summary[Log summary + Excel report]
```

---

## Repository layout

| File | Role |
|------|------|
| [`run_download.py`](run_download.py) | Main entry point |
| [`src/planner_download.py`](src/planner_download.py) | Orchestrator: config → Chrome → login → facility loop → summary |
| [`src/onedrive_planner.py`](src/onedrive_planner.py) | Selenium: Microsoft login, Planner, downloads, In-Process |
| [`src/config.py`](src/config.py) | `read_app_config()`, `read_facility_list()` |
| [`src/totp_utils.py`](src/totp_utils.py) | TOTP secret sanitization / validation |
| [`scripts/validate_config.py`](scripts/validate_config.py) | Pre-flight config check (no browser) |
| [`src/chrome.py`](src/chrome.py) | Chrome debug startup, Selenium attach, proxy auth |
| [`scripts/bootstrap_input_xlsx.py`](scripts/bootstrap_input_xlsx.py) | Create config template |
| [`tests/test_download_config.py`](tests/test_download_config.py) | Config smoke tests |

```
.
├── README.md
├── requirements.txt
├── run_download.py
├── docs/
├── Input/
│   ├── input.xlsx.example
│   ├── input.xlsx          # gitignored
│   └── PDFs/
├── Output/
├── scripts/
├── tests/
└── src/
    ├── planner_download.py
    ├── onedrive_planner.py
    ├── config.py
    └── chrome.py
```

---

## Configuration (`Input/input.xlsx`)

```powershell
copy Input\input.xlsx.example Input\input.xlsx
copy .env.example .env
```

Fill credentials in **either** place:

- **`Input/input.xlsx`** → Sheet1, column **B** (`OneDrive Username`, `OneDrive Password`, `Microsoft TOTP Secret`), **or**
- **`.env`** in the project root (`ONEDRIVE_USERNAME`, `ONEDRIVE_PASSWORD`, `MICROSOFT_TOTP_SECRET`)

Check config before running:

```powershell
python scripts\validate_config.py
```

### Sheet `Sheet1`

| Key | Purpose |
|-----|---------|
| `OneDrive Username` | Microsoft login email |
| `OneDrive Password` | Password |
| `Microsoft TOTP Secret` | Base32 authenticator secret (`pyotp`) |
| `OneDrive URL` | OneDrive landing URL |
| `pdf_folder_path` | Download root (default: `Input/PDFs`) |
| `Proxy IP` / `Proxy Port` / `Proxy Username` / `Proxy Password` | Optional |

### Sheet `Facilities` — **put all 54 facility names here**

Open **`Input/input.xlsx`** (not the `.example` file) and edit the **`Facilities`** sheet:

| Column | Purpose |
|--------|---------|
| `Facility Name` | **One row per facility** (54 rows). Use the exact text you type in Planner search. |

Example:

| Facility Name |
|---------------|
| Amaya |
| Brighton Place San Diego |
| … (52 more rows) |

The bot reads this sheet via `read_facility_list()` in `src/config.py`. For a first test, keep only **2 rows** and run `python run_download.py --limit 2`.

---

## Production `.exe` (double-click)

Build a Windows folder you can copy to any PC:

```powershell
powershell -ExecutionPolicy Bypass -File scripts\build_production_exe.ps1
```

Output at **project root**: `MISF_VOB_Download.exe` and `_internal\` (same folder as `Input/`, `Output/`, `run_download.py`).

**Deploy:** zip the project folder (omit `.venv`, `build`, `src` if you only ship the exe build). On the target machine:

1. Unzip anywhere
2. Copy `Input\input.xlsx.example` → `Input\input.xlsx` and configure credentials + facilities
3. Double-click `MISF_VOB_Download.exe`

Config, `Input/`, and `Output/` are read from the **project root** (folder containing the `.exe`). See [`README_PRODUCTION.txt`](README_PRODUCTION.txt) after build.

---

## Quick start

```powershell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
copy Input\input.xlsx.example Input\input.xlsx
# Edit Input\input.xlsx

python run_download.py --limit 2              # dry-run (first N facilities)
python run_download.py --facility "Amaya"     # single facility
python run_download.py --resume               # continue after last checkpoint
python run_download.py --keep-browser         # leave browser open when done
python run_download.py                        # full run
```

**Outputs:**

- `Input/PDFs/<FacilityName>/*.pdf`
- `Output/<date>/Summary/Download_Summary_<date>.xlsx` (includes run timestamp and Planner URL)
- `Output/<date>/logs/download.log`
- `Output/<date>/checkpoint.json` (while a run is in progress; cleared on success)

Set `MISF_DEBUG=1` in `.env` to enable NDJSON debug traces (`debug-fc545e.log`).

---

## Tests

```powershell
python -m pytest
python scripts\validate_config.py
```

---

## Troubleshooting

| Issue | What to check |
|-------|----------------|
| Login / MFA fails | TOTP secret; system clock (NTP) |
| Facility not found | `Facility Name` matches Planner search |
| No PDFs downloaded | Attachments exist; tune `src/onedrive_planner.py` selectors |
| Proxy errors | Proxy IP/Port in `input.xlsx`; profile `%USERPROFILE%\Automation\ChromiumProfile_MISF_Download` |
| Browser not found | Install [Chromium](https://www.chromium.org/getting-involved/download-chromium/) or set `CHROMIUM_PATH` in `.env` (Google Chrome only with `MISF_ALLOW_CHROME=1`) |
| `ChromeDriver only supports Chrome version 148` / attach fails | Install Chromium matching driver version; driver auto-downloads to `.drivers/` |

---

## Browser

The automation uses **Chromium only** (minimized in the background by default). It looks for, in order:

1. `CHROMIUM_PATH` or `BROWSER_PATH` environment variable
2. **`chrome-win/chrome-win/chrome.exe`** (portable bundle at project root — default for production)
3. Bundled **`chromedriver.exe`** in the same folder (no download needed)
4. System Chromium under `Program Files\Chromium`, `%LOCALAPPDATA%\Chromium`, Scoop (`MISF_USE_INSTALLED_CHROME=1` to prefer this over `chrome-win`)
5. Google Chrome only when `MISF_ALLOW_CHROME=1`

Set `MISF_BROWSER_FOREGROUND=1` to show/maximize the browser window. The production `.exe` runs without a console window; see `Output/<date>/logs/download.log`.

**Chromedriver:** `webdriver-manager` downloads a matching driver. If attach fails, install Chromium that matches your driver or set `CHROMIUM_PATH` explicitly.

User profile directory: `%USERPROFILE%\Automation\ChromiumProfile_MISF_Download`

---

## UiPath migration

An equivalent **UiPath Studio (Modern)** project lives in [`uipath/MISF_Planner_Download/`](uipath/MISF_Planner_Download/). Open `project.json` in UiPath Studio. Pre-flight check:

```powershell
powershell -File scripts\validate_uipath_pilot.ps1
python -m pytest tests\test_uipath_pilot.py -v
```

---

Internal MISF / Access Psychology automation.