Rpa_Skills/tests/evals/results/handoff/h-mail-intake-full-r1.md

# Mail intake bot - ops handover

Version: none (the project has no version number - see section 13, item 11) · Written 2026-10-23 · Escalation: RPA dev team, Teams channel **#rpa-support**

## 1. What the bot does

Every 10 minutes Windows Task Scheduler starts the bot. It reads every **unread** email in the referrals mailbox, saves each attachment, and checks that it is a PDF of 20 MB or less that can be opened. Good PDFs are copied to the **shared intake folder**. Every attachment becomes one item in the bot's queue with a status and a note, which ops read in the queue viewer workbook. The bot has no window of its own and does not drive any desktop app. It only talks to the mailbox and the shared intake folder.

## 2. How it works

```
Windows Task Scheduler (every 10 minutes)
   └─ starts → Mail intake bot (no window, runs a few seconds to minutes)
                   ├─ reads  → referrals mailbox (unread mail only; marks it read)
                   ├─ saves  → data/tmp/  (a working copy of every attachment)
                   ├─ copies → shared intake folder (good PDFs only)
                   ├─ writes → the queue (one item per attachment: status + note)
                   │              └─ ops read it in queue_viewer.xlsx (Data > Refresh)
                   └─ writes → logs/intake.log
```

- **Task Scheduler** decides when the bot runs. Nothing runs between the 10-minute starts.
- **The bot** is a background script. There is no screen to watch while it works.
- **Referrals mailbox:** the bot only sees mail that is still unread. As soon as it reads a mail, the mail is marked read and the bot never looks at it again.
- **data/tmp/** holds a copy of every attachment the bot downloaded. Files are never deleted from here.
- **Shared intake folder:** where the good PDFs end up. The file keeps the name it had in the email.
- **The queue** is the bot's list of items. Ops see it through **queue_viewer.xlsx** (columns id, sender, subject, status, note).
- **logs/intake.log** is the bot's only log file. One line per finished item, plus warnings and errors.

**One run, step by step:**
1. The bot checks the queue. If any item is still **Processing** from an earlier run, it writes an error to the log and stops. It does **not** read mail (see E02).
2. It signs in to the mailbox. If that fails it tries 3 times, 30 seconds apart (see E01).
3. It reads every unread mail. Each attachment is saved to `data/tmp/` and added to the queue as **New**.
4. It takes each New item in order and sets it to **Processing**.
5. It checks the file: PDF, 20 MB or less, not password-protected.
6. It copies the file to the shared intake folder and sets the final status and note.
7. It writes one log line per item: `item <id> -> <status> (<note>)`.

The log has no "run started" or "run finished" line.

## 3. Before a run

The bot runs by itself, so "before a run" means "before you hand the machine over, and after any change to it".

- [ ] The scheduled task for the bot exists and is enabled in Task Scheduler, set to every 10 minutes.
- [ ] The settings file `.env` is in the bot's folder and has the mailbox server, mailbox user, mailbox password and shared-folder entries. If an entry is missing, every run crashes and **nothing** is written to the log (E06).
- [ ] The folders `data/`, `data/tmp/` and `logs/` exist in the bot's folder, and the queue file is in `data/`. The bot does not create them. If one is missing, every run crashes silently (E06).
- [ ] The machine can reach the shared intake folder. Open it in Explorer as the account the scheduled task runs as. If you can't, items end as Failed (E05).
- [ ] The mailbox password has not expired. When it expires, the log fills with "IMAP connect failed" warnings (E01). Ask the RPA dev team when it is next due.
- [ ] What the bot accepts. Ops can't control what senders send, but this tells you what to expect:
  - Accepted: a PDF ending in `.pdf` (any case), 20 MB or less, with no password. Example: `referral_1023.pdf`, 2 MB.
  - Quarantined: anything else, including Word files, photos, signature logos and PDFs over 20 MB. Example: `scan.jpg` or `image001.png` from an email signature.
  - Lost or overwritten: two attachments with the **same file name**. The second one overwrites the first in the shared intake folder (E08).

## 4. During a run

- **Don't open new mail in the referrals mailbox.** Opening a mail in Outlook or webmail marks it read, and the bot then skips it for good (E07). If you opened one by mistake, mark it **unread** again. The next run picks it up.
- Don't move, rename or open files in `data/tmp/`. Each queue item points at its file there.
- Don't edit or rename files in the shared intake folder while the bot is copying. A run is usually short, but there is no "finished" signal.
- **To stop the bot:** in Task Scheduler, **disable** the bot's task. This stops future runs. Don't use "End" on a run that is in progress: stopping it mid-item leaves that item in Processing, and that blocks every later run (E02). If you don't have access to Task Scheduler, ask in #rpa-support.
- **A run looks stuck when:**
  - the log shows `ERROR Previous item still Processing - aborting` again every 10 minutes, or
  - the queue viewer shows an item in Processing for more than 10 minutes.
- **The bot does not recover from this by itself.** There is no watchdog and no time limit: the block lasts until a developer releases it. Escalate straight away (E02).

## 5. Where to look when something goes wrong

Ops can open: the queue viewer (`queue_viewer.xlsx`), the shared intake folder, and the `logs/` folder. Ops cannot open the queue database itself.

| Question | Where to look |
|---|---|
| Is the bot running at all? | `logs/intake.log`: the time of the last line. Runs with no new mail write nothing, so a quiet log alone is not an error. Send a test mail with a small PDF to the referrals mailbox. Within about 10 minutes, a new `item <id> -> Done` line should appear. |
| Did the last run finish? | There is no "finished" line. If no item in the queue viewer is in **Processing** and the log does not repeat `Previous item still Processing`, the last run finished. |
| Which items failed? | Queue viewer → Data > Refresh → status column: **Failed** and **Quarantined**, and also **Done** items whose note says `encrypted - not readable` (these were not delivered, E04). |
| Why did one item fail? | Queue viewer → note column. Then `logs/intake.log`, search for `item <id> ->`. |
| Did the bot already retry? | Mailbox sign-in: lines `IMAP connect failed (1/3)`, `(2/3)`, `(3/3)` in `logs/intake.log`. Items are **never** retried: Failed and Quarantined items stay that way. |
| Is the file really in the shared intake folder? | Open the shared intake folder and look for the file name from the item. Compare the page count with the note (`<n> pages saved`). |
| A referral never showed up in the queue | See E07. The bot keeps no record of mail it read without queueing anything. |
| Which version is running? | You can't see it. The project has no version number (section 10). |

## 6. Statuses

The bot sets no colours. The queue viewer shows the status as plain text in the status column.

| Status (note) | Colour | Meaning | What ops does |
|---|---|---|---|
| New (empty note) | none | Attachment saved and queued, not yet checked. | Nothing. It is handled in the same run or the next one. If it is still New after 20 minutes, see E02 and E01. |
| Processing (empty note) | none | The bot is working on this item right now. | Nothing for the first 10 minutes. **Still Processing after 10 minutes = the bot crashed on this item and every later run is blocked** → E02. |
| Done - "`<n>` pages saved" | none | Copied to the shared intake folder. | Nothing. |
| Done - "encrypted - not readable" | none | **Needs follow-up.** The PDF is password-protected. The status says Done, but the file was **not** copied to the shared intake folder. | E04: ask the sender for a copy without a password. |
| Failed - "share unreachable: `<reason>`" | none | The copy to the shared intake folder failed. The bot never tries again. | E05. |
| Quarantined - "not a PDF or over 20 MB" | none | Not a PDF, or larger than 20 MB. Not copied. The bot never tries again. | E03: usually nothing for signature logos. For a real referral, ask the sender for a PDF under 20 MB. |

## 7. Exception catalogue

| ID | What you see | How the bot identifies it | Where it is recorded | Type | Who acts | What to do | Escalate when |
|---|---|---|---|---|---|---|---|
| E01 | No new items arrive. The log shows `WARNING IMAP connect failed (1/3)`, `(2/3)`, `(3/3)` and then nothing more for that run. | Signing in to the mailbox fails. It tries 3 times, waiting 30 seconds after each try. After the third failure the run stops. That last failure is **not** written to the log. | `logs/intake.log` warning lines only. The text after the colon gives the reason: "getaddrinfo failed" means network or name lookup; an authentication or login message means a wrong or expired password, or a locked account. | System | Bot, then Dev | Wait for the next run (10 minutes later). A one-off network blip clears by itself. If the reason mentions login or authentication, the mailbox password has probably expired or the account is locked. | Three runs in a row (about 30 minutes) each end at `(3/3)`, or any login/authentication reason. Send: the time of the first `(1/3)` line, the reason text from the warning (never the password), and `logs/intake.log`. |
| E02 | Nothing new for a long time. The log shows `ERROR Previous item still Processing - aborting` every 10 minutes. One item in the queue viewer is stuck in **Processing**. | At the start of every run, the bot looks for any item still in Processing. If it finds one, it stops at once without reading mail. This happens after a crash in the middle of an item, for example on a damaged PDF, a missing file in `data/tmp/`, or someone ending the task. | `logs/intake.log` error line; the queue viewer status column. | System | **Dev** | The bot will **not** clear this by itself, and ops can't release it (it needs the database). Note the id, sender and subject of the Processing item. Check whether its file is already in the shared intake folder. Meanwhile, new mail waits safely in the mailbox as unread. | Always, as soon as you see the line twice. Send: the item id, the time the first error line appeared, whether the file is in the shared intake folder, and `logs/intake.log`. |
| E03 | Item **Quarantined**, note `not a PDF or over 20 MB`. | The file name does not end in `.pdf`, or the file is larger than 20 MB. | Queue viewer status and note; log line `item <id> -> Quarantined`. | Business | Ops | Look at the subject and sender. Signature logos and inline pictures (for example `image001.png`) also land here: ignore them. For a real referral, ask the sender to resend it as a PDF under 20 MB. The resend arrives as a new item. | The file really is a PDF under 20 MB and was still quarantined. Send: the item id and the file name. |
| E04 | Item **Done**, note `encrypted - not readable`, and the file is **not** in the shared intake folder. | The PDF is password-protected. The bot stops there and does not copy it, but it records the status as Done. | Queue viewer **note** column only. The status column alone looks like success. | Business | Ops | Ask the sender for a copy without a password. The new copy arrives as a new item. Check for these every day by filtering the note column on "encrypted". | Never needed for the item itself. The misleading status is on the dev list (section 13). |
| E05 | Item **Failed**, note `share unreachable: <reason>`. | Copying the file to the shared intake folder fails: folder offline, no permission, disk full, or a file with the same name is locked. | Queue viewer status and note; log line `item <id> -> Failed`. | System | Ops, then Dev | 1. Check that you can open the shared intake folder and that it has free space. 2. Look for the file there: a partial copy can be left behind. 3. The bot **never** retries a Failed item, so ask dev to put it back to New once the folder works again. **Don't** ask the sender to resend: that creates a duplicate. | Always, to re-queue it. Also escalate straight away if several items fail in a row. Send: the item ids, the reason text from the note, and the time. |
| E06 | The log stops completely: no new lines at all, not even warnings, although new mail with attachments is waiting. | The bot crashes at start-up because something it needs is missing: an entry in the settings file, the `data/` or `logs/` folder, or the queue file. It writes nothing to the log. | Nowhere ops can see. Only in Task Scheduler's "Last Run Result" for the task. | System | Dev | Check that the scheduled task is enabled (section 3). Send a small test PDF to the mailbox. If no `item` line appears within 20 minutes, escalate. | Always. Send: the time of the last log line, and the Task Scheduler "Last Run Result" if you can see it. |
| E07 | A sender says they sent a referral, but there is no item for it in the queue viewer. | No check. The bot only reads **unread** mail and marks it read straight away. A mail that someone opened first, a mail with no attachment, or a run that crashed while reading mail all leave no item and no log line. | Nowhere. | Business / System | Ops, then Dev | 1. If you can see the mailbox, find the mail. If it is there with an attachment, mark it **unread**: the next run picks it up. 2. If it has no attachment, ask the sender to attach the PDF. 3. Otherwise escalate. | The mail is unread with an attachment and still not queued after 20 minutes, or several referrals are missing. Send: the time sent and the subject (no patient details in chat). |
| E08 | Two items show **Done**, but the shared intake folder holds only one file, or the file doesn't match one of the senders. | No check. The bot saves files under the name from the email, so a second attachment with the same name overwrites the first, both in `data/tmp/` and in the shared intake folder. | Queue viewer (two items, same file name in the subject or sender context); shared intake folder. | System | Ops, then Dev | Ask the sender whose file was overwritten to resend it with a different file name. | Always. Send: both item ids and the file name. |

## 8. What the bot recovers by itself

| Problem | What the bot does | Limit | After the limit |
|---|---|---|---|
| Mailbox sign-in fails | Tries again after 30 seconds | 3 tries, about 90 seconds in total | The run stops silently. The next scheduled run tries again 10 minutes later, forever, with no alert (E01). |
| Run stopped before items were handled | New items left over are picked up | Next scheduled run | n/a |
| Unread mail not yet read | Stays unread in the mailbox | Until a run gets past sign-in | n/a |

The bot does **not** recover from any of these: an item stuck in Processing (E02), a Failed or Quarantined item (never retried), a crash at start-up (E06), or a missing referral (E07). Ops can wait out an E01 for about 30 minutes before escalating. For everything else, act as soon as you see it.

## 9. Known incidents

All three are recorded in the project history on 2026-10-23. Each was marked "fixed", but **the bot's behaviour did not change**: the fix records contain notes only. Treat all three as still open.

| When / version | What ops saw | Cause | What to do now | Ops could handle? |
|---|---|---|---|---|
| 2026-10-23 / no version | No new referrals all day. No errors except the repeating "IMAP connect failed" warnings. | The mailbox password had expired. The bot only writes warnings and never logs that it gave up. | E01: three runs in a row ending at `(3/3)`, or any login reason, → escalate for a password reset. **Still not fixed.** | Partly: the warnings are visible, but nothing alerts anyone. |
| 2026-10-23 / no version | Referrals showed **Done** but never reached the shared intake folder (password-protected PDFs). | Password-protected PDFs are recorded as Done with the note "encrypted - not readable", and not copied. | E04: filter the note column on "encrypted" every day and ask senders for an unlocked copy. **Still not fixed.** | Partly: possible only if ops reads the note, not the status. |
| 2026-10-23 / no version | Nothing reached the shared intake folder for a whole day. The log repeated "Previous item still Processing - aborting". | The bot crashed in the middle of one item. That item stayed in Processing and blocked every later run. Ops reset it by hand. | E02: escalate on the second error line. **Still not fixed.** | Partly: ops can spot it quickly with this doc, but releasing it needs dev (database). |

## 10. Releases and version check

- **How an update arrives:** there is no updater. A developer replaces the bot's files in its folder, and the next scheduled run uses the new files.
- **How to confirm it applied:** ops can't. The bot has no version number and doesn't write one to the log. Ask the developer to confirm in #rpa-support which change went in, and check that the next runs write normal `item <id> -> ...` lines.
- **If the behaviour didn't change after an update:** report it in #rpa-support with the time of the update and a log line that shows the old behaviour.
- Only one machine should run the bot's scheduled task. Two machines reading the same mailbox would split the mail between them.

## 11. Escalating to dev

Post in Teams **#rpa-support**. Send:
- the exception ID from section 7 (for example "E02")
- the item id(s) from the queue viewer
- the time of the first related log line
- the `logs/intake.log` file (attach it; don't paste lines that contain sender addresses)
- whether the file is in the shared intake folder

**Never send:** passwords, the `.env` settings file, patient details, attachments, or the contents of referrals in chat.

## 12. Failure-class check

- **F1** Missing prerequisite: covered in section 3 (settings file, folders, queue file, folder access, password expiry), E06, and section 13 items 5 and 13.
- **F2** Wrong window / focus: not applicable. The bot drives no desktop app or window; it only talks to the mailbox and the file share.
- **F3** Target slower or different than expected: covered in E01 (mailbox, 3 tries x 30 s) and E05 (shared folder). There are no screen timeouts. Section 13 item 14 covers the lack of time limits.
- **F4** Input read differently: covered in section 3 (accepted and quarantined examples, same-name files), section 4 (opening mail marks it read), E03, E07 and E08.
- **F5** Marked done when nothing was done: covered in E04 (Done without copying) and E07 (mail marked read with no item). Section 13 items 3 and 7.
- **F6** Action happened but not confirmed: covered in E02 (file may already be in the shared folder; check before re-queueing) and E05 (partial copy; don't ask for a resend).
- **F7** Business exception shown as success: covered in section 6 (Done - "encrypted - not readable" = needs follow-up), E04, and section 13 item 3.
- **F8** Endless re-runs / watchdog: covered in section 8 (mailbox retries forever, every 10 minutes, with no alert), E01 and E02. There is no watchdog. Section 13 items 1 and 4.
- **F9** Stopping the wrong way: covered in section 4 (disable the task; never end a run mid-item).
- **F10** One stuck item blocks the queue: covered in E02, section 4 and section 6 (Processing). Section 13 item 1.
- **F11** Cause hidden from ops: covered in E06 (crash cause only in Task Scheduler) and E07. Section 13 items 4, 5, 9, 10 and 12.
- **F12** Update didn't arrive: covered in section 10 (no version, no updater; one machine only). Section 13 item 11. The part about shared resources being rewritten on start is not applicable: the bot rewrites nothing shared on start.

## 13. Fix before handover (dev team)

| # | Gap | Why it hurts ops | Where (file:line) |
|---|---|---|---|
| 1 | One item left in `Processing` aborts every later run. There is no stale timeout, no auto-reset and no alert. Commit 4d3e78c only added a comment. | The bot stops for hours or days. Ops can't release it without the DB. | `intake.py:74-76`, `intake.py:78-80`, `intake.py:89` |
| 2 | `process()` has no handling for a damaged or unreadable PDF (`PdfReader`) or a missing temp file (`p.stat()`). The run crashes and leaves the row `Processing`, which causes #1. | Turns one bad attachment into a full outage. | `intake.py:59`, `intake.py:61` |
| 3 | An encrypted PDF returns `"Done"` without copying. It should be its own status (for example Quarantined or Failed). Commit 4beebcc only added a comment. | Undelivered referrals look like successes. | `intake.py:62-63`, `intake.py:90` |
| 4 | The final mailbox failure (`RuntimeError("Mailbox unreachable")`) is never caught or logged. A login failure is not told apart from a network failure. There is no alert. Commit 115f761 only added a comment. | Silent outage when the password expires (incident 1). | `intake.py:34-37`, `intake.py:77`, `intake.py:91` |
| 5 | Unhandled exceptions and tracebacks go to stderr only, not to `logs/intake.log`. A missing env var (`KeyError`), a missing folder, or a missing table crash with no log line. | The cause is visible only in Task Scheduler, which ops may not have access to (not confirmed). | `intake.py:23-24`, `intake.py:31-32`, `intake.py:73`, `intake.py:87-88` |
| 6 | Mail is fetched with `RFC822`, which marks it read on the server, but the rows are committed only after the whole loop. A crash during the fetch loses referrals with no trace. | Referrals vanish (E07). | `intake.py:44`, `intake.py:54` |
| 7 | Mail with no attachment, or mail read by a person first, leaves no record anywhere. | Ops can't prove whether a referral was received. | `intake.py:42`, `intake.py:46-49` |
| 8 | The attachment name is used as-is for `data/tmp` and the share. Same names overwrite each other, and the name is not sanitised (a path like `..\` can write outside the folder). | Lost or swapped referrals (E08); a security risk. | `intake.py:50`, `intake.py:65` |
| 9 | `Failed` and `Quarantined` are never retried, and re-queueing needs a DB edit that ops can't make. | Every share outage needs dev time (E05). Add a re-queue option in the viewer or a retry for `Failed`. | `intake.py:66-67`, `intake.py:78` |
| 10 | `queue_viewer.xlsx`, the ops screen named in the README, is not in the repo. Its columns and refresh source can't be checked. | Ops' only view of the queue is unversioned. | `README.md:6` |
| 11 | No version string anywhere, and none written to the log. | Ops can't confirm an update applied (F12). | `intake.py:1-2`, `intake.py:71` |
| 12 | No "run started / run finished" log line, and no count of mails read. | Ops can't tell a healthy quiet run from a dead bot. | `intake.py:71-84` |
| 13 | `.env` (mailbox credentials) and `logs/intake.log` are tracked in git since the initial import. The log contains a sender email address. Rotate the mailbox password, remove both files from history, and add them to `.gitignore`. | Credential and personal-data exposure. | `.env`, `logs/intake.log:6` |
| 14 | No timeout on the IMAP connection or on the share copy, beyond the OS defaults. | A hung run holds an item in `Processing` and causes #1. | `intake.py:31`, `intake.py:65` |
| 15 | No lock against overlapping runs. Two runs can pick up the same `New` rows if Task Scheduler starts one while another is still running. | Duplicate copies; confusing statuses. | `intake.py:74`, `intake.py:78` |
| 16 | Nothing creates `data/`, `data/tmp/`, `logs/` or the `queue` table. On a fresh machine, sqlite silently creates an empty `queue.db` and the run fails. | Setup on a new machine fails silently (E06). | `intake.py:17-18`, `intake.py:23`, `intake.py:73` |
| 17 | Every MIME part with a filename is queued, including inline signature images, which are then Quarantined. | Clutter that hides real quarantines. | `intake.py:46-53` |
| 18 | `data/tmp/` is never cleaned up. | The disk slowly fills with copies of referrals (personal data at rest). | `intake.py:50-51` |
| 19 | The share path falls back to a hard-coded default when `INTAKE_SHARE` is unset. | A misconfigured machine copies to an unexpected location. | `intake.py:19` |
| 20 | Confirm whether ops can open Task Scheduler (the task's history and the Disable option). Sections 4 and E06 depend on it. | Ops may not be able to stop the bot or see crash results. | n/a (access question) |