Skills/Rpa_Skills
8
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add rpa-bot-handoff skill: ops guide for bot handover

Browse Source 
The skill scans a bot with subagents (never runs it) and writes
HANDOVER.md, an 11-section ops guide in the operator's order: where is
what, how to run, how long to wait, after the run, how to see an
exception, exception list, how you will know, how to tell us, don'ts,
issues seen before. Dev material (check against 12 failure classes from
Tebra Auto Post, gaps with file:line, incidents) goes to HANDOVER-DEV.md.

Also adds ADR-0004, glossary terms, five fixture bots and a handoff eval
runner (--resume, --regrade) with results from 47 runs of the earlier
13-section version. The grader still expects that structure and must be
updated before the next eval run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
VasanthandClaude Opus 5.5 committed 2026-10-28 15:01:42 +05:30
1 parent 3d2db6aca6
commit 3942fad6cd
143 files changed
+11815 -2
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tests/evals/results/handoff/h-mail-intake-screens-r5.md
+158
View File
@@ -0,0 +1,158 @@
# Mail intake bot - ops handover
Version: none (the project has no version number; current code is commit 958759d) · Written 2026-10-23 · Escalation: raise a ticket in the **RPA Support** queue
## 1. What the bot does
Every 10 minutes the bot reads new (unread) mail in the referrals mailbox, saves every attachment, checks that it is a PDF of 20 MB or less, and copies each good PDF into the **shared intake folder**. It records every attachment in its own queue with a status and a short note. It drives no desktop apps or websites: it talks to the mailbox and the shared folder directly, so there are no windows to keep clear while it runs.
**What ops can see:** only the shared intake folder (the PDFs the bot copied there). The queue, its statuses and notes, and the log are not open to ops today (see section 13). In practice the one sign ops has is: **new referral PDFs stop appearing in the intake folder.**
## 2. How it works
```
Windows Task Scheduler (every 10 minutes)
└─ starts → Mail intake bot ──reads──> Referrals mailbox (unread mail)
│
├─ saves → bot's temporary folder (every attachment)
├─ copies → Shared intake folder (good PDFs) <- ops can open
├─ writes → Queue (one row per attachment: <- ops cannot open
│ sender, subject, file, status, note)
└─ writes → Bot log <- ops cannot open
```
- **Task Scheduler** starts the bot every 10 minutes. Nobody needs to click anything.
- **Referrals mailbox:** the bot reads only unread mail. Reading a mail marks it as read, so the bot never looks at it again.
- **Temporary folder:** every attachment is first saved here under its original file name.
- **Shared intake folder:** the bot's output. Each good PDF lands here under its original file name.
- **Queue:** the bot's record of every attachment and what happened to it.
- **Bot log:** one line per item processed, plus mailbox connection warnings.
**One run, step by step:**
1. Check the queue. If any item is still marked Processing from an earlier run, **stop the whole run** (see E02).
2. Sign in to the mailbox (up to 3 tries, 30 s apart).
3. For each unread mail: mark it read, save each attachment to the temporary folder, add one queue row per attachment as New.
4. For each New item, oldest first: mark it Processing, check it (PDF? 20 MB or less? password-protected?), copy it to the intake folder, then set the final status and note.
## 3. Before a run
The bot runs on its own; ops has nothing to start. These are the rules that decide whether a referral makes it through. Pass them on to referrers when a referral goes missing.
- [ ] **The referral must be an attachment**, not text or a link in the mail body. A mail with no attachment is read and then dropped with no record (E07).
- Bad: "Please see the referral below" with the details typed in the mail. Good: the referral attached as a `.pdf` file.
- [ ] **PDF only, 20 MB or less.** Word files, images, zip files and PDFs over 20 MB are set aside, not copied (E03).
- Bad: `referral.docx`, `scan.jpg`, a 35 MB PDF. Good: `referral.pdf` of 3 MB.
- [ ] **Not password-protected.** A password-protected PDF is marked Done but is **not** copied (E04).
- [ ] **A unique file name.** If two referrals arrive with the same file name (for example `scan.pdf`), the later one replaces the earlier one in the intake folder (E08).
- Bad: every referral named `referral.pdf`. Good: a name that includes the date or a reference, such as `referral_2026-10-23_A.pdf`.
- [ ] Set-up (Dev confirms once, at install): the settings file with the mailbox sign-in is next to the bot, the bot's data folder, temporary folder and queue exist, and the shared intake folder setting points to the right folder. Without these the bot stops at the start of every run (E10).
- [ ] The bot has no version number to check (see section 10).
## 4. During a run
- There is nothing to keep clear on screen: the bot drives no windows.
- **Do not stop the bot mid-run** (for example by ending its process in Task Manager). An item caught half-way stays marked Processing, and then **every later run stops at step 1** until a developer clears it (E02).
- To stop the bot: ops cannot. Raise a ticket asking Dev to **disable the scheduled task**; Dev waits for the current run to finish first.
- A run looks stuck when no new PDFs appear in the intake folder although referrals are expected. A normal run takes seconds to a couple of minutes. The bot gives up on the mailbox after about 1.5 minutes (3 tries x 30 s) and tries again at the next 10-minute start. **If no new PDF has appeared for 30 minutes during working hours while referrals are expected, raise a ticket.** The bot has no watchdog of its own for a run that hangs.
## 5. Where to look when something goes wrong
| Question | Where to look |
|---|---|
| Did the run finish? | Ops cannot see this. The only sign is new PDFs in the shared intake folder (check the file times). If none for 30 min while referrals are expected, raise a ticket. |
| Which items failed? | Ops cannot see this (the queue is not open to ops). Raise a ticket and ask Dev for queue items not Done. |
| Why did one item fail? | Ops cannot see this (the status note and the log are not open to ops). Ask Dev in the ticket, giving the referral's sender, subject and roughly when it was sent. |
| Did the bot already retry? | It retries only the mailbox sign-in (section 8). Nothing else is retried. A Failed or Quarantined item stays that way until Dev resets it. |
| Did a referral arrive at all? | Shared intake folder: look for its file name. Not there means it was set aside (E03, E04), failed (E05), dropped (E07, E09), replaced (E08) or the bot is blocked (E01, E02, E06, E10). |
| Which version is running? | No version number (section 10). |
## 6. Statuses
The statuses live in the bot's queue. The bot has no screen, so there are no colours; ops sees them only when Dev reads them out (section 13, item 2).
| Status | Colour | Meaning | What ops does |
|---|---|---|---|
| New | none (no screen) | Attachment saved from the mailbox, waiting to be checked. Picked up later in the same run. | nothing |
| Processing | none (no screen) | The bot is working on this item right now. **If it stays Processing after the run, every later run stops** (E02). | If no new PDFs for 30 min, raise a ticket (E02) |
| Done - "<n> pages saved" | none (no screen) | PDF copied to the intake folder. | nothing |
| Done - "encrypted - not readable" | none (no screen) | **Needs follow-up. Not copied to the intake folder** although it says Done: the PDF is password-protected. | Ask the sender for a copy without a password (E04) |
| Quarantined - "not a PDF or over 20 MB" | none (no screen) | Not a PDF, or larger than 20 MB. Not copied. Never retried. | Ask the sender to resend as a PDF of 20 MB or less (E03) |
| Failed - "share unreachable: <reason>" | none (no screen) | The intake folder could not be reached when copying. Not copied. **Never retried by the bot.** | Raise a ticket so Dev re-queues it once the folder is back (E05) |
## 7. Exception catalogue
Almost every problem looks the same to ops: **a referral PDF does not appear in the intake folder.** Quote the ID in your ticket when you can tell which one it is; if you can't, say "missing referral" and give the sender, subject and time sent.
| ID | What you see | How the bot identifies it | Where it is recorded | Type | Who acts | What to do | Escalate when |
|---|---|---|---|---|---|---|---|
| E01 | No new PDFs for every run since some point; referrers say they sent mail | It tries to sign in to the mailbox 3 times, 30 s apart. If all 3 fail (wrong or expired password, mailbox or network down) the run ends. It tries again 10 minutes later. | Bot log: "IMAP connect failed (1/3)…(3/3)" with the reason. Not visible to ops. | System | Bot, then Dev | Wait 30 minutes; the bot retries every 10 minutes by itself. | Still no new PDFs after 30 minutes. Send: time the PDFs stopped, and ask Dev to check the mailbox password (it has expired before). |
| E02 | No new PDFs at all, for hours, and it never recovers by itself | At the start of every run it checks for any item still marked Processing. If it finds one it stops the run before reading any mail. This happens after an earlier run was cut off half-way (crash, stopped by hand, machine restart). | Bot log: "Previous item still Processing - aborting", every 10 minutes. Not visible to ops. | System | Dev | Raise a ticket at once. The bot **never** clears this by itself. New mail waits unread in the mailbox meanwhile and is picked up once it is cleared. | Always. Send: time the PDFs stopped. Ask Dev to check the stuck item in the intake folder and in the target system before resetting it. |
| E03 | One referral missing from the intake folder; other referrals arrive normally | It checks the file name ends in ".pdf" and the size is 20 MB or less. | Queue: Quarantined, note "not a PDF or over 20 MB". Not visible to ops. | Business | Ops | Ask the sender to resend as a PDF of 20 MB or less (split large scans). The resent mail is picked up automatically. | The sender says it was already a PDF under 20 MB. Send: sender, subject, time sent. |
| E04 | One referral missing from the intake folder; the sender says it was a protected PDF | It opens the PDF and checks whether it is password-protected. If so it stops there and does **not** copy it, but marks it Done. | Queue: Done, note "encrypted - not readable". Not visible to ops. | Business | Ops | Ask the sender for a copy without a password. | The sender's copy has no password. Send: sender, subject, time sent. |
| E05 | Several referrals missing from about the same time; the intake folder itself was unavailable or full | The copy to the intake folder fails. | Queue: Failed, note "share unreachable: <reason>". Bot log line for that item. Not visible to ops. | System | Dev | Raise a ticket once the intake folder works again. The bot does **not** retry Failed items; Dev must re-queue them. | Always. Send: the time window when the folder was down. |
| E06 | No new PDFs at all, starting right after one particular referral arrived | A damaged PDF (or a file that vanished from the temporary folder) makes the bot crash while checking it. The item is left as Processing, which then triggers E02 on every later run. | Item stays Processing. The crash reason is **not** written to the log. Not visible to ops. | System | Dev | Raise a ticket as for E02. Then ask the sender for a fresh copy of that referral. | Always. Send: time the PDFs stopped, and the last referral you know arrived. |
| E07 | Referral missing; the sender put the details in the mail body or sent a link | Mails with no attachment are read and skipped. No queue row is made. | Nowhere. The mail is only marked read. | Business | Ops | Ask the sender to resend with the referral attached as a PDF. | The sender is sure there was an attachment. Send: sender, subject, time sent. |
| E08 | A PDF in the intake folder holds a different referral than its name suggests, or one of two referrals is missing | Files keep the sender's file name. A second file with the same name replaces the first, in the temporary folder and in the intake folder. | Nowhere as an error; both queue rows look normal. | Business | Ops, then Dev | Ask senders to use unique file names. Ask the sender of the lost referral to resend it with a different name. | Always tell Dev (they can check which referral was replaced). Send: the file name and both senders. |
| E09 | Referrals from a short window missing, though nothing else looks wrong | If the bot crashes while saving attachments, the mails it already read stay marked read but get no queue rows. They are never picked up again. | Nowhere. Not in the queue, not in the log. | System | Dev | Raise a ticket. Dev must find these mails in the mailbox and mark them unread. | Always. Send: the time window and any known senders. |
| E10 | No new PDFs ever, starting right after the bot was installed or moved | The settings file, the bot's data folder, temporary folder or queue is missing, so the bot stops at the start. | The reason is **not** written to the log. Not visible to ops. | System | Dev | Raise a ticket. | Always. Send: when the bot was installed or moved. |
## 8. What the bot recovers by itself
| Problem | What the bot does | Limit | After the limit |
|---|---|---|---|
| Mailbox sign-in fails | Tries again after 30 s | 3 tries (about 1.5 minutes) | Run ends. Task Scheduler starts a fresh run 10 minutes later, forever. Raise a ticket after 30 minutes without new PDFs. |
| Anything else (folder down, bad file, stuck item) | Nothing | - | Item stays Failed / Quarantined / Processing until Dev acts. A stuck Processing item stops every run (E02). |
## 9. Known incidents
Newest first. Note: the three fix records in the project history changed **only a comment line**, not the bot's behaviour (see section 13, item 1). Treat these problems as still possible.
| When / version | What ops saw | Cause | What to do now | Ops could handle? |
|---|---|---|---|---|
| commit 958759d | Every run failed with no sign anywhere; no new PDFs | The mailbox password expired | The sign-in error goes to the bot log, which ops can't open. Treat "no new PDFs for 30 min" as E01 and raise a ticket. | Partly: ops can spot it and escalate; only Dev can fix the password |
| commit 5b6f24d | Password-protected referrals arrived in the intake folder as empty files | The bot copied PDFs it could not read | Now marked Done "encrypted - not readable" and **not copied**. Ask the sender for an unprotected copy (E04). | Partly: ops acts once Dev tells them which referral |
| commit e59f92b | Nothing came through for a whole day | A crash left one item Processing; every run then stopped (E02). Ops reset it by hand. | Still happens. Ops no longer has queue access, so raise a ticket at once (E02). | Partly: ops can spot it within 30 minutes; only Dev can clear it |
## 10. Releases and version check
- How an update arrives: there is no updater. A developer replaces the bot's files on the machine.
- How to confirm it applied: the bot has no version number and no screen, so ops cannot. Ask Dev to confirm in the ticket which commit is running.
- If the version did not change: ask Dev in the RPA Support queue.
## 11. Escalating to dev
Raise a ticket in the **RPA Support** queue. Send: the exception ID from section 7 (or "missing referral"), the time new PDFs stopped or the time the referral was sent, the sender and subject of the affected mail, and the file name if you know it.
Never send: passwords, the settings file, the referral PDFs themselves, or patient details in the ticket text.
## 12. Failure-class check
- F1 covered in section 3 (set-up item) and E10; gaps in section 13, items 10 and 13.
- F2 not applicable: the bot drives no windows; it talks to the mailbox and the shared folder directly.
- F3 covered in E01 and section 8 (mailbox sign-in timing); no time limit on a hanging mailbox connection in section 13, item 9. No screens, so screen-loading checks do not apply.
- F4 covered in section 3 input rules and E03, E07, E08.
- F5 covered in E04 and E09; section 13, items 4 and 7.
- F6 covered in E09 (mail marked read before the bot recorded it); section 13, item 7.
- F7 covered in section 6 and E04 ("Done" for a protected PDF that was not copied); section 13, item 4.
- F8 covered in E02 (the same "aborting" stop repeats every 10 minutes forever) and section 8.
- F9 covered in section 4 (never stop it mid-run; ask Dev to disable the scheduled task).
- F10 covered in E02 and E06; section 13, items 1 and 5.
- F11 covered in section 5 (ops sees only the intake folder); section 13, items 2 and 3.
- F12 covered in section 10 and section 9 note (fix records did not change behaviour); section 13, items 1 and 11.
## 13. Fix before handover (dev team)
| # | Gap | Why it hurts ops | Where (file:line) |
|---|---|---|---|
| 1 | The three "fix" commits (e59f92b, 5b6f24d, 958759d) each add only a comment line; no code changed. The stuck-Processing abort is unchanged. | Ops and dev believe these incidents are fixed; E02 can still block a whole day. | `intake.py:89-91`; abort at `intake.py:74-76` |
| 2 | No ops view of the queue. README says ops use `queue_viewer.xlsx`, but it is not in the repo and nothing creates it; ops have no database access. | Ops can't see any status or note, so every exception becomes a Dev ticket. | `README.md:6`; queue at `intake.py:17` |
| 3 | Crashes are not logged: "Mailbox unreachable" and PDF read errors are uncaught and go to stderr only; and the log folder is not open to ops anyway. No alert of any kind. | E06/E10 causes are invisible to everyone; ops learns only from missing PDFs. | `intake.py:37`, `intake.py:61`, `intake.py:77`; logging `intake.py:23-24` |
| 4 | Password-protected PDF is stored as Done although it was not copied. | Reads as success; referral silently missing. | `intake.py:62-63` |
| 5 | A damaged PDF or missing temp file crashes the run and leaves the item Processing, which blocks every later run. No try/except per item, no stale-Processing reset. | One bad attachment stops all intake until Dev steps in. | `intake.py:59`, `intake.py:61`, `intake.py:79-82`, `intake.py:74` |
| 6 | Failed items are never retried (only New is picked up). | A short share outage loses referrals until Dev re-queues by hand. | `intake.py:66-67`, `intake.py:78` |
| 7 | Mails are marked read during fetch, but queue rows are committed only after all mails; a crash in between loses those referrals. | Silent referral loss (E09). | `intake.py:42-54` |
| 8 | Attachment file names are used as-is: same name overwrites in temp and in the share; names are not sanitised. | Referrals silently replaced (E08); unsafe file names reach the share. | `intake.py:50-51`, `intake.py:65` |
| 9 | No timeout on the mailbox connection. | A hung connection can stall the bot with no sign. | `intake.py:31` |
| 10 | Mail with no attachment is dropped with no record. | Ops can't tell "never sent" from "dropped". | `intake.py:47-49` |
| 11 | No version string anywhere. | Nobody can confirm which build is running. | `intake.py:1-2` |
| 12 | `.env` (mailbox credentials) and `logs/intake.log` are committed to git; the log contains a sender email address. | Credential and personal-data exposure; rotate the password and remove both from history. | `.env`, `logs/intake.log:6` |
| 13 | Data folder, temp folder and queue table are never created; missing settings stop the bot at start with no clear message. | A fresh install or move fails silently (E10). | `intake.py:17-18`, `intake.py:31-32`, `intake.py:73` |