Skills/Rpa_Skills
8
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add rpa-bot-handoff skill: ops guide for bot handover

Browse Source 
The skill scans a bot with subagents (never runs it) and writes
HANDOVER.md, an 11-section ops guide in the operator's order: where is
what, how to run, how long to wait, after the run, how to see an
exception, exception list, how you will know, how to tell us, don'ts,
issues seen before. Dev material (check against 12 failure classes from
Tebra Auto Post, gaps with file:line, incidents) goes to HANDOVER-DEV.md.

Also adds ADR-0004, glossary terms, five fixture bots and a handoff eval
runner (--resume, --regrade) with results from 47 runs of the earlier
13-section version. The grader still expects that structure and must be
updated before the next eval run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
VasanthandClaude Opus 5.5 committed 2026-10-28 15:01:42 +05:30
1 parent 3d2db6aca6
commit 3942fad6cd
143 files changed
+11815 -2
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tests/evals/results/handoff/h-mail-intake-screens-r3.md
+162
View File
@@ -0,0 +1,162 @@
# Mail intake bot - ops handover
Version: no version number (repo commit 6a62717) · Written 2026-10-23 · Escalation: raise a ticket in the **RPA Support** queue
## 1. What the bot does
Every 10 minutes the bot reads new (unread) emails in the referrals mailbox and saves each email attachment. It checks that each attachment is a PDF under 20 MB and not password-protected. It copies good PDFs to the shared intake folder (the output folder). Each attachment gets one row in the intake queue, with a status and a note. Ops see the queue in the **queue viewer Excel**, which shows id, sender, subject, status and note; click Data > Refresh to update it. The bot has no window of its own and does not drive any app on screen. It runs in the background on a schedule.
## 2. How it works
```
Windows Task Scheduler (every 10 min)
└─ starts → Mail intake bot (background, no window)
├─ reads → referrals mailbox (unread mail only)
├─ saves → temporary folder on the bot machine
├─ copies → shared intake folder (output folder - ops can open)
├─ writes → intake queue (one row per attachment)
│ └─ shown in → queue viewer Excel (ops can open)
└─ writes → bot log file (ops cannot open)
```
- **Task Scheduler** starts the bot every 10 minutes. Nobody clicks anything.
- **The mail intake bot** does all the work and closes when finished.
- **The referrals mailbox** is where referrals arrive. When the bot reads an email, the email is marked as read.
- **The temporary folder** holds every attachment the bot saved. It is never emptied.
- **The shared intake folder** is where finished PDFs go. A PDF is "delivered" only when it is here.
- **The intake queue** has one row per attachment with a status and a note. Ops see it through the queue viewer Excel.
- **The bot log** records each item's result and mailbox warnings. Only dev can open it.
**One run, step by step:**
1. The bot checks the queue. If any row is still **Processing** from an earlier run, it stops at once and does nothing else (see E04).
2. It signs in to the mailbox. If that fails it tries 3 times, waiting 30 s each time.
3. It reads every unread email, saves every attachment, and adds one **New** row per attachment.
4. For each New row, oldest first, it sets the row to **Processing**, checks the file, copies it to the shared intake folder and writes the final status and note.
5. The run ends. The next run starts 10 minutes later.
## 3. Before a run
The bot runs by itself, so these are things to keep true, not steps to do each time.
- [ ] The Task Scheduler task is enabled on the bot machine (dev/IT).
- [ ] The mailbox password in the bot's settings has not expired (dev/IT). An expired password stops all intake (E05).
- [ ] The shared intake folder is reachable from the bot machine. Check by opening it yourself.
- [ ] The bot's settings file, queue, temporary folder and log folder exist on the bot machine (dev). The bot does not create them, and if one is missing, nothing happens (E05).
- [ ] Rules for senders' attachments:
- **PDF only.** Bad: `referral.docx`, `scan.jpg`. Good: `referral.pdf` (`.PDF` in capitals is fine too).
- **Under 20 MB.** Bad: a 35 MB scan. Good: the same scan split into two files.
- **Not password-protected.** Bad: a PDF that asks for a password when opened. Good: the same PDF saved without a password.
- **A unique file name.** Bad: two referrals both called `scan.pdf`, since the second one overwrites the first (E07). Good: `scan_smith_1.pdf`, `scan_smith_2.pdf`.
- [ ] There is no version check. See section 10.
## 4. During a run
- Ops do not need to stay away from the machine. The bot has no window and clicks nothing.
- Do not rename, move or delete files in the shared intake folder while they are still New or Processing in the queue viewer.
- **To stop the bot:** ask dev/IT (RPA Support ticket) to disable the Task Scheduler task. Do not end the bot's process in the middle of a run. If it is stopped while an item is Processing, that item stays Processing and blocks every later run (E04).
- **A run looks stuck when** one row shows **Processing** on two refreshes of the queue viewer at least 15 minutes apart. Normally an item is in Processing for a few seconds only. **The bot has no watchdog and never frees a stuck item by itself.** Don't wait it out. Raise a ticket (E04).
## 5. Where to look when something goes wrong
Ops can open only the queue viewer Excel and the shared intake folder. Everything else (bot log, queue database, Task Scheduler history, temporary folder) is dev-only.
| Question | Where to look |
|---|---|
| Is the bot running at all? | Queue viewer → Data > Refresh. New rows should appear within about 10 minutes of a referral email. If none appear for 30 minutes, see E05. |
| Is the bot blocked? | Queue viewer → any row with status **Processing** that stays that way for 15+ minutes (E04). |
| Which items failed? | Queue viewer → filter status = **Failed** or **Quarantined**, and status = **Done** with note "encrypted - not readable". |
| Why did one item fail? | Queue viewer → the **note** column for that row. Then look up the exception in section 7. |
| Did the file really arrive? | Shared intake folder → look for the file name. **Done is not proof by itself.** An encrypted PDF is marked Done but is not copied (E02). |
| Did the bot already retry? | Only the mailbox sign-in is retried, and you can't see it (dev log only). Nothing else is ever retried. |
| Which version is running? | You can't tell (section 10). |
## 6. Statuses
The queue viewer shows statuses as plain text. The bot sets no colours. If the Excel adds colour, that is a formatting rule inside the Excel and not something this document can confirm.
| Status | Note | Meaning | What ops does |
|---|---|---|---|
| New | (empty) | Attachment saved, waiting to be checked in this or the next run | nothing |
| Processing | (empty) | Being checked right now. Normal for a few seconds. | If it stays for 15+ minutes, raise a ticket (E04) |
| Done | "<n> pages saved" | PDF copied to the shared intake folder | nothing |
| Done | "encrypted - not readable" | **Not delivered - needs follow-up.** Password-protected PDF, **not** copied to the shared intake folder. | Ask the sender for an unlocked copy (E02) |
| Quarantined | "not a PDF or over 20 MB" | **Not delivered.** Wrong file type or too big. The note does not say which. | Check the attachment and ask the sender to resend (E01) |
| Failed | "share unreachable: ..." | **Not delivered.** Could not copy to the shared intake folder. **Never retried.** | Check the shared intake folder, then raise a ticket (E03) |
## 7. Exception catalogue
| ID | What you see | How the bot identifies it | Where it is recorded | Type | Who acts | What to do | Escalate when |
|---|---|---|---|---|---|---|---|
| E01 | Quarantined - "not a PDF or over 20 MB" | The file name does not end in .pdf, or the file is bigger than 20 MB | Queue viewer status and note; dev log | Business | Ops | Look at the file name in the row. If it is a logo or signature image from the email (for example `image001.png`), ignore the row. Otherwise ask the sender to resend as a PDF under 20 MB. The resent email gets a new row. | A real PDF under 20 MB is quarantined |
| E02 | Done - "encrypted - not readable", and the file is not in the shared intake folder | The PDF is password-protected. The bot skips the copy but still marks the row Done. | Queue viewer note; dev log | Business | Ops | Filter for this note every day; the Done status hides it. Ask the sender for a copy without a password. | Never, unless the sender says the file has no password |
| E03 | Failed - "share unreachable: ..." | Copying the PDF to the shared intake folder gave an error | Queue viewer status and note; dev log | System | Ops, then Dev | 1. Open the shared intake folder. If you can't, the folder or network is down; tell IT. 2. If a file with that name is there, it may be a half-copied file; don't trust it. 3. The bot **never** retries Failed rows. Raise a ticket so dev can put the row back to New once the folder works. | Always (only dev can re-queue the row). Send: item id, time of the row, "E03". |
| E04 | One row stuck on **Processing**, and no new rows appear at all. Every referral since then is waiting. | At the start of every run the bot checks for any Processing row. If it finds one, it stops at once. This happens when the bot crashed on that item (for example a damaged PDF) or was stopped mid-item. | Queue viewer (the stuck row); dev log "Previous item still Processing - aborting" every 10 minutes | System | Dev | Raise a ticket at once. The bot will **never** free itself. Intake is stopped until dev resets the row. No mail is lost: emails stay unread in the mailbox and are picked up after the reset. | Always, as soon as it is seen. Send: the stuck item's id, the time you first saw it, "E04". |
| E05 | No new rows appear for 30+ minutes although referrals were emailed, and no row is stuck in Processing | The bot cannot sign in to the mailbox (expired password, network or mailbox problem). It tries 3 times, 30 s apart, then gives up until the next run. The same symptom appears if the bot's settings, queue, temporary folder or log folder is missing. | Nowhere ops can see. The dev log shows "IMAP connect failed" warnings. The final failure is not in the log at all; it is only in Task Scheduler history. | System | Bot, then Dev | Wait 30 minutes (3 runs). Then raise a ticket. Mention whether the mailbox password is due to expire, if you know. | No new rows for 30 minutes during business hours. Send: time of the last new row, time the missing referral was sent, "E05". |
| E06 | A sender says they emailed a referral, the email is already marked read in the mailbox, but there is no row for it | The bot crashed while saving attachments, after the emails were marked read but before the rows were saved. The next run only looks at unread mail, so it skips them. | Nowhere. No row and no log line. | System | Dev | Raise a ticket. Until it is fixed, ask the sender to resend, or have someone with mailbox access mark the email unread again. | Always. Send: approximate time the email was sent, "E06". Don't paste the sender or subject into chat. |
| E07 | Two rows are Done with "<n> pages saved", but the shared intake folder holds only one file, or a file has the wrong content | Two attachments had the same file name. The second overwrote the first, both in the temporary folder and in the shared intake folder. The bot does not check for duplicate names. | Queue viewer shows two rows with the same file name; nothing flags it | Business | Ops, then Dev | Ask the senders to resend with unique file names. Raise a ticket so dev knows it happened. | Always. Send: both item ids, "E07". |
## 8. What the bot recovers by itself
| Problem | What the bot does | Limit | After the limit |
|---|---|---|---|
| Mailbox sign-in fails | Tries again after 30 s | 3 tries per run (about 1.5 minutes) | The run ends with no new rows. The next run in 10 minutes tries again. |
| Rows left as New at the end of a run | Picked up by the next run | Every 10 minutes | Nothing more is needed |
| Anything else (Failed rows, stuck Processing, lost attachments) | **Nothing.** No retry, no watchdog, no restart. | - | Needs dev (E03, E04, E06) |
So the longest ops should ever wait before acting is **30 minutes** (E05). For a stuck Processing row (E04), don't wait.
## 9. Known incidents
Newest first. All three were logged as fixes, but **none of them changed how the bot behaves**, so each can happen again.
| When / version | What ops saw | Cause | What to do now | Ops could handle? |
|---|---|---|---|---|
| commit 6a62717 | No new referrals came in, and nothing obvious showed anywhere | The mailbox password expired. Every run failed to sign in and the final failure was not recorded anywhere ops or dev would look. | E05: no new rows for 30 minutes → ticket. Dev renews the password. | Yes, with E05 |
| commit 72c7f55 | PDFs in the shared intake folder were empty or unreadable | Password-protected PDFs were copied as if fine | They are no longer copied, but they are marked **Done**. Filter for the note "encrypted - not readable" (E02). | Partly. The Done status hides them. |
| commit a45615d | No referrals processed for a whole day | One item crashed mid-processing and stayed Processing. Every later run stopped at the start. Someone reset the row by hand. | E04: raise a ticket at once. Still happens on any damaged PDF. | Partly. Ops can spot it, but only dev can reset the row. |
## 10. Releases and version check
- **How an update arrives:** there is no updater. Dev copies the new code onto the bot machine.
- **How to confirm it applied:** ops can't. The bot has no version number and no screen. Ask dev to confirm in the ticket that the new code is in place on the bot machine.
- **If the version did not change:** raise a ticket in RPA Support. There is evidence that the bot on the machine has at some point been different from the code in the repo (Fix before handover #13).
## 11. Escalating to dev
Raise a ticket in the **RPA Support** queue. Send: the exception ID from section 7, the item id(s) from the queue viewer, the time you first saw the problem, and the time of the last row that looked normal.
Never send: passwords, the bot's settings file, or the sender, subject or file content of a referral (patient data). Item ids are enough.
## 12. Failure-class check
- F1 (missing prerequisite) is covered in section 3 and E05, plus Fix #8 and #12.
- F2 (wrong window or focus) is not applicable: the bot drives no app on screen and uses no windows.
- F3 (target slower or different than expected) is covered in E05 (mailbox) and E03 (shared folder). Mailbox sign-in is the only wait the bot has.
- F4 (input read differently than written) is covered in section 3 attachment rules, E01 and E07, plus Fix #6 and #7.
- F5 (marked done when nothing was done) is covered in E02 and E07, plus Fix #3.
- F6 (action happened but not confirmed) is covered in E03 step 2: a failed copy may leave a partial file on the share.
- F7 (business exception shown as success) is covered in section 6 (Done / "encrypted"), E02, and Fix #3.
- F8 (endless re-runs or watchdog kill) is covered in E04 and section 8. The bot has no watchdog; a stuck row makes it stop at the start of every run, forever.
- F9 (stopping the bot the wrong way) is covered in section 4: disable the scheduled task, and never kill a run mid-item.
- F10 (one stuck item blocks the queue) is covered in E04 and Fix #1.
- F11 (cause hidden from ops) is covered in section 5, plus Fix #2, #7, #9 and #10.
- F12 (update didn't arrive or old build) is covered in section 10, plus Fix #11 and #13.
## 13. Fix before handover (dev team)
| # | Gap | Why it hurts ops | Where (file:line) |
|---|---|---|---|
| 1 | Any error inside one item (damaged PDF, missing temp file) is not caught, so the row stays Processing. Every later run then aborts, and there is no stale-row timeout. | One bad file stops all intake until dev resets it by hand (E04, incident a45615d) | `intake.py:59`, `:61`, `:68`, `:81`; abort at `:74-76` |
| 2 | The final "Mailbox unreachable" error is never logged. Login and network errors are not told apart. There is no alert. | Expired password = silent outage (E05, incident 6a62717) | `intake.py:34-37`, `:77` |
| 3 | An encrypted PDF is stored as **Done** although it was not delivered | Ops treats it as finished (F7) | `intake.py:62-63` |
| 4 | Failed rows are never retried, and ops can't re-queue them | Every share outage needs a dev ticket per item | `intake.py:66-67`, `:78` |
| 5 | Mail is marked read before the queue rows are saved, and rows are committed once after all messages | A crash while saving attachments loses referrals for good (E06) | `intake.py:44`, `:54` |
| 6 | Attachment file names are used as paths without being cleaned or checked for duplicates | Same name overwrites another referral (E07); a crafted name could write outside the temp folder | `intake.py:50`, `:65` |
| 7 | The quarantine note doesn't say whether the file was the wrong type or too big, and every named email part (logos, signatures) gets queued | Ops must guess, and their list fills with noise rows | `intake.py:48-49`, `:59-60` |
| 8 | The code never creates the temp folder, log folder or queue table | A fresh or rebuilt machine fails silently (E05) | `intake.py:18`, `:23`, `:73` |
| 9 | Crashes go only to Task Scheduler history, not the log. Failed items are logged at INFO level. Ops can't open the log anyway. | Causes are invisible to ops (F11) | `intake.py:23-24`, `:84` |
| 10 | The queue viewer Excel, the only screen ops has, is not in the repo | It can't be checked or rebuilt; its colours and columns are unknown | `README.md:6` |
| 11 | No version number anywhere and no update process | Nobody can confirm which code is running | `intake.py` (whole file) |
| 12 | The settings file with the mailbox credentials is committed to git, and the shared-folder path is hard-coded as a fallback | Credential leak; rotate the mailbox password and remove the file from history | `.env`, `intake.py:19` |
| 13 | The log contains a line format the current code does not write, and that line includes a sender's email address | The running bot may differ from the repo; personal data is in the log | `logs/intake.log:6` |
| 14 | The three "fix" commits add only comments | The incidents in section 9 are not fixed | `intake.py:89-91` |
| 15 | The temporary folder is never emptied | Patient PDFs pile up on the bot machine; disk fills | `intake.py:50-51` |