read-trans: Tanglish explainer for YouTube transcripts, with eval

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

tests/evals/read_trans/README.md

@@ -0,0 +1,9 @@
+# read-trans eval
+
+- `fixture_make_it_better.txt`: short transcript with timestamps, chapter markers and a "like and subscribe" plug.
+- `results/make_it_better.md`: output of a fresh subagent that followed only `skills/read-trans/SKILL.md`.
+- `check_output.py <output.md>`: PASS/FAIL on structure (4 headings, a prompt block), Tanglish (no Tamil script, enough Tanglish words), no leaked timestamps/chapters/plugs, costs kept, under 700 words.
+
+Run: `python tests/evals/read_trans/check_output.py tests/evals/read_trans/results/make_it_better.md`
+
+History: run 1 passed the check but put its own arithmetic in the takeaway and did not mark its own ideas. SKILL.md was tightened (speaker-only sections 1-3, own ideas marked `(En idea)`); run 2 is the stored result.

tests/evals/read_trans/check_output.py

@@ -0,0 +1,24 @@
+"""Check a read-trans output file: python check_output.py output.md  (exit 1 on failure)."""
+import re, sys
+
+text = open(sys.argv[1], encoding="utf-8").read()
+low = text.lower()
+fails = []
+
+def need(ok, msg):
+    if not ok:
+        fails.append(msg)
+
+for h in ("video enna solluthu", "medium explanation", "final takeaway", "projects-ku enna pannanum"):
+    need(h in low, f"missing section: {h}")
+need(not re.search(r"[\u0b80-\u0bff]", text), "Tamil script found (should be Tanglish)")
+need(not re.search(r"\b\d+:\d{2}\b", text), "timestamp leaked into output")
+need(not re.search(r"chapter \d", low), "chapter marker leaked")
+need("```" in text, "no ready-to-copy prompt block in the projects section")
+need(len(re.findall(r"\b(irukku|panna|pannanum|solluthu|romba|idhu|athu|aana|illa)\b", low)) >= 8, "not enough Tanglish words")
+need("subscribe" not in low, "sponsor/subscribe plug repeated")
+need("$137" in text and "$22" in text, "model costs missing")
+need(len(text.split()) < 700, "too long (>700 words)")
+
+print("\n".join(["FAIL: " + f for f in fails]) or "PASS")
+sys.exit(1 if fails else 0)

tests/evals/read_trans/fixture_make_it_better.txt

@@ -0,0 +1,15 @@
+0:00 0 seconds Today I gave Claude Code a horrible coffee shop website and told it make it better, 100 times in a row.
+0:15 15 seconds I did this with Sonnet 5.5 on medium effort and Opus 5.5 on medium effort.
+Chapter 2: The prompt
+0:25 25 seconds The prompt was: look at the site first, then innovate. Think about user experience, think outside the box. Make one bold change. You must ship a change every time.
+0:50 50 seconds I had to force it to change every time because earlier it stopped at round 20 or 25 and said nothing else can be done.
+Chapter 3: Setup
+1:20 1 minute, 20 seconds Every round ran in a clean room: fresh session, no skills, no CLAUDE.md, no plugins, no memory. Only the site code carried over from the last round.
+Chapter 4: Sonnet
+2:10 2 minutes, 10 seconds Sonnet cost about $22 and ran almost 2 hours. Early rounds were good: sticky order bar, receipt style order, an AI video, an order form. After round 32 there were no big milestones. Lines of code kept rising linearly but quality did not.
+Chapter 5: Opus
+7:40 7 minutes, 40 seconds Opus cost $137 and ran 8.5 hours. Round 1 was already better. It added a QR code that texts your order, and a mood-based drink picker. By round 100 it had 21,000 lines: leaves, pumpkin carving, caffeine tracker, music, a Spanish toggle that did not work. Some sections broke. It was AI slop.
+Chapter 6: Surprises
+12:30 12 minutes, 30 seconds Neither model used web search, even though I said innovate. They never looked at award winning sites for inspiration.
+Chapter 7: Final thoughts
+13:15 13 minutes, 15 seconds The model sets the ceiling and the prompt sets the direction. Innovate without a vision turns into adding. It tried to upsell everything, a hard sell instead of a soft close. Please like and subscribe.

tests/evals/read_trans/results/make_it_better.md

@@ -0,0 +1,31 @@
+### 1. Video enna solluthu (short)
+Speaker oru horrible coffee shop website-ai Claude Code kitta "make it better" nu 100 times sonnaar, Sonnet 5.5 and Opus 5.5 (rendum medium effort) la.
+Opus better ideas kudutthuchu aana 100 rounds la AI slop aayiduchu; Sonnet cheap ah irunthuchu aana 32 round-ku apram quality nikkala.
+
+### 2. Medium explanation
+1. Setup: oru mosamaana coffee shop website. Prompt: "look at the site first, then innovate, think about user experience, think outside the box, make one bold change, you must ship a change every time." Idhu 100 times loop.
+2. "You must ship a change" nu force pannaan, because munnadi round 20 or 25 la model "innum onnum panna mudiyaadhu" nu niruthiduchu.
+3. Every round clean room: fresh session, no skills, no CLAUDE.md, no plugins, no memory. Site code mattum next round-ku poachu.
+4. Sonnet: about $22, almost 2 hours. Early rounds nalla irunthuchu: sticky order bar, receipt style order, AI video, order form. Round 32-ku apram big milestone illa. Lines of code linear ah ericchu, aana quality ericchila.
+5. Opus: $137, 8.5 hours. Round 1 la ye better. QR code (scan pannaa order text aagum), mood-based drink picker. Round 100 la 21,000 lines: leaves, pumpkin carving, caffeine tracker, music, Spanish toggle (adhu work aagala). Sila sections broke. Speaker: "AI slop".
+6. Surprise: rendu model-um web search use pannala, "innovate" nu sonnaalum. Award winning sites paathu inspiration edukkala.
+
+### 3. Final takeaway
+- Model dhaan ceiling set pannum; prompt dhaan direction set pannum.
+- Vision illaama "innovate" nu sonna, model add pannikite irukkum, improve pannaadhu.
+- Model ellathayum upsell panna paathuchu: hard sell, soft close illa.
+- Repeat panna panna lines of code ericchu, quality ericchila (Sonnet 32 round-ku apram).
+- Model ah force pannaama vittaa round 20-25 la niruthidum.
+
+### 4. Unga projects-ku enna pannanum
+1. "Make it better" maari vague prompt kudukaadhe. Enna improve venum nu direction sollu (vision, goal, target user).
+2. Loop la run pannum bodhu "look at the site/code first, then make one bold change" nu one change per round nu limit pannu.
+3. Model web search use pannala, so inspiration venumna neeye sollu: "search award winning sites first" nu prompt la explicit ah eludhu. (En idea)
+4. 20-30 rounds-ku mela poga vendaam; Sonnet 32-ku apram gain illa. Oru stop point vechuko. (En idea)
+5. Ovvoru round-um fresh session, clean context la run pannu, code mattum carry over aagattum. Idhu speaker-oda setup.
+6. Round mudinjadhum feature work aagudha nu check pannu (Spanish toggle maari broken things pidikka). (En idea)
+7. Cost paaru: Sonnet ~ $22 vs Opus ~ $137. Chinna improvement-ku Sonnet podhum, Opus ku vision/prompt strong ah irukkanum. (En idea)
+
+```
+Look at the current site first. Then make ONE bold change that improves user experience for [target user] toward this goal: [goal]. Before coding, search the web for 2-3 award-winning sites in this space and borrow one idea. Do not add features unrelated to the goal. Do not upsell. Make sure the change actually works, and fix anything broken from earlier rounds before adding new things.
+```