Portal server mode for Dokploy: Dockerfile, PORTAL_URLS, PORTAL_SERVER
The results Portal in a Linux container (python 3.13-slim + ODBC Driver 18). PORTAL_SERVER=1: no Start / Stop (the bot runs on the Windows PC). PORTAL_URLS: allowed hosts / origins, Secure cookie when all https. RESULTS_DB_CONN (a SQL login) goes in Dokploy's environment, never here. 71 test files, 1222 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
ce9bdbd974
commit
c049cece23
9 files changed
+162
-12
No files matched your search
@@ -300,3 +300,10 @@ LOGIN_TIMEOUT_MS=120000
|
|||||||
NAV_TIMEOUT_MS=300000
|
NAV_TIMEOUT_MS=300000
|
||||||
NAV_RETRIES=3
|
NAV_RETRIES=3
|
||||||
CAPTCHA_ATTEMPTS=2
|
CAPTCHA_ATTEMPTS=2
|
||||||
|
|
||||||
|
# ── Results Portal (web/) ─────────────────────────────────────────────────────
|
||||||
|
# Unset = this PC: Windows login to the DB, http://127.0.0.1:8000, Start / Stop.
|
||||||
|
# On the server (Dokploy) set these as the app's environment, never in Git:
|
||||||
|
# RESULTS_DB_CONN=DRIVER={ODBC Driver 18 for SQL Server};SERVER=<host>;DATABASE=Charge_Validation;UID=<sql login>;PWD=<password>;TrustServerCertificate=yes;
|
||||||
|
# PORTAL_URLS=https://<the address people type>
|
||||||
|
# PORTAL_SERVER=1
|
||||||
+13
-4
@@ -187,7 +187,7 @@ Get-ChildItem tests\test_*.py | ForEach-Object { "$($_.Name): $((python $_.FullN
|
|||||||
- Tests run offline against fake HTML pages in `tests/fixtures/` and fake DB
|
- Tests run offline against fake HTML pages in `tests/fixtures/` and fake DB
|
||||||
connections; they never touch eCW or SQL Server (`test_cdp_browser.py`
|
connections; they never touch eCW or SQL Server (`test_cdp_browser.py`
|
||||||
starts a local Chromium on a debug port). As of 2026-10-27: 71 test
|
starts a local Chromium on a debug port). As of 2026-10-27: 71 test
|
||||||
files, **1220 tests** (full run 2026-10-27: 1220/1220) (earlier flaky, each
|
files, **1222 tests** (full run 2026-10-27: 1222/1222) (earlier flaky, each
|
||||||
failed once and passed on rerun: test_verify_no_live_writes,
|
failed once and passed on rerun: test_verify_no_live_writes,
|
||||||
depression_dx_pointer, progress_note_wait — watch them) (about 5 minutes for all of them).
|
depression_dx_pointer, progress_note_wait — watch them) (about 5 minutes for all of them).
|
||||||
|
|
||||||
@@ -623,7 +623,8 @@ the Portal (restarted 2026-10-26, [portal] terminal tab).
|
|||||||
|
|
||||||
History: `git log --oneline`.
|
History: `git log --oneline`.
|
||||||
|
|
||||||
Last code commits: `f99dc17` (2026-10-27, Portal roles Super Admin /
|
Last code commits: `70ba709` (2026-10-27, Portal server mode for Dokploy —
|
||||||
|
§10), `f99dc17` (2026-10-27, Portal roles Super Admin /
|
||||||
Admin / Viewer — §2), `61338b4` (2026-10-26, Sign-in page redesign + tab
|
Admin / Viewer — §2), `61338b4` (2026-10-26, Sign-in page redesign + tab
|
||||||
icon — §5), `a436489` (2026-10-26, Portal look: aligned header /
|
icon — §5), `a436489` (2026-10-26, Portal look: aligned header /
|
||||||
footer, 7-day chart, today's reasons, pass rate — §5), `2d4d4fc`
|
footer, 7-day chart, today's reasons, pass rate — §5), `2d4d4fc`
|
||||||
@@ -761,8 +762,16 @@ test claim ids — new posts must not (§7).
|
|||||||
- **Gitea:** one overall snapshot update at the end of a day, when the
|
- **Gitea:** one overall snapshot update at the end of a day, when the
|
||||||
operator asks (from the clone `D:\ecw-charge-validation`; operator
|
operator asks (from the clone `D:\ecw-charge-validation`; operator
|
||||||
pushes; 2026-10-26: `ba15a57` → `120fd68`).
|
pushes; 2026-10-26: `ba15a57` → `120fd68`).
|
||||||
- **Portal for the client — Gitea + Dokploy (asked 2026-10-23, not
|
- **Portal for the client — Gitea + Dokploy (asked 2026-10-23; server
|
||||||
started).** Gitea `giteanew.ecareindia.com` (10.200.0.15, internal; its
|
mode built 2026-10-27 `70ba709`, not deployed yet).** `Dockerfile`
|
||||||
|
(python 3.13-slim + ODBC Driver 18, copies only `src/` + `web/`,
|
||||||
|
`requirements-portal.txt`); env in Dokploy, never in Git:
|
||||||
|
`RESULTS_DB_CONN` (a SQL login; a Linux container cannot use
|
||||||
|
Trusted_Connection or, likely, the named instance — host,port),
|
||||||
|
`PORTAL_URLS` (hosts / origins; all-https = Secure cookie),
|
||||||
|
`PORTAL_SERVER=1` (set in the image: NoRunner, no Start / Stop). The
|
||||||
|
Runs page there is empty (events files stay on the bot PC). Earlier
|
||||||
|
notes: Gitea `giteanew.ecareindia.com` (10.200.0.15, internal; its
|
||||||
HTTPS certificate is not in Git Bash's curl bundle) and Dokploy
|
HTTPS certificate is not in Git Bash's curl bundle) and Dokploy
|
||||||
(`http://10.200.0.15:3000`, same server). The operator joins Dokploy and
|
(`http://10.200.0.15:3000`, same server). The operator joins Dokploy and
|
||||||
creates a **Private** Gitea repo themselves; nothing is pushed until they
|
creates a **Private** Gitea repo themselves; nothing is pushed until they
|
||||||
|
|||||||
+30
@@ -0,0 +1,30 @@
|
|||||||
|
# The results Portal on a Linux server (Dokploy, 2026-10-27): Sign-in,
|
||||||
|
# Dashboard, Claim status, Runs. The bot runs only on the Windows PC, so
|
||||||
|
# PORTAL_SERVER=1 turns Start / Stop off. Set RESULTS_DB_CONN (a SQL login)
|
||||||
|
# and PORTAL_URLS in Dokploy's environment — never in Git (.env.example).
|
||||||
|
FROM python:3.13-slim-bookworm
|
||||||
|
|
||||||
|
# Microsoft ODBC Driver 18 for SQL Server, for pyodbc.
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends curl gnupg ca-certificates \
|
||||||
|
&& curl -fsSL https://packages.microsoft.com/keys/microsoft.asc \
|
||||||
|
| gpg --dearmor -o /usr/share/keyrings/microsoft-prod.gpg \
|
||||||
|
&& curl -fsSL https://packages.microsoft.com/config/debian/12/prod.list \
|
||||||
|
-o /etc/apt/sources.list.d/mssql-release.list \
|
||||||
|
&& apt-get update \
|
||||||
|
&& ACCEPT_EULA=Y apt-get install -y --no-install-recommends msodbcsql18 \
|
||||||
|
&& apt-get purge -y curl gnupg && apt-get autoremove -y \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY requirements-portal.txt .
|
||||||
|
RUN pip install --no-cache-dir -r requirements-portal.txt
|
||||||
|
COPY src ./src
|
||||||
|
COPY web ./web
|
||||||
|
|
||||||
|
RUN useradd --create-home portal
|
||||||
|
USER portal
|
||||||
|
ENV PORTAL_SERVER=1 PYTHONUNBUFFERED=1
|
||||||
|
EXPOSE 8000
|
||||||
|
CMD ["python", "-m", "uvicorn", "web.app:app", "--host", "0.0.0.0", \
|
||||||
|
"--port", "8000", "--proxy-headers", "--forwarded-allow-ips", "*"]
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# The results Portal (web/) on a Linux server — Dockerfile. The bot's own
|
||||||
|
# packages (playwright, pyotp, openpyxl) are not needed there: the bot runs
|
||||||
|
# only on the Windows PC. Versions as on that PC, 2026-10-27.
|
||||||
|
fastapi==0.141.1
|
||||||
|
starlette==1.3.1
|
||||||
|
uvicorn==0.52.1
|
||||||
|
jinja2==3.1.6
|
||||||
|
python-multipart==0.0.32
|
||||||
|
pyodbc==5.3.0
|
||||||
@@ -21,7 +21,10 @@ import os
|
|||||||
import time
|
import time
|
||||||
import json
|
import json
|
||||||
import ctypes
|
import ctypes
|
||||||
|
try:
|
||||||
from ctypes import wintypes
|
from ctypes import wintypes
|
||||||
|
except ValueError: # some non-Windows Pythons; the server Portal
|
||||||
|
wintypes = None # (PORTAL_SERVER=1) never calls the kernel32 parts
|
||||||
from datetime import datetime, timedelta
|
from datetime import datetime, timedelta
|
||||||
|
|
||||||
from .. import settings as cfg
|
from .. import settings as cfg
|
||||||
|
|||||||
@@ -688,6 +688,13 @@ RESULTS_DB_CONN = _env(
|
|||||||
"DATABASE=Charge_Validation;Trusted_Connection=yes;"
|
"DATABASE=Charge_Validation;Trusted_Connection=yes;"
|
||||||
"TrustServerCertificate=yes;")
|
"TrustServerCertificate=yes;")
|
||||||
|
|
||||||
|
# The Portal on a server (Dokploy, 2026-10-27). PORTAL_URLS: the address(es)
|
||||||
|
# people type, comma separated (e.g. https://cv.example.com); empty = this
|
||||||
|
# PC only (127.0.0.1:8000). PORTAL_SERVER=1: no Start / Stop — the bot runs
|
||||||
|
# only on the Windows PC.
|
||||||
|
PORTAL_URLS = _env("PORTAL_URLS")
|
||||||
|
PORTAL_SERVER = _env_bool("PORTAL_SERVER", False)
|
||||||
|
|
||||||
# eCW builds the file server-side over a wide date range, so this is generous.
|
# eCW builds the file server-side over a wide date range, so this is generous.
|
||||||
DOWNLOAD_TIMEOUT = _env_int("DOWNLOAD_TIMEOUT_MS", 300_000) # 5 min
|
DOWNLOAD_TIMEOUT = _env_int("DOWNLOAD_TIMEOUT_MS", 300_000) # 5 min
|
||||||
|
|
||||||
|
|||||||
@@ -332,6 +332,52 @@ def the_sign_in_page_says_what_the_portal_is_and_helps_with_the_password():
|
|||||||
assert '<link rel="icon"' in page, "a tab icon, no /favicon.ico 404"
|
assert '<link rel="icon"' in page, "a tab icon, no /favicon.ico 404"
|
||||||
|
|
||||||
|
|
||||||
|
@case
|
||||||
|
def a_server_portal_never_starts_the_bot():
|
||||||
|
"""Dokploy, 2026-10-27: the bot runs only on the Windows PC, so the
|
||||||
|
Portal on the server shows results and never offers Start / Stop."""
|
||||||
|
store = FakeUserStore()
|
||||||
|
portal_users.Accounts(store).create_user("sam", "Sam", "Super Admin", PW)
|
||||||
|
app = web_app.create_app(FakeRepo(), users_store=store, can_start=False,
|
||||||
|
logs_dir=tempfile.mkdtemp())
|
||||||
|
c, _ = _sign_in(app, "sam")
|
||||||
|
assert "const CAN_RUN = false;" in c.get("/").text
|
||||||
|
h = {**ORIGIN, "X-Run-Token": app.state.run_token}
|
||||||
|
assert c.post("/api/bot/start", headers=h).status_code == 403
|
||||||
|
assert c.post("/api/bot/stop", headers=h).status_code == 403
|
||||||
|
st = c.get("/api/bot/status").json()
|
||||||
|
assert st["running"] is False, st
|
||||||
|
assert c.get("/runs").status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
@case
|
||||||
|
def the_portal_url_gives_the_hosts_origins_and_a_secure_cookie():
|
||||||
|
env = web_app.portal_settings("https://cv.example.com, "
|
||||||
|
"http://10.0.0.5:8080/", server=True)
|
||||||
|
assert env == {"allowed_hosts": ["cv.example.com", "10.0.0.5"],
|
||||||
|
"allowed_origins": ["https://cv.example.com",
|
||||||
|
"http://10.0.0.5:8080"],
|
||||||
|
"secure_cookie": False, "can_start": False}, env
|
||||||
|
assert web_app.portal_settings("https://cv.example.com",
|
||||||
|
server=True)["secure_cookie"] is True
|
||||||
|
local = web_app.portal_settings("", server=False)
|
||||||
|
assert local == {"allowed_hosts": list(web_app.ALLOWED_HOSTS),
|
||||||
|
"allowed_origins": list(web_app.ALLOWED_ORIGINS),
|
||||||
|
"secure_cookie": False, "can_start": True}, local
|
||||||
|
base = "https://cv.example.com"
|
||||||
|
store = FakeUserStore()
|
||||||
|
portal_users.Accounts(store).create_user("sam", "Sam", "Viewer", PW)
|
||||||
|
app = web_app.create_app(FakeRepo(), runner=FakeRunner(),
|
||||||
|
users_store=store, logs_dir=tempfile.mkdtemp(),
|
||||||
|
**web_app.portal_settings(base, server=True))
|
||||||
|
r = TestClient(app, base_url=base).post(
|
||||||
|
"/signin", headers={"Origin": base}, follow_redirects=False,
|
||||||
|
data={"username": "sam", "password": PW,
|
||||||
|
"token": app.state.run_token, "next": "/"})
|
||||||
|
assert r.status_code == 303, r.status_code
|
||||||
|
assert "; secure" in r.headers["set-cookie"].lower(), r.headers
|
||||||
|
|
||||||
|
|
||||||
@case
|
@case
|
||||||
def no_signed_in_page_carries_an_inline_style():
|
def no_signed_in_page_carries_an_inline_style():
|
||||||
app, _, _ = _portal()
|
app, _, _ = _portal()
|
||||||
|
|||||||
+30
-8
@@ -58,7 +58,7 @@ from .users import (Accounts, CAN_ADMIN, CAN_RUN, # noqa: E402
|
|||||||
IDLE_MINUTES, LOCK_AFTER, LOCK_MINUTES, MANAGES,
|
IDLE_MINUTES, LOCK_AFTER, LOCK_MINUTES, MANAGES,
|
||||||
Sessions, SqlUserStore, UserError)
|
Sessions, SqlUserStore, UserError)
|
||||||
from .bot_runner import (AlreadyRunning, BotRunner, # noqa: E402
|
from .bot_runner import (AlreadyRunning, BotRunner, # noqa: E402
|
||||||
NotRunning, StopFailed)
|
NoRunner, NotRunning, StopFailed)
|
||||||
|
|
||||||
CATEGORIES = ("Pass", "Fail", "Review")
|
CATEGORIES = ("Pass", "Fail", "Review")
|
||||||
DB_DOWN = "Results database not reachable"
|
DB_DOWN = "Results database not reachable"
|
||||||
@@ -261,13 +261,32 @@ def _reasons(rows, top=6) -> list:
|
|||||||
for (cat, text), n in ordered[:top]]
|
for (cat, text), n in ordered[:top]]
|
||||||
|
|
||||||
|
|
||||||
|
def portal_settings(urls: str, server: bool) -> dict:
|
||||||
|
"""create_app's host / origin / cookie / Start settings from
|
||||||
|
PORTAL_URLS (comma separated, as people type them in the browser) and
|
||||||
|
PORTAL_SERVER. No URLs: this PC only. A server Portal never starts the
|
||||||
|
bot; an all-https Portal sends the cookie only over https."""
|
||||||
|
parts = [urlsplit(u.strip()) for u in (urls or "").split(",")
|
||||||
|
if u.strip()]
|
||||||
|
if not parts:
|
||||||
|
return {"allowed_hosts": list(ALLOWED_HOSTS),
|
||||||
|
"allowed_origins": list(ALLOWED_ORIGINS),
|
||||||
|
"secure_cookie": False, "can_start": not server}
|
||||||
|
return {"allowed_hosts": [u.hostname for u in parts],
|
||||||
|
"allowed_origins": [f"{u.scheme}://{u.netloc}" for u in parts],
|
||||||
|
"secure_cookie": all(u.scheme == "https" for u in parts),
|
||||||
|
"can_start": not server}
|
||||||
|
|
||||||
|
|
||||||
def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
|
def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
|
||||||
allowed_origins=ALLOWED_ORIGINS, logs_dir=None,
|
allowed_origins=ALLOWED_ORIGINS, logs_dir=None,
|
||||||
screenshots_root=None, users_store=None,
|
screenshots_root=None, users_store=None,
|
||||||
sessions=None) -> FastAPI:
|
sessions=None, can_start=True,
|
||||||
|
secure_cookie=False) -> FastAPI:
|
||||||
app = FastAPI(title="eCW Charge Validation — Results",
|
app = FastAPI(title="eCW Charge Validation — Results",
|
||||||
docs_url=None, redoc_url=None, openapi_url=None)
|
docs_url=None, redoc_url=None, openapi_url=None)
|
||||||
runner = runner if runner is not None else BotRunner()
|
if runner is None:
|
||||||
|
runner = BotRunner() if can_start else NoRunner()
|
||||||
logs_dir = logs_dir if logs_dir is not None else cfg.LOGS
|
logs_dir = logs_dir if logs_dir is not None else cfg.LOGS
|
||||||
screenshots_root = (screenshots_root if screenshots_root is not None
|
screenshots_root = (screenshots_root if screenshots_root is not None
|
||||||
else cfg.ROOT / "Screenshots")
|
else cfg.ROOT / "Screenshots")
|
||||||
@@ -379,7 +398,8 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
|
|||||||
"mode": "LIVE-WRITE" if cfg.SAVE_CORRECTIONS else "VERIFY",
|
"mode": "LIVE-WRITE" if cfg.SAVE_CORRECTIONS else "VERIFY",
|
||||||
"run_token": app.state.run_token, "crumbs": crumbs,
|
"run_token": app.state.run_token, "crumbs": crumbs,
|
||||||
"categories": CATEGORIES, "user": user,
|
"categories": CATEGORIES, "user": user,
|
||||||
"can_run": bool(user and user["role"] in CAN_RUN),
|
"can_run": bool(can_start and user
|
||||||
|
and user["role"] in CAN_RUN),
|
||||||
"is_admin": bool(user and user["role"] in CAN_ADMIN),
|
"is_admin": bool(user and user["role"] in CAN_ADMIN),
|
||||||
**context})
|
**context})
|
||||||
|
|
||||||
@@ -521,7 +541,7 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
|
|||||||
@app.post("/api/bot/start")
|
@app.post("/api/bot/start")
|
||||||
def bot_start(request: Request):
|
def bot_start(request: Request):
|
||||||
_check_writer(request)
|
_check_writer(request)
|
||||||
user = _need(request, CAN_RUN)
|
user = _need(request, CAN_RUN if can_start else ())
|
||||||
try:
|
try:
|
||||||
return run_status.summarize(
|
return run_status.summarize(
|
||||||
runner.start(started_by=user["username"]))
|
runner.start(started_by=user["username"]))
|
||||||
@@ -531,7 +551,7 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
|
|||||||
@app.post("/api/bot/stop")
|
@app.post("/api/bot/stop")
|
||||||
def bot_stop(request: Request):
|
def bot_stop(request: Request):
|
||||||
_check_writer(request)
|
_check_writer(request)
|
||||||
user = _need(request, CAN_RUN)
|
user = _need(request, CAN_RUN if can_start else ())
|
||||||
try:
|
try:
|
||||||
return run_status.summarize(
|
return run_status.summarize(
|
||||||
runner.stop(stopped_by=user["username"]))
|
runner.stop(stopped_by=user["username"]))
|
||||||
@@ -576,7 +596,8 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
|
|||||||
else _safe_next(next))
|
else _safe_next(next))
|
||||||
response = RedirectResponse(target, status_code=303)
|
response = RedirectResponse(target, status_code=303)
|
||||||
response.set_cookie(COOKIE, sessions.new(user), httponly=True,
|
response.set_cookie(COOKIE, sessions.new(user), httponly=True,
|
||||||
samesite="strict", path="/")
|
samesite="strict", path="/",
|
||||||
|
secure=secure_cookie)
|
||||||
return response
|
return response
|
||||||
|
|
||||||
@app.post("/signout")
|
@app.post("/signout")
|
||||||
@@ -709,4 +730,5 @@ def _safe_next(target) -> str:
|
|||||||
|
|
||||||
|
|
||||||
app = create_app(SqlResultsRepo(cfg.RESULTS_DB_CONN),
|
app = create_app(SqlResultsRepo(cfg.RESULTS_DB_CONN),
|
||||||
users_store=SqlUserStore(cfg.RESULTS_DB_CONN))
|
users_store=SqlUserStore(cfg.RESULTS_DB_CONN),
|
||||||
|
**portal_settings(cfg.PORTAL_URLS, cfg.PORTAL_SERVER))
|
||||||
@@ -50,6 +50,23 @@ class StopFailed(Exception):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class NoRunner:
|
||||||
|
"""The Portal on a server (Dokploy, 2026-10-27): the bot runs only on
|
||||||
|
the Windows PC, so nothing here ever starts, stops or looks for it."""
|
||||||
|
|
||||||
|
def start(self, started_by: str = "") -> dict:
|
||||||
|
raise NotRunning("Runs start on the bot PC, not from this server")
|
||||||
|
|
||||||
|
def stop(self, stopped_by: str = "") -> dict:
|
||||||
|
raise NotRunning("Runs start on the bot PC, not from this server")
|
||||||
|
|
||||||
|
def status(self) -> dict:
|
||||||
|
return {"running": False, "starting": False, "pid": None,
|
||||||
|
"started_at": None, "ended_at": None, "exit_code": None,
|
||||||
|
"stopped_by_operator": False, "source": None,
|
||||||
|
"started_by": None, "log_path": None, "out_path": None}
|
||||||
|
|
||||||
|
|
||||||
class BotRunner:
|
class BotRunner:
|
||||||
# How long stop() waits for a killed process to be gone.
|
# How long stop() waits for a killed process to be gone.
|
||||||
stop_wait = 10.0
|
stop_wait = 10.0
|
||||||
|
|||||||
Reference in new issue
Block a user