add logging and retry to connect to the rabbitmq server

This commit is contained in:
2026-09-14 18:34:15 +03:00
parent a93c6d5fca
commit e124d897eb
10 changed files with 241 additions and 90 deletions
+112
View File
@@ -0,0 +1,112 @@
# The DisExcel Project — Context for Claude Code
Backend: FastAPI + SQLAlchemy (async) + Pydantic v2 + PostgreSQL. JWT auth
(access + refresh tokens), RBAC permissions, Redis cache, RabbitMQ background
workers. Python >=3.13,<4.0, Poetry for dependency management.
## Auth & Permissions
- Users have `direct_permissions` (list of `Permissions`) and `group`
(list of `PermissionsGroups`, each with its own `permissions`) —
many-to-many both ways.
- Effective permissions = `direct_permissions (union of all groups' permissions)`.
- `require_permissions(*permissions)` in `src/web/protected_routes` is a
FastAPI dependency factory — wraps `CurrentUserService.get_current_user`.
Call with no args (`require_permissions()`) for "just authenticated, no
specific permission needed".
- Access tokens carry a `jti` claim. Logout writes `revoked_access_token:{jti}`
to Redis with TTL = remaining token lifetime — `get_current_user` checks
this key before anything else.
- `secure` flag on refresh_token cookie is driven by `env_settings.PROD_MODE`
(bool) — `False` locally/tests so cookies work over plain HTTP, `True` in
prod.
## Redis (`src/cache/`)
- `RedisClient(redis.Redis)` — module-level shared singleton, subclasses
`redis.Redis` directly (inherits all commands, no manual wrapping needed).
- Three uses: permissions is-cache was considered and rejected (no real DB
savings — `get_user_by_id` already eager-loads everything via `selectin`
in one call); rate limiting on login (`RateLimit.rate_limit(ip)`
`INCR` + `EXPIRE` on first attempt, blocks >5/60s); access-token revoke
blacklist (see above).
- Rate limit is only triggered inside `except HTTPException` on `/protected/token`
— i.e. only on failed logins, not successful ones (otherwise legitimate
repeated logins would trip it).
## RabbitMQ (`src/messaging/`)
- `RabbitMQClient` — shared class, lazy `connect()` (can't be async `__init__`),
holds one `connection` + one `channel`, `get_channel()` ensures setup.
- Topic exchange named `"email"`. Each message type gets its own routing key
(`email.welcome`, `email.reset`) and its own durable queue
(`queue_welcome_email`, `queue_reset_email`), each queue explicitly bound
to the exchange with its own key.
- **Important**: both producer and consumer must declare/bind the queues —
if only the consumer does it and the consumer has never run, `publish`
on a not-yet-existing queue silently loses the message. `EmailProducer.setup()`
also declares+binds both queues defensively.
- `message.process()` async context manager = manual ack (auto-ack on
success, requeue/nack on exception) — this is the right choice for email,
not `no_ack=True`.
- Email templates: Jinja2, inline CSS (email clients don't support `<style>`
reliably), `EmailMessage` with `set_content()` (plain-text fallback) +
`add_alternative(html, subtype="html")`.
## Daemons / workers (`src/daemons/`)
- `BaseDaemon` ABC (`name` + async `run()`), one subclass per consumer
(`WelcomeEmailDaemon`, `ResetEmailDaemon`, more to come — e.g. reports).
- `DAEMONS` registry dict maps string name → daemon class.
- `run_daemon.py` (project root) is the single entrypoint: `python
run_daemon.py <name>` runs one daemon, `python run_daemon.py --all` reads
`configs/daemons.json` (`{"daemons": [...]}`) and runs all enabled ones
concurrently via `asyncio.gather`.
- Each daemon runs as its own Docker service/container (`command: ["python",
"run_daemon.py", "<name>"]`), same pattern as the `migration` service.
- `daemons.json` is read with a plain Pydantic `BaseModel` + manual
`json.load`, NOT `pydantic-settings` `json_file` — that requires wiring
`settings_customise_sources` manually in this pydantic-settings version
and isn't worth the complexity here.
## Docker / Poetry groups
- `pyproject.toml` uses PEP 621 `[project.dependencies]` for shared deps
(sqlalchemy, redis, aio-pika, pydantic, bcrypt, jose, aiofiles, asyncpg,
psycopg2-binary, alembic, greenlet).
- `web` group: fastapi, uvicorn, gunicorn, python-multipart — only needed
by the API server.
- `daemon` group: worker-only deps (currently empty, grows as needed).
- `dev` group: pytest stack, allure, httpie, requests-async.
- Dockerfile has parallel builder→final stage pairs: `builder`→`prod`
(installs `main,web`) and `worker-builder`→`worker` (installs
`main,daemon`). Same base pattern: venv builder stage copies
`/opt/venv` into a clean final stage, poetry itself is uninstalled
after install to keep the final image lean.
- Alembic runs against a **separate sync engine** (`asyncpg` swapped out,
psycopg2 used instead) — async SQLAlchemy engine can't drive Alembic
directly without the `run_sync` bridge, and a dedicated sync engine is
simpler than that bridge.
- `DB_HOST` differs between contexts: `psql` (Docker service name) for
containers talking to each other, `localhost` for anything run on the
host (e.g. local `alembic revision --autogenerate`). Compose services
override `DB_HOST` via `environment:`; the `.env` file's own default is
for host-side runs.
## Testing (`tests/unit`, `tests/integrated`, `tests/e2e`)
- **Recurring root cause of "different event loop" / `MissingGreenlet`-style
errors**: prod code uses module-level singletons (`engine`, `redis_client`)
created once at import time and reused for the app's whole lifetime — this
is correct for prod (one event loop, whole uptime) but breaks under
pytest-asyncio's default `function`-scoped event loop (a new loop per
test, but the singleton's connections stay bound to the *first* loop).
Fix: test fixtures create a **fresh** `engine`/`RedisClient` per test and
monkeypatch or inject them in place of the global singleton, then dispose
on teardown — not a global `session`-scoped event loop (that would mask
real isolation bugs).
- e2e `MySession` must subclass `httpx.AsyncClient` (not `requests_async.
AsyncSession` — that library silently drops cookies between requests,
which broke refresh-token-cookie-dependent tests like logout).
- `test_user_fixture` is `indirect=True` parametrized with
`(direct_permissions, group)` tuples.