158 lines
4.8 KiB
Markdown
158 lines
4.8 KiB
Markdown
# The_DisExcel_project
|
|
|
|
A FastAPI project combining Excel and digital data ("The Great Excel project that is going to be built from Excel and digital projects").
|
|
|
|
## Stack
|
|
|
|
- Python >= 3.13
|
|
- FastAPI + Uvicorn / Gunicorn
|
|
- SQLAlchemy 2.0 (async) + Alembic (migrations)
|
|
- PostgreSQL (asyncpg, psycopg2)
|
|
- Pydantic 2 / Pydantic Settings
|
|
- Poetry — dependency management
|
|
- Redis — caching, rate limiting, token revocation
|
|
- RabbitMQ (aio-pika) — background email workers
|
|
- Jinja2 — HTML email templates
|
|
- Docker, Ansible — deployment
|
|
|
|
## Architecture
|
|
|
|
```
|
|
src/
|
|
├── cache/ # Redis client, rate limiting
|
|
├── daemons/ # background worker entrypoints (BaseDaemon, registry)
|
|
├── database/ # DB CRUD operations
|
|
├── errors/ # HTTP errors
|
|
├── logging/ # queue-based logging infra + HTTP middleware
|
|
├── messaging/ # RabbitMQ client, producers, consumers, topology
|
|
├── migrations/ # Alembic migrations
|
|
├── models/ # Pydantic and SQLAlchemy models, configs, RabbitMQ topology
|
|
├── reports/ # reports
|
|
├── service/ # business logic (auth, users_crud, email sending)
|
|
└── web/ # routes (protected_routes)
|
|
```
|
|
|
|
Layers are connected top to bottom: `web → service → database → models`.
|
|
|
|
## Background workers (RabbitMQ)
|
|
|
|
Email sending (welcome / password-reset) runs as separate daemon processes,
|
|
decoupled from the web API via a RabbitMQ topic exchange:
|
|
|
|
```
|
|
main.py / daemon_run.py → apply_topology() → RabbitMQ ("email" exchange)
|
|
├── queue_welcome_email → WelcomeEmailConsumer
|
|
└── queue_reset_email → ResetEmailConsumer
|
|
```
|
|
|
|
Each queue has a matching dead-letter queue for messages that fail
|
|
permanently (bad data, non-retryable errors) instead of retrying forever.
|
|
Run a worker locally with:
|
|
|
|
```bash
|
|
python daemon_run.py welcome_email # single daemon
|
|
python daemon_run.py --all # all daemons enabled in configs/daemons.json
|
|
```
|
|
|
|
## Authentication
|
|
|
|
- JWT access + refresh tokens
|
|
- Refresh token is stored in the DB as a SHA256 hash, with rotation and revocation support
|
|
- Passwords are hashed with bcrypt
|
|
- RBAC: direct user permissions + permissions via groups, checked through `require_permissions()`
|
|
|
|
## Testing
|
|
|
|
```
|
|
tests/
|
|
├── unit/
|
|
├── integrated/
|
|
└── e2e/
|
|
```
|
|
|
|
Uses pytest, pytest-asyncio, pytest-cov, pytest-mock, allure-pytest.
|
|
|
|
### Allure report
|
|
|
|
`make allure` needs the Allure **command-line tool** (Java-based, not a pip
|
|
package) — `allure-pytest` only writes raw result files, the CLI turns them
|
|
into an HTML report.
|
|
|
|
1. Download Allure **2.44.0** from
|
|
[github.com/allure-framework/allure2/releases](https://github.com/allure-framework/allure2/releases).
|
|
2. Extract it into `.venv/allure-2.44.0/` so that `.venv/allure-2.44.0/bin/allure`
|
|
exists — this matches the `ALLURE` variable already set in `makefile`.
|
|
3. If you install it somewhere else (or on Windows), update the `ALLURE`
|
|
variable at the top of `makefile` to point to your actual `allure`
|
|
binary path — the Windows path is already there, commented out.
|
|
|
|
```bash
|
|
make test # runs pytest, writes results to tests/allure-results/reports
|
|
make allure # builds tests/allure-results/html/index.html from those results
|
|
```
|
|
|
|
## Installation
|
|
|
|
Requires Poetry itself to be installed first (it's a global tool, not a
|
|
project dependency). Recommended via [pipx](https://pipx.pypa.io/):
|
|
|
|
```bash
|
|
pipx install poetry
|
|
```
|
|
|
|
or via the official installer:
|
|
|
|
```bash
|
|
curl -sSL https://install.python-poetry.org | python3 -
|
|
```
|
|
|
|
Then install the project dependencies:
|
|
|
|
```bash
|
|
poetry install
|
|
```
|
|
|
|
## Codebase knowledge graph (graphify)
|
|
|
|
The repo can be explored as a navigable knowledge graph via the
|
|
[graphify](https://github.com/safishamsi/graphify) Claude Code skill —
|
|
useful for onboarding or tracing how a change ripples across modules.
|
|
It lives in its own Poetry group (`claude`) so it's never installed in
|
|
`web`/`daemon`/prod images:
|
|
|
|
```bash
|
|
poetry install --with claude
|
|
```
|
|
|
|
Then, inside a Claude Code session in this repo, run:
|
|
|
|
```text
|
|
/graphify
|
|
```
|
|
|
|
This builds `graphify-out/graph.html` (open directly in a browser),
|
|
`graphify-out/GRAPH_REPORT.md` (god nodes, surprising cross-module
|
|
connections, suggested questions), and `graphify-out/graph.json` (raw
|
|
graph data). Ask follow-up questions about the codebase directly — once
|
|
`graphify-out/graph.json` exists, Claude answers from the graph instead
|
|
of rebuilding it. `graphify-out/` is gitignored: it's regenerable local
|
|
output, not part of the codebase.
|
|
|
|
## Running migrations
|
|
|
|
```bash
|
|
alembic upgrade head
|
|
```
|
|
|
|
## CI/CD
|
|
|
|
Pipeline is set up via Gitea Actions (`.gitea/workflows/ci.yml`).
|
|
|
|
## Deployment
|
|
|
|
The repository includes ready-made `docker/` and `ansible/` configs for containerization and deployment.
|
|
|
|
## License
|
|
|
|
MH.Dmitrii's project
|