Skip to content

petreleon/rust-learn

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,029 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RustLearn

RustLearn is an incentivized e-learning platform built with Rust. It combines a modular Actix Web API, PostgreSQL/Diesel persistence, S3-compatible media storage, background video processing, Ethereum smart contracts, and a Next.js frontend to support learning experiences where achievement can be rewarded with LearnToken.

Why RustLearn?

Traditional learning platforms often make progress feel abstract and delayed. RustLearn is designed around immediate, auditable progress: learners complete coursework, teachers and organizations manage educational programs, and reward workflows can connect learning achievements to token-backed incentives.

The long-term goal is to provide a trustworthy platform where:

  • learners can discover courses, consume content, complete assessments, and receive progress notifications;
  • teachers can create and publish content, manage enrollments, and moderate course activity;
  • organizations can manage members, courses, reports, and scoped permissions;
  • platform administrators can operate global roles, permissions, wallets, exports, and integrations;
  • blockchain-backed reward flows can be tested and audited before broader deployment.

See VISION.md for the product and architecture direction, and TODO/ for the current roadmap. The target LearnToken reward flow is defined in REWARD_LIFECYCLE.md.

Repository layout

.
├── src/                    # Rust API, Level 2 rings, and bootstrap
│   ├── http/               # Actix routes, middleware, extractors, DTOs
│   ├── application/        # Use cases, ports, commands, outputs
│   ├── domain/             # Pure domain vocabulary and rules
│   ├── infra/              # PostgreSQL schema/models, storage, Ethereum, adapters
│   ├── bootstrap/          # App state, app data, route wiring, startup
│   └── bin/worker.rs       # Background upload/video-processing worker
├── ethereum/               # Solidity contracts and generated ABI/bin artifacts
├── migrations/             # Diesel migrations
├── tests/                  # Integration and permission tests
├── web/                    # Next.js frontend
├── k8s/                    # Kubernetes manifests
├── skills/                 # Repo-local Codex skill definitions
├── docker-compose.yml      # Local app, web, Postgres, RustFS, Anvil, worker stack
├── Makefile                # Common development, Docker, Kubernetes, and test commands
├── PERMISSIONS.md          # Current role/permission matrix
├── VISION.md               # Product and architecture vision
├── TODO/                   # Prioritized project roadmap
└── AGENTS.md               # Contributor/AI-agent guidance

Architecture flow

flowchart LR
    Browser[Browser] --> Web[Next.js frontend]
    Web --> API[Actix Web API]
    API --> Auth[JWT and permission middleware]
    Auth --> HTTP[HTTP routes and extractors]
    HTTP --> Application[Application use cases]
    Application --> Domain[Domain rules]
    Application --> Infra[Infra adapters]
    Infra --> Postgres[(PostgreSQL / Diesel)]
    Infra --> RustFS[(RustFS / S3 objects)]
    Infra --> Ethereum[Anvil or Ethereum RPC]
    API --> Jobs[Upload and video jobs]
    Jobs --> Worker[Worker binary]
    Worker --> Infra
Loading

Core components

Backend API

  • Rust 2021 with Actix Web.
  • JWT-protected /api scope.
  • Route modules for authentication, users, wallets, courses, organizations, and roles.
  • Registration rejects weak passwords: passwords must be at least 12 characters and include lowercase, uppercase, numeric, and symbol characters.
  • Diesel and Diesel Async with PostgreSQL.
  • Application use cases coordinate business workflows; context-owned infra adapters own persistence and external integrations.

Permissions and roles

RustLearn models permissions at three scopes:

  • platform roles and permissions;
  • organization roles and permissions;
  • course roles and permissions.

Middleware and hierarchy checks are used to protect sensitive actions. Keep PERMISSIONS.md updated when role capabilities change.

Storage and media worker

The project uses S3-compatible object storage through the AWS SDK. The Docker Compose stack runs RustFS for local development. A separate worker binary processes upload jobs, runs ffmpeg-related work, retries failures, and writes a heartbeat file used by the worker health check.

Blockchain integration

The ethereum/ directory contains LearnToken-related Solidity contracts and generated artifacts. Startup code can deploy LearnToken idempotently using persistent state, and integration tests cover important contract behavior.

Operational backup/restore expectations and the Ethereum artifact release process are documented in docs/operations.md.

Frontend

The web/ directory contains a Next.js app that keeps browser API calls on the same origin through /api by default. API_URL points the Next proxy at the Rust API in container/server contexts; /health and /ready are proxied for runtime status surfaces. NEXT_PUBLIC_API_URL can override the browser root when needed. The default / route resolves the current product session, sending anonymous users to /login and users with a stored token to /session. The internal workflow console for teacher applications, reward decisions, reports, fraud blocks, delegations, exports, and local API debugging lives at /ops while the route-based product frontend replaces it. /admin is now a platform operator dashboard for summary metrics, reward audit, fraud controls, CSV exports, and health/readiness status. /admin/teacher-applications is now a platform teacher-review queue with filters, inline applicant/sponsor detail, audit history, permission-gated decisions, conflict refresh, and mobile-first layout; reward amount review, delegation management, and wallet reconciliation remain separate product work.

Prerequisites

Recommended local tools:

  • Rust stable toolchain and Cargo
  • Docker and Docker Compose
  • PostgreSQL client tooling only for manual database inspection
  • Node.js/npm for frontend development
  • ffmpeg for local worker/media-processing scenarios
  • OpenSSL for RSA key generation

On macOS, Colima or Docker Desktop can provide the Docker VM. The worker release build can be memory-intensive; 8GB+ allocated to the Docker VM is recommended when building worker images.

Configuration

Create a local .env file from the example:

make setup

make setup copies .env.example to .env, restricts it to the current user with 0600 permissions, and, when OpenSSL and Python 3 are available, replaces the committed JWT placeholders with a freshly generated local RSA key pair. Then edit .env for your environment-specific database, object-storage, Ethereum, and bootstrap-admin values.

RSA keys for JWT signing

.env.example intentionally contains only non-secret JWT key placeholders. Use make setup for local development, or generate a private/public RSA key pair manually:

openssl genpkey -algorithm RSA -out private.key -pkeyopt rsa_keygen_bits:2048
openssl rsa -pubout -in private.key -out public.key

Add the contents to .env:

PRIVATE_KEY="-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----"

PUBLIC_KEY="-----BEGIN PUBLIC KEY-----
...
-----END PUBLIC KEY-----"

The API publishes the configured public key as JWKS at /.well-known/jwks.json and /api/.well-known/jwks.json for external JWT verification. Set JWT_KEY_ID when you need a stable kid value across key rollout or multiple environments.

Do not commit .env, real private keys, or production secrets.

Health and readiness

The API exposes GET /health for shallow liveness and GET /ready for dependency readiness. Readiness checks PostgreSQL, S3-compatible storage, and the configured Ethereum JSON-RPC endpoint before returning ready.

Structured logging

The API and worker initialize structured terminal logging at startup. Logs use the RUST_LOG filter and default to info; set values such as RUST_LOG=rust_learn=debug,info when you need more detail from application code without enabling verbose logs for every dependency.

The worker emits operational metric events to the same log stream: worker_queue_metrics reports queue depth, ready and delayed queued jobs, processing jobs, failed jobs, and in-flight tasks. Idle metric logging is throttled by WORKER_QUEUE_METRICS_INTERVAL_SECONDS to keep Compose and Kubernetes logs readable. Per-job events include worker_job_claimed, worker_job_started, worker_job_processed, worker_job_retry_scheduled, and worker_job_terminal_failure with attempt numbers and processing duration in milliseconds.

Mock email preview

Registration creates an email-verification token and prints a local-development mock verification email to the API terminal or container logs. No email provider is called yet. Set APP_PUBLIC_URL to control the base URL used in the printed link; the Compose default points directly at the API verification endpoint. Password reset requests print a separate mock reset email. Set WEB_PUBLIC_URL to control the browser-facing base URL used in that reset link.

Preview the mock email without registering a user:

make mock-email

Override the preview values when needed:

make mock-email MOCK_EMAIL=learner@example.com MOCK_NAME='Demo Learner' MOCK_TOKEN=mock-preview-token

Wallet linking API

Authenticated users can create and read their own internal wallet link with POST /api/wallets/me/link and GET /api/wallets/me. Platform wallet managers can link or read another user wallet with /api/wallets/users/{id} routes. Organization wallet managers can link and read organization wallets with /api/wallets/organizations/{id} routes. Link endpoints are idempotent and return the existing wallet on repeated calls. Wallet audit endpoints are available at /api/wallets/me/audit, /api/wallets/users/{id}/audit, and /api/wallets/organizations/{id}/audit; organization wallet audit reads are available to organization wallet managers, reward-budget managers, and organization reward-report viewers.

Authenticated users can move LearnToken between their centralized platform wallet and an Ethereum wallet with POST /api/wallets/me/deposits and POST /api/wallets/me/retirements. Requests include gas_payer as user or platform; when the platform pays Ethereum gas, the configured token tax is applied as a separate wallet debit. Deposit requests create a pending deposit intent and do not credit the internal wallet immediately; the worker deposit indexer credits the wallet only after it observes the matching confirmed Ethereum event. Transfer responses include wallet_provider, metamask_required, and wallet_action: MetaMask is implied for every transfer path except a platform-paid retirement, where the person is only receiving tokens and the platform sends the transfer. Authenticated users can read current token taxes with GET /api/wallets/token-taxes; operators update them through PUT /api/wallets/token-taxes/deposit or PUT /api/wallets/token-taxes/retire, gated by platform SET_DEPOSIT_TAX and SET_RETIRE_TAX respectively.

Student reward history API

Authenticated students can read GET /api/reward-candidates/me/history for their own reward candidate timeline. The response includes candidate status, approved amount, wallet credit details, and token transaction references when available. Rows are included only for courses where the requester has VIEW_COURSE_REWARD_STATUS.

Reward fraud and delegation APIs

Platform reward-fraud operators can manage reward blocks through POST /api/reward-fraud-blocks, GET /api/reward-fraud-blocks, PUT /api/reward-fraud-blocks/{id}/revoke, and GET /api/reward-fraud-blocks/{id}/audit. Teacher blocks require BLOCK_REWARD_TEACHER or MANAGE_REWARD_FRAUD_BLOCKS; organization blocks require BLOCK_REWARD_ORGANIZATION or MANAGE_REWARD_FRAUD_BLOCKS; course and reward-policy blocks require MANAGE_REWARD_FRAUD_BLOCKS. Listing and audit history require VIEW_REWARD_AUDIT or MANAGE_REWARD_FRAUD_BLOCKS.

Central administrators with DELEGATE_REWARD_APPROVAL can grant, list, and revoke delegated reward permissions with POST /api/delegated-permissions, GET /api/delegated-permissions, and PUT /api/delegated-permissions/{id}/revoke.

Course discovery API

GET /api/courses supports search, organization_id, limit, and offset query parameters. The response includes courses, total, limit, offset, search, and organization_id so learners and organization views can paginate and filter discovery results consistently.

Organization dashboard API

Organization operators with any organization dashboard access can read GET /api/organizations/{id}/dashboard. The response includes organization health, member and course counts, sponsored teacher-application counts, reward volume, approved amount totals, failed/reconciliation reward counts, organization wallet balance, operator permission booleans, gated-section missing-permission lists, and dashboard alerts for triage.

Organization course API

Organization operators with platform or organization-scoped VIEW_ORGANIZATION can read GET /api/organizations/{id}/courses. The response supports search, lifecycle_status, reward_available, limit, and offset and includes organization labels, course lifecycle, teacher labels, content summaries, roster counts, reward-policy summaries, reward queue counts, and operator permission booleans for the organization course workspace.

Organization member API

Organization operators with platform or organization-scoped VIEW_ORGANIZATION can read GET /api/organizations/{id}/members. The response supports search, role, permission, limit, and offset and includes organization labels, member names and emails, email/KYC readiness, role labels, direct permissions, active delegated organization permissions, effective permissions, and operator permission booleans for the member directory.

Organization teacher application API

Organization operators with platform or organization-scoped VIEW_ORG_TEACHER_APPLICATIONS or NOMINATE_TEACHER_FOR_PLATFORM_REVIEW can read GET /api/organizations/{id}/teacher-applications. The response supports search, status, limit, and offset and includes organization labels, applicant names and emails, requested scope, requested organization/course labels, portfolio links, status, reviewer and decision context, audit summary, dashboard summary counts, and operator permission booleans for sponsored application tracking. Nomination submission still uses POST /api/organizations/{id}/teacher-applications; product UI submission should wait for searchable applicant lookup so operators do not type raw user ids.

Reporting exports

Platform administrators can read GET /api/reports/platform/summary and export GET /api/reports/platform/summary.csv. Organization administrators can read GET /api/reports/organizations/{id}/summary and export GET /api/reports/organizations/{id}/summary.csv. CSV endpoints return text/csv with attachment filenames. Organization reward operators with VIEW_ORG_REWARD_REPORTS can read and export GET /api/reports/organizations/{id}/reward-dashboard and GET /api/reports/organizations/{id}/reward-dashboard.csv for sponsored teacher applications, course reward volume, approved amounts, and organization wallet balances. Additional platform exports require EXPORT_DATA: GET /api/reports/platform/teacher-applications.csv, GET /api/reports/platform/reward-approvals.csv, GET /api/reports/platform/token-payouts.csv, GET /api/reports/platform/wallet-credits.csv, and GET /api/reports/platform/delegated-permissions.csv.

Notification events

Notifications are persisted for key product events: course enrollment, content publication, platform/organization/course role assignment, terminal worker job failures, and reward records. Existing upload-processing notifications continue to use the video:* titles.

PostgreSQL

Typical Docker Compose values look like:

DATABASE_URL=postgres://your_username:your_password@db:5432/your_db_name
POSTGRES_DB=your_db_name
POSTGRES_USER=your_username
POSTGRES_PASSWORD=your_password

The Compose stack exposes PostgreSQL on host port 5433 and container port 5432.

S3-compatible storage

Local development uses RustFS through S3-compatible settings:

S3_ACCESS_KEY=rustfsadmin
S3_SECRET_KEY=rustfsadmin
S3_INTERNAL_DOMAIN=rustfs
S3_INTERNAL_PORT=9000
S3_EXTERNAL_DOMAIN=localhost
S3_EXTERNAL_PORT=9000
S3_INTERNAL_SCHEME=http
S3_EXTERNAL_SCHEME=http

RustFS API is exposed on port 9000; the console is exposed on port 9001.

Ethereum provider

The Compose stack runs Anvil on port 8545. App and worker containers override the provider URL to the Compose service:

ETH_HOST=anvil
ETH_PORT=8545
ETH_RPC_URL=http://anvil:8545

Startup deploys and persists LearnToken plus the wallet transfer helper contracts when they are missing. If you use pre-deployed contracts, configure LEARN_TOKEN_ADDRESS, WALLET_DEPOSIT_IMPORTER_ADDRESS, and the treasury receiver values in .env.

Bootstrap admin

The database setup code reads admin bootstrap values from the environment. Configure these in .env before first startup:

ADMIN_NAME=admin
ADMIN_EMAIL=admin@example.com
ADMIN_PASSWORD=replace-with-strong-admin-password
ADMIN_DATE_OF_BIRTH=1990-01-01

ADMIN_PASSWORD must be changed before startup. Bootstrap rejects known placeholder/default values and applies the same password policy as registration: at least 12 characters with lowercase, uppercase, numeric, and symbol characters.

Running with Docker Compose

Start the complete local stack through Make:

make dev

For foreground Compose logs, use the raw Compose command as an escape hatch:

docker compose up

make dev starts the current Compose images. After changing Rust API code, frontend dependencies, or Dockerfiles, rebuild the app/web images before testing the full userflow:

make dev-refresh

Set COMPOSE_REFRESH_SERVICES='app web worker' when worker image changes also need to be rebuilt for the same local run. The refresh target rebuilds only the selected services, so a web-only refresh does not also rebuild the Rust API image through Compose dependencies. Rust app and worker image builds use BuildKit Cargo cache mounts; the first rebuild after a Dockerfile or cache reset is still slow, but later source-only refreshes can reuse downloaded crates and compiled dependencies.

Default local service ports:

Service URL/port
Rust API http://localhost:8080
Next.js web app http://localhost:3000
PostgreSQL localhost:5433
RustFS S3 API http://localhost:9000
RustFS console http://localhost:9001
Anvil Ethereum RPC http://localhost:8545

The API container runs scripts/app-entrypoint.sh, which initializes submodules, waits/retries migrations, and starts the app when PROD_MODE=TRUE.

Host-run backend and frontend

The primary local stack is make dev. For a host-run API or worker, start only the shared dependencies first:

make dev-deps

This starts PostgreSQL, RustFS, and Anvil without starting the API, web app, or worker containers. Run the Rust API through the Make target:

make dev-run

Run the worker on the host:

make dev-worker

Run the frontend locally:

make web-dev

Worker service

The worker service processes background upload jobs and media transformations. It is built from docker/worker.Dockerfile and runs /usr/local/bin/worker directly to avoid runtime compilation.

Useful configuration:

WORKER_CONCURRENCY=1
WORKER_MAX_ATTEMPTS=5
WORKER_BASE_BACKOFF_SECONDS=60
WORKER_QUEUE_METRICS_INTERVAL_SECONDS=60
WALLET_DEPOSIT_INDEXER_ENABLED=true
WALLET_DEPOSIT_INDEXER_POLL_SECONDS=15
WALLET_DEPOSIT_INDEXER_IDLE_LOG_SECONDS=60
WALLET_DEPOSIT_INDEXER_CONFIRMATIONS=1
WALLET_DEPOSIT_INDEXER_BATCH_BLOCKS=500
WALLET_DEPOSIT_INDEXER_LOOKBACK_BLOCKS=100
LEARN_TOKEN_DECIMALS=18

The worker also runs the wallet deposit indexer. It scans LearnToken Transfer events for user-paid deposits into the configured treasury and PlatformImporter Imported events for platform-paid deposits. It advances its cursor in persistent state as wallet_deposit_indexer_next_block; use WALLET_DEPOSIT_INDEXER_START_BLOCK only for the first scan of a fresh environment. Empty polls are logged at WALLET_DEPOSIT_INDEXER_IDLE_LOG_SECONDS while polls that credit deposits are logged immediately. Poll failures log an initial retry notice at info level, escalate to warning only after the same interval has elapsed, and emit a recovery notice when polling succeeds again.

Recommended worker build/start flow:

make dev-deps
make dev-refresh COMPOSE_REFRESH_SERVICES=worker

If the worker build fails with an out-of-memory linker error, increase Docker VM memory and rebuild. With Colima, for example:

colima start --memory 8192
make worker-build

Inspect worker logs:

make logs SERVICE=worker

The worker writes /tmp/worker_alive; Docker Compose and Kubernetes run /usr/local/bin/worker-healthcheck against this heartbeat for health checks. Use make logs SERVICE=worker to watch heartbeat-adjacent metric events for queue depth, attempts, processing duration, retries, and failed jobs.

Testing and quality checks

Hosted GitHub Actions CI is intentionally disabled. Run the quality gates locally, preferably through Docker Compose, to avoid spending hosted CI minutes. Use Make targets as the primary development command surface; they encode the host wrapper, Compose service networking, and repo-specific defaults. Raw cargo, npm, docker compose, or Diesel commands are escape hatches when no Make target exists. Diesel CLI work should stay in Compose through make migrate, make migrate-redo, make schema, make migration-generate NAME=..., or make diesel-compose DIESEL_ARGS='...'.

Run Rust tests through Docker Compose service networking:

make test-compose

To pass a narrower Compose test filter through the Make target:

make test-compose CARGO_TEST_ARGS='--lib'
make test-compose CARGO_TEST_ARGS='--test authentication_flow'

Host tests are still available when you specifically want the host toolchain:

make test

For ad hoc host Cargo commands, use the host wrapper so Compose-only service names, native library paths such as Homebrew libpq, and target/host-tests are configured consistently:

./scripts/run-host-tests.sh cargo test --test authentication_flow

Business-flow verification through Docker Compose:

# Teacher application and central review queue.
make test-compose CARGO_TEST_ARGS='--test teacher_applications'

# Course enrollment and join-request reward prerequisites.
make test-compose CARGO_TEST_ARGS='--test course_enrollment_api'
make test-compose CARGO_TEST_ARGS='--test course_join_requests'

# Reward candidate submission, teacher approval, amount approval, fraud blocks,
# delegated permissions, student history, and reporting.
make test-compose CARGO_TEST_ARGS='--test reward_candidates'
make test-compose CARGO_TEST_ARGS='--test reward_fraud_blocks'
make test-compose CARGO_TEST_ARGS='--test reward_management_api'
make test-compose CARGO_TEST_ARGS='--test delegated_permissions'
make test-compose CARGO_TEST_ARGS='--test student_reward_history'
make test-compose CARGO_TEST_ARGS='--test reporting_exports'

# Wallet credit, notification idempotency, reconciliation, and transaction links.
make test-compose CARGO_TEST_ARGS='--test reward_execution'
make test-compose CARGO_TEST_ARGS='--test wallet_linking'
make test-compose CARGO_TEST_ARGS='--test notification_events'

# Anvil-backed token contract behavior.
make test-integration

Run blockchain integration tests:

make test-integration

Check formatting:

make fmt

If you need a one-off formatting check inside the test-runner container:

make fmt-compose

Frontend checks:

make web-lint
make web-api-helper-tests
make web-page-tests
make web-architecture-scan
make web-build

Docker Compose equivalent:

make web-lint-compose
make web-build-compose

web-lint-compose runs lint inside the bootstrapped Compose web service, so the anonymous node_modules volume is populated before ESLint runs. web-build-compose validates the production Dockerfile build path instead of running next build inside the development service environment.

Build the Linux ARM64 Compose images:

DOCKER_DEFAULT_PLATFORM=linux/arm64 docker compose build app web worker

This build was last verified locally for rust-learn-app, rust-learn-web, and rust-learn-worker. Kubernetes builds use separate rust-app and rust-worker images so the API runtime does not carry ffmpeg. The web image may report npm audit advisories during dependency installation; those advisories do not fail the image build.

Makefile shortcuts:

make fmt
make clippy
make preflight
make test
make test-integration
make web-dev
make web-lint
make web-api-helper-tests
make web-page-tests
make web-architecture-scan
make web-build
make web-lint-compose
make web-build-compose
make migrate
make migrate-redo
make schema
make migration-generate NAME=create_learning_paths
make diesel-compose DIESEL_ARGS='migration list'
make health
make runtime-verify
make runtime-log-scan
make runtime-disk
make docker-prune-build-cache

Test dependency notes:

Check External requirements
make preflight Local Rust and Node toolchains plus running Docker Compose and Kubernetes runtime environments; runs formatting, Clippy, frontend lint/API-helper tests/build, Kubernetes manifest rendering, runtime verification, and recent runtime log scanning. It intentionally does not run the full host or Compose test suites; use make test and make test-compose for those.
make clippy Local Rust toolchain plus a valid host Cargo environment; runs Clippy through the host wrapper across all targets and the app, worker, and tool feature flags with warnings denied.
make test A valid .env; many integration tests open DATABASE_URL, so start PostgreSQL first with make dev-deps when running the full suite. The host wrapper maps Compose-only service names to localhost ports, adds local native library paths such as Homebrew libpq, and uses target/host-tests so Docker and host artifacts do not collide.
make test-compose Docker plus a valid .env; starts PostgreSQL, RustFS, and Anvil, then runs Cargo in the test-runner profile so host native libraries are not required.
make web-lint-compose Docker; runs ESLint in a one-shot Compose web container after npm ci, so stale anonymous node_modules volumes cannot hide missing dependencies.
make web-api-helper-tests Local Node toolchain; runs frontend API helper contract tests for session/auth JSON success, text errors, 401, 403, timeout, network failure, and verification-token states.
make web-page-tests Local Node toolchain; runs fast Vitest page/route-level suites for ProductShell, admin, learner, organization, session, and teacher routes without Playwright, screenshots, Docker Compose, or persisted browser state.
make web-architecture-scan Local Node toolchain; reports frontend file-size, dense-line, API-boundary, and view-side-effect findings for the TODO/19 modularity migration.
make web-build-compose Docker; builds the web image through the production Dockerfile, which is the supported Compose production-build check for the frontend.
make runtime-log-scan Running Docker Compose stack and Kubernetes rust-learn namespace; scans recent app, worker, and web logs for warning/error patterns, explicit HTTP 500 statuses, status=500 fields, and standalone 500 status tokens without matching routine counters such as failed=0, config values such as batch_blocks=500, or timings such as 500ms. Override the window with LOG_SCAN_SINCE=10m. Fails if a matching log line is found or a required log source is unreachable.
cargo test --test s3 RustFS/S3-compatible storage reachable through the S3_* settings. With Compose, run from the container network or set S3_INTERNAL_DOMAIN/S3_EXTERNAL_DOMAIN appropriately for the host.
make test-integration Docker plus a valid .env; starts Anvil, then runs ignored blockchain tests in the test-runner profile.
./scripts/run-host-tests.sh cargo test --test blockchain_integration_tests -- --ignored Anvil or another Ethereum JSON-RPC endpoint plus ETH_MNEMONIC and provider settings in .env; the host wrapper supplies Compose-to-localhost rewrites, native library paths, and target/host-tests.
make runtime-verify Running Docker Compose stack and Kubernetes rust-learn namespace; fails if Compose endpoints, the product entry shell, the worker heartbeat, K8s deployments/pods, or in-cluster web/API readiness are unhealthy.
Worker/media-processing checks ffmpeg on PATH, PostgreSQL, and RustFS/S3. Keep WORKER_CONCURRENCY=1 on small Docker VMs.

Docker is the recommended way to provide PostgreSQL, RustFS, and Anvil for local test runs:

make dev-deps

Database migrations

Run migrations through the Make target. Make is the primary development interface: it starts the Compose PostgreSQL service, runs Diesel inside the Compose tool container, then refreshes the source-controlled schema file:

make migrate

No host Diesel CLI install is required. make migrate runs diesel migration run in the Compose tool container and then runs make schema, which executes diesel print-schema in the same tool container. The schema target writes that output to DIESEL_SCHEMA_FILE, formats it with the tool container's rustfmt, and defaults to src/infra/postgres/schema.rs, matching the diesel.toml print_schema.file setting. The generated schema is written through the mounted workspace.

Redo the latest migration:

make migrate-redo

Refresh only the generated schema file:

make schema

Generate a new migration directory through the same Compose Diesel image:

make migration-generate NAME=create_learning_paths

For less common Diesel commands, keep the Makefile as the entrypoint:

make diesel-compose DIESEL_ARGS='migration list'
make schema

Direct host Diesel invocations are not the normal development path.

When adding migrations, include reversible up.sql and down.sql files whenever possible and update/check src/infra/postgres/schema.rs through make migrate, make migrate-redo, or make schema when schema changes require it.

Troubleshooting

Diesel migration failures usually mean the API cannot reach PostgreSQL, the database credentials in .env do not match the Compose container, or a migration failed partway through. Check the database service first:

make ps
make logs SERVICE=db

When running the API on the host, DATABASE_URL should point at localhost:5433; inside Compose it should point at db:5432. After fixing the connection string or database state, rerun:

make migrate

S3 or RustFS connectivity errors usually come from using container-only hostnames from the host, mismatched credentials, or RustFS not being ready. Check the service and console:

make ps
make logs SERVICE=rustfs

Containers should use S3_INTERNAL_DOMAIN=rustfs. A host-run API or worker should use S3_INTERNAL_DOMAIN=localhost with S3_INTERNAL_PORT=9000 unless it is attached to the Compose network.

Ethereum RPC startup issues usually mean Anvil is still starting, the wrong host name is configured, or an old local state volume is being reused. Check Anvil and query the chain ID:

make ps
make logs SERVICE=anvil
curl -s -X POST -H 'Content-Type: application/json' \
  --data '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}' \
  http://localhost:8545

Containers should use ETH_HOST=anvil or ETH_RPC_URL=http://anvil:8545. A host-run API should prefer ETH_RPC_URL=http://localhost:8545.

Worker builds and ffmpeg processing can be memory-heavy. Keep WORKER_CONCURRENCY=1 on small machines, increase the Docker VM memory when release builds fail, and inspect worker logs before raising concurrency:

make logs SERVICE=worker
colima start --memory 8192
make worker-build

Docker and Minikube can also run out of disk after repeated local image builds. Check runtime storage before long Compose or Kubernetes verification runs:

make runtime-disk

If the build cache is the pressure source, prune only Docker build cache without removing images, containers, or volumes:

make docker-prune-build-cache

Kubernetes

Kubernetes manifests live under k8s/. Common commands:

make k8s-build
make k8s-validate
make k8s-apply
make k8s-dev-refresh-app
make k8s-dev-refresh-web
make k8s-status
make k8s-logs SERVICE=rust-app
make k8s-forward SERVICE=web PORT=3000
make k8s-forward SERVICE=web PORT=3000 LOCAL_PORT=33030
make k8s-delete

Documentation

  • VISION.md — mission, architecture direction, strategic pillars, and near-term outcomes.
  • TODO/ — prioritized implementation roadmap.
  • PERMISSIONS.md — current permissions assigned to platform, organization, and course roles.
  • THIRD_PARTY.md — third-party Rust crates, Docker images, and external tools.
  • AGENTS.md — repository guidance for AI agents and contributors.

Contributing notes

Before opening a pull request:

  1. Keep patches focused and documented.
  2. Run make fmt and relevant tests.
  3. Update README/TODO/PERMISSIONS/environment docs when behavior, setup, or permissions change.
  4. Do not commit .env, private keys, target/, or web/node_modules/.

If you need help or want to propose a larger direction change, open an issue with the problem statement, expected behavior, and any operational constraints.

About

No description or website provided.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors