104 lines
5.3 KiB
Markdown
104 lines
5.3 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Overview
|
|
|
|
Proxmox VE homelab cluster running Docker Swarm services. Three Proxmox hosts connected via VXLAN overlay on vmbr1 (10.10.10.0/24). VMs run Docker CE and form a swarm with pve-postgres as manager.
|
|
|
|
## Hardware
|
|
|
|
| Node | Machine | RAM | Storage | Role |
|
|
|------|---------|-----|---------|------|
|
|
| pve (Meerkat) | Intel TNTGL357 mini PC | 64GB | 4TB NVMe | Proxmox host, data tier |
|
|
| adder | System76 Adder WS | 32GB | 2TB NVMe + RTX 2070 | Proxmox host, GPU/crypto tier |
|
|
| game (CyberPower) | Old gaming PC | 16GB | 500GB NVMe + 2TB HDD | Proxmox host |
|
|
| Synology NAS 425+ | — | — | 48TB usable | NFS backend (VM backups, ISOs, Nextcloud data — NOT live DBs) |
|
|
|
|
Primary workstation (i9-13900KF, 96GB, RTX 4090) is **standalone — not in the Proxmox cluster**. Joins as NATS leaf node for LLM inference.
|
|
|
|
## Cluster Topology
|
|
|
|
| Node | Host IP | Swarm IP | Role |
|
|
|------|---------|----------|------|
|
|
| pve | 192.168.40.198 | 10.10.10.1 | Proxmox host |
|
|
| adder | 192.168.40.150 | 10.10.10.86 | Proxmox host |
|
|
| game | 192.168.40.109 | 10.10.10.172 | Proxmox host |
|
|
| pve-postgres | — | 10.10.10.2 | VM on pve, swarm manager |
|
|
| pve-tools | — | 10.10.10.3 | VM on pve, swarm worker |
|
|
| pve-social | — | 10.10.10.4 | VM on pve, swarm worker |
|
|
| adder-ghost | — | 10.10.10.20 | VM on adder, swarm worker |
|
|
|
|
All VMs on vmbr1 (internal, not directly LAN-reachable). A gateway LXC on pve is dual-homed vmbr0+vmbr1 and handles WireGuard + iptables DNAT.
|
|
|
|
## VM Allocation (Node 1 / pve — current plan)
|
|
|
|
```
|
|
LXC: gateway 256MB — WireGuard client, iptables DNAT, SSH jump host
|
|
VM: infra 8GB — NATS/JetStream, n8n
|
|
VM: data 20GB — PostgreSQL (shared: Ghost, Forgejo, Synapse, all apps)
|
|
VM: apps 8GB — Ghost instances
|
|
VM: social 24GB — Forgejo, Synapse (Matrix), Snikket (XMPP); 1TB NVMe, 4 cores
|
|
VM: redis 14GB — Redis (own VM required — needs vm.overcommit_memory=1 kernel tuning)
|
|
VM: nextcloud 8GB — Nextcloud AIO (manages its own isolated internal Postgres)
|
|
```
|
|
|
|
No MariaDB/MySQL anywhere — PostgreSQL only. Redis must be a VM (not LXC) for kernel tuning.
|
|
Nextcloud's internal DB must stay isolated — opinionated AIO lifecycle, upgrade migrations must not affect other app DBs.
|
|
|
|
## NFS / Database Rules
|
|
|
|
- **NFS is NOT for live database files.** Use local NVMe for PostgreSQL, Redis, Neo4j, monerod/zanod data.
|
|
- LMDB-based services (monerod, zanod) are especially NFS-hostile — mmap is the architecture.
|
|
- PostgreSQL tolerates NFS better (WAL + buffer pool) but still prefer local.
|
|
- NAS use: VM backups (PBS encrypted), ISOs, Nextcloud user data, blockchain snapshots only.
|
|
|
|
## Deployment
|
|
|
|
Active service definitions: `proxmox/services/`. The `services/` directory contains older pre-migration copies.
|
|
|
|
**Preferred: use `deploy-stack.sh`** (in `~/private/Knowledge/Homelab/deploy-stack.sh`) — scans the compose file for `${VAR}` references, pulls secrets from `pass` or KeePassXC, writes an ephemeral chmod-600 `.env`, deploys, then nukes `.env` on exit via trap.
|
|
|
|
```bash
|
|
# Deploy using pass backend (default)
|
|
./deploy-stack.sh proxmox/services/<service>.yml <stack_name>
|
|
|
|
# Deploy to remote swarm manager
|
|
./deploy-stack.sh proxmox/services/<service>.yml <stack_name> --host pve-postgres
|
|
|
|
# Remove a stack
|
|
ssh pve-postgres "docker stack rm <stack_name>"
|
|
```
|
|
|
|
Secrets live in `pass` under `homelab/<VARNAME>`. Never hardcode secrets in files.
|
|
|
|
## Service YML Conventions
|
|
|
|
- All use `version: '3.8'` and Docker Swarm deploy mode
|
|
- Each service pinned to a specific node via `node.hostname` constraint
|
|
- All connect to external overlay network `homelab-net` via `${OVERLAY_NETWORK:-homelab-net}`
|
|
- Named volumes for persistence, `on-failure` restart policy with 3 max attempts
|
|
- Every service with a web UI or external connections must publish its ports
|
|
- Only include environment variables required to start the service — no optional/nice-to-have vars
|
|
|
|
## SSH Access
|
|
|
|
Use SSH config aliases (`pve-postgres`, `pve-tools`, `adder-ghost`, `pve`, `adder`, `game`). ProxyJump is configured in `~/.ssh/config`. Never manually hop through intermediate nodes.
|
|
|
|
## External Infrastructure
|
|
|
|
- **BuyVM (Luxembourg):** VPS + 1TB block storage, $7/month, Monero payment. Runs: Peertube, Ghost+BTCPay, Nextcloud, Jitsi, Caddy, WireGuard server.
|
|
- **Hetzner AX102 (pending):** Bare metal Proxmox. One VM runs Caddy + WireGuard server — terminates TLS for public subdomains, tunnels traffic to home cluster via WireGuard.
|
|
- **Njalla domains:** Accepts Monero, Swedish jurisdiction.
|
|
|
|
## Planned Services
|
|
|
|
NATS JetStream (3-replica cluster service), PostgreSQL, Redis, Neo4j, FusionAuth, Authentik, n8n, Nextcloud, Garage (object storage), Ghost, Forgejo, Synapse (Matrix), Snikket (XMPP), monerod, monero-wallet-rpc, zanod, Peertube, BTCPay, Jitsi, Caddy, WireGuard.
|
|
|
|
## Key Files
|
|
|
|
- `proxmox/services/*.yml` — Active swarm stack definitions
|
|
- `proxmox/services/nats.conf` — NATS server config (JetStream, websocket, monitoring)
|
|
- `proxmox/services/01-init.sql` — Postgres init script (creates users/databases)
|
|
- `proxmox/post_init_node.org` — Fresh Proxmox node setup steps
|
|
- `~/private/Knowledge/Homelab/deploy-stack.sh` — Preferred deployment script
|