Bud

Features

• Catch-up Sweep workflow published to n8n (ID: NXwPAwurVz56Mfdx) — runs every 30 minutes, fetches up to 25 emails from Gmail starting 1h before the watermark in email_log
• Watermark logic: queries MAX(received_at) from email_log, subtracts 1h safety buffer, builds after:YYYY/MM/DD Gmail filter — incremental by design, idempotent on re-run
• Ollama classification (qwen2.5:3b) wired into the sweep — same Parse Classification logic as the main pipeline, handles code fences and "null" string normalization
• Historical backfill workflow ported to Ollama (gmail-historical-backfill.json v3) — covers Apr 17–May 16 gap period, manual-trigger, limit 50

Bug Fixes

• Prefer: resolution=ignore-duplicates header alone returned 409 on duplicate inserts — PostgREST requires ?on_conflict=message_id as a URL query param in addition to the Prefer header; added to both sweep and backfill Log Unclassified nodes

Infrastructure

• n8n CLI workflow import: docker cp file.json n8n:/tmp/ && docker exec n8n n8n import:workflow --input=/tmp/file.json — usable when n8n API key is unavailable or the workflow was never registered
• Bud CLAUDE.md updated: Gmail OAuth re-auth note added (SSH tunnel pattern: ssh -L 5678:localhost:5678 root@192.168.1.152), Steps 6-7 marked complete
• Historical gap validated via direct Supabase row count: 78 rows in email_log for Apr 24–May 16; first backfill batch (50 emails) returned all 409 duplicates, confirming the Jul 5 work had already covered the period

Lessons

• PostgREST resolution=ignore-duplicates needs two things: the Prefer header AND ?on_conflict=<column> in the URL — the header alone silently returns 409 and the error looks identical to a real constraint violation
• Expired Gmail OAuth in n8n manifests as ETIMEDOUT on a Google IP — looks like a network failure, not a credential error; check n8n Credentials before debugging connectivity
• A gap in email_log doesn't mean a gap in coverage — the Jul 5 backfill had already filled Apr 24–May 16; first backfill run hitting 409 on all 50 rows was the confirmation signal, not a failure

Features

• Extraction backend swapped from Claude Haiku to local Ollama (qwen2.5:3b on CT 100:11434) — zero API costs, inference stays on the homelab
• All 24 purchase emails dropped during the API credit outage recovered — 39 active purchases now in Supabase, zero errors remaining
• _fix_null_strings() added to ollama_client.py — walks the parsed JSON dict before Pydantic validation to replace "null" strings with Python None
• HTML fallback in n8n "Build Handler Payload": when plain-text body is under 500 characters, strips and uses HTML body instead — small models extract nothing useful from 50-word plain-text bodies

Bug Fixes

• n8n "Build Handler Payload" read email.payload.headers (raw Gmail API format) — n8n's Gmail node returns a parsed object with top-level fields (from.value[0], subject, text); rebuilt JS to match the actual output shape
• n8n HTTP Request timeout (300s) fired before Ollama processed deep-queue items — 24 concurrent requests queue sequentially; item 24 waits 23×67s ≈ 25 min before Ollama starts it; raised to 1800s in n8n and httpx
• docker restart was used after code edits — command reuses the baked image and silently runs old code; switched to docker compose up --build -d

Infrastructure

• src/services/ollama_client.py — new extraction service using httpx against Ollama's /v1/chat/completions endpoint; 1800s timeout; JSON fence stripping; null-string normalization
• src/config.py updated: anthropic_api_key now optional (defaults to ""); ollama_url and ollama_model added
• Migrations 003 (unsubscribe support) and 004 (amount nullable on purchases) applied
• scripts/find_dupes.py — one-shot soft-delete deduplication script; idempotent; dry-run flag
• workflows/backfill/ — n8n recovery workflow JSON export and Python backfill runner

Lessons

• docker restart reuses the baked image — Python file changes are invisible until docker compose up --build -d; the container logs look healthy the whole time
• n8n manual-trigger workflows cannot be triggered via REST API; POST /api/v1/workflows/{id}/run returns "POST method not allowed"; the UI Execute button is the only path
• Ollama queues inferences sequentially — n8n's HTTP timeout counts from when the request is sent, not when Ollama starts it; large concurrent batches will always exceed any reasonable n8n timeout
• Small models (qwen2.5:3b) emit "field": "null" as a string, not JSON null — Pydantic's date and float parsers reject it; the fix is a pre-validation dict walk, not a prompt change

TODO

• Bud Web UI — Step 1 scaffold at bud.understorylabs.co (Split Brain: Vercel reads, Taproot acts); plan at ~/.claude/plans/bud-web-ui-plan.md

Features

• Gmail purchase router workflow published and active in n8n — polls inbox every minute for new emails
• Classification pipeline end-to-end: Gmail trigger → Claude Haiku classify → confidence-based routing → handler POST or Supabase log
• Read status filter set to all emails (read + unread) — prevents missed classification if email is opened before n8n polls

Bug Fixes

• Handler crashed on emails where Claude returned null for amount — PurchaseExtraction.amount was required, now optional
• email_log audit trail never written when extraction failed — crash happened in extract() before store() was reached; moved email_log write before compliance checks
• Handler returned 200 with error body but route still reported status: "ok" — now returns "partial" when email is logged but no purchase row created
• Deployed model mismatch: EmailPayload on docker-host was missing unsubscribe_link and is_read fields — scp'd the updated model file

Infrastructure

• Lost session reconstructed from git state and uncommitted files — committed as dab51e7
• Google Cloud OAuth, n8n Gmail credential, and workflow import confirmed already complete from prior session
• Gmail Trigger node had stale "Every Day" poll entries from JSON import — removed, set to Every Minute
• Deployment confirmed: scp to /opt/bud/ on docker-host, docker compose build && up -d

Lessons

• When deploying via scp, every changed file must be pushed — not just the files edited in the current session; a model file mismatch caused an AttributeError that only surfaced at runtime
• n8n workflow JSON imports can carry orphaned poll configuration entries that prevent activation — the .trim() error gave no indication which node or field was the problem
• Handler pipelines should write the audit log (email_log) unconditionally at the top of store(), not after validation — a crash in extract() leaves zero trace otherwise

TODO

• Confirm end-to-end with a real purchase email landing in both email_log and purchases tables — fix deployed but not yet validated

Features

• PurchaseHandler local test passed — 3/3 email formats (Amazon, DoorDash, Netflix) extracted and written to Supabase
• Handler deployed on docker-host — FastAPI container running at 192.168.1.153:8001, health endpoint confirmed from network
• n8n operational — CT 102 (192.168.1.152:5678) created, Docker installed, n8n running and survives reboot
• Gmail API enabled on Google Cloud project life-dashboard — OAuth credentials next

Bug Fixes

• Claude Haiku wraps JSON extraction responses in ```json ``` fences despite explicit instructions — stripped before model_validate_json() in claude_client.py

Infrastructure

• Supabase schema live — email_log and purchases tables with RLS, indexes, and soft delete
• .env.example committed — documents SUPABASE_URL, SUPABASE_SERVICE_KEY, ANTHROPIC_API_KEY
• services.md created at ~/.claude/services.md — global inventory of cloud platforms, APIs, and self-hosted services; updated by /wrap going forward
• n8n compose file uses N8N_SECURE_COOKIE=false — required for HTTP access on local network

Lessons

• Smaller models (Haiku) reliably ignore "no markdown formatting" instructions — fence stripping is a required defensive layer, not optional
• scp with ~ fails in PowerShell; full Windows paths (C:\Users\nrisa\...) required — write files locally and copy over rather than fighting heredocs or rsync
• Services inventory needs to exist before the second integration, not after — build the index early so "do I already have a GCP project?" has a real answer

TODO

• Step 5: create Google OAuth 2.0 credentials in life-dashboard, wire Gmail trigger in n8n
• Add bud to save-state project registry via /ship

Features

• PurchaseHandler complete — extract → validate → store pipeline wired to /api/v1/handle/purchase
• Extraction prompt with three few-shot examples (Amazon order, DoorDash delivery, Netflix subscription) — Haiku model, JSON-only output
• HandlerResponse returns email_log_id and purchase_id on success — n8n gets traceable IDs, not just a status string
• amount > 0 and non-empty vendor validation gates every write — no silent bad data reaches Supabase

Infrastructure

• email_log and purchases tables live in Supabase — RLS enabled, dedup index on message_id, FK from purchases into email_log
• Migration files in migrations/ numbered 001/002 — ordering enforces the FK dependency at run time
• Sync Supabase client wrapped in asyncio.to_thread — avoids supabase-py async API fragility across 2.x minor versions
• email_log row written before purchases insert — if the purchases write fails, the audit trail exists and carries the error

Lessons

• Supabase service_role bypasses RLS by design — adding a write policy is redundant and misleading; only anon/authenticated reads need explicit policies
• supabase-py async API changed names between 2.x minor versions (acreate_client vs create_async_client) — sync + asyncio.to_thread is the portable fix
• Prompt templates with JSON few-shot examples can't use .format() — JSON braces read as format variables; .replace() per slot avoids the trap
• Write the audit log row first, then the derived row — partial failures are recoverable; a missing anchor row is not

TODO

• Step 2 checkpoint: venv setup, .env, curl tests with three email formats against localhost
• Steps 3–6 after checkpoint: Docker deployment on docker-host, n8n LXC, Gmail trigger workflow, end-to-end validation