From c2625318d23f91ead3adf7b9a303887ff9302914 Mon Sep 17 00:00:00 2001 From: jwinterm Date: Thu, 9 Jul 2026 17:22:48 -0400 Subject: [PATCH] docs(readme): remove em-dashes + stray bold; update to deployed/PMMR-index state --- README.md | 226 ++++++++++++++++++++++++++---------------------------- 1 file changed, 107 insertions(+), 119 deletions(-) diff --git a/README.md b/README.md index 3116043..dc4df8d 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,21 @@ -# grin-lws — Grin light-wallet-server +# grin-lws: Grin light-wallet-server -A **real** light-wallet-server for Grin, mirroring [monero-lws]: register a -view credential, run a **background scanner** over the Grin chain, **store** -recognized outputs (with recovered derivation paths) in a **database**, and -serve balance / unspent-outputs / broadcast / height fast from that store. +A light-wallet-server for Grin, mirroring [monero-lws]: register a view +credential, run a background scanner over the Grin chain, store recognized +outputs (with recovered derivation paths) in a database, and serve +balance, unspent-outputs, broadcast, and height fast from that store. -It is the Grin analogue of monero-lws: a wallet backend proxies to it exactly the -way a backend proxies to the Monero LWS — forward a view credential, serve the -stored results back to the wallet. +It is the Grin analogue of monero-lws. A wallet backend proxies to it the same +way it proxies to a Monero LWS: forward a view credential, serve the stored +results back to the wallet. -**Status: FUNCTIONAL.** The view-only rangeproof rewind, the PMMR-index node -reads, and the background scanner (discover / spend-reconcile / reorg / backfill) -are all implemented and tested. The rewind is cross-validated against grin's own -reference `ProofBuilder` and passed an adversarial money-safety review; an -end-to-end test drives the real scanner against a mock grin node (modeled on the -Foreign API v2) over both SQLite and Postgres. Remaining before production: a -smoke test against a live grin node with a funded wallet. See -[Build order](#build-order). +Status: functional and running in production. The view-only rangeproof rewind, +the PMMR-index node reads, and the background scanner (discover, spend-reconcile, +reorg, backfill) are implemented and tested. The rewind is cross-validated +against grin's own reference `ProofBuilder` and passed an adversarial money-safety +review. An end-to-end test drives the real scanner against a mock grin node +(modeled on Foreign API v2) over both SQLite and Postgres, and it has been +validated against a live grin node with a funded wallet. [monero-lws]: https://github.com/vtnerd/monero-lws @@ -24,32 +23,31 @@ smoke test against a live grin node with a funded wallet. See ## Why this exists (and why it needs a DB) -A common wallet-backend approach is an **on-demand, stateless** grin scan: each +A common wallet-backend approach is an on-demand, stateless grin scan: each request forwards a `rewind_hash` to grin-wallet's `scan_rewind_hash`, which -rescans the chain and returns matches, **storing nothing**. That works for -Grin's small UTXO set, but it is *not* a light-wallet-server — every read -re-pays the scan cost, and it has no output store to recover derivation paths -from. +rescans the chain and returns matches, storing nothing. That works for Grin's +small UTXO set, but it is not a light-wallet-server: every read re-pays the scan +cost, and it has no output store to recover derivation paths from. -A real LWS — like monero-lws — keeps a **per-account output store** maintained by -a **background scanner**, so client reads are O(rows) not O(chain), and it can -return each output's **derivation path**. That store is a database. Hence this -service, and hence the three tables in `migrations/`. +A real LWS, like monero-lws, keeps a per-account output store maintained by a +background scanner, so client reads are O(rows) not O(chain), and it can return +each output's derivation path. That store is a database, hence this service and +the three tables in `migrations/`. -The payoff for the wallet: `get_unspent_outs` returns outputs *with* `key_id` / -`n_child`, so the client spends directly with **no client-side identify search** -(it closes the path-identification gap that on-demand scan leaves open). +The payoff for the wallet: `get_unspent_outs` returns outputs with `key_id` and +`n_child`, so the client spends directly, with no client-side identify search. It +closes the path-identification gap that an on-demand scan leaves open. -**Non-custodial, always.** An account is a `rewind_hash` — a *view credential* -derived from the wallet's public root key (Grin has had view/spend separation -since 2021). It can recognize outputs and read amounts but **cannot spend**. This +Non-custodial, always. An account is a `rewind_hash`, a view credential derived +from the wallet's public root key (Grin has had view/spend separation since +2021). It can recognize outputs and read amounts, but it cannot spend. This service never stores a private or spend key. --- ## Architecture -Three parts, matching monero-lws's user-api + admin-api + scanner: +Three parts, matching monero-lws's user-api, admin-api, and scanner: ``` ┌──────────────────────────────────────────┐ @@ -59,10 +57,10 @@ Three parts, matching monero-lws's user-api + admin-api + scanner: │ rewind_hash -> accounts row │ │ │ │ 2. SCANNER (background task) │ - │ follow tip -> per block: │ - │ rewind outputs vs each account, │ - │ store matches + spend detection, │ - │ reorg-safe chain_cursor │ + │ walk the output PMMR by index, │ + │ rewind each output vs each account, │ + │ store matches, spend-reconcile, │ + │ reorg-safe chain_cursor │ │ │ │ │ ▼ │ │ 3. USER API ┌─────────┐ │ @@ -71,138 +69,124 @@ Three parts, matching monero-lws's user-api + admin-api + scanner: └───────────────────────┬──────────────────┘ │ grin node (Foreign API v2) - get_tip / get_block / push_transaction ``` -1. **Registration** — `POST /register {rewind_hash, start_height?}` inserts an - `accounts` row that the scanner will follow. Idempotent (the monero-lws +1. Registration: `POST /register {rewind_hash, start_height?}` inserts an + `accounts` row that the scanner will follow. It is idempotent (the monero-lws `add_account` analog, but self-service and view-only, so no spend risk). -2. **Background scanner** (the hard part; its own long-lived task) — follows the - chain tip. For each new block: rewind each output's rangeproof against every - registered `rewind_hash`; on a match, store the output **and** the derivation - path recovered from the proof message; detect spends by matching input - commitments against stored outputs; maintain a reorg-safe `chain_cursor` - (roll back rows above the fork on a reorg). -3. **User API** — reads served from the DB store (fast), plus broadcast + tip. +2. Background scanner: Grin's chain state is the set of unspent outputs in a + Merkle Mountain Range, so the scanner walks that set in PMMR insertion-index + order (not block by block; `get_block` does not even return rangeproofs). Each + tick pages new outputs from the cursor via `get_unspent_outputs` (with + proofs), rewinds each against every registered `rewind_hash`, and on a match + stores the output together with the derivation path recovered from the proof + message. Spends are detected by absence: a stored commit the node no longer + lists has been spent. A reorg-safe `chain_cursor` rolls back rows above the + fork. +3. User API: reads served from the DB store (fast), plus broadcast and tip. --- ## HTTP surface -The **user API** is unauthenticated at this layer — grin-lws binds to loopback / -a private network and trusts the proxying backend (which does JWT auth) as its -sole caller. The **admin API** is gated by a shared bearer (`GRINLWS_ADMIN_KEY`) -and disabled when that is unset. +The user API is unauthenticated at this layer: grin-lws binds to loopback or a +private network and trusts the proxying backend (which does JWT auth) as its sole +caller. The admin API is gated by a shared bearer (`GRINLWS_ADMIN_KEY`) and +disabled when that is unset. ### User API (mirrors monero-lws user endpoints) | Route | Body | Returns | |-------|------|---------| | `POST /register` | `{rewind_hash, start_height?}` | `{registered, new_account, scan_height, start_height}` | -| `POST /get_balance` | `{rewind_hash}` | `{total, count, scanned_height, blockchain_height}` | +| `POST /get_balance` | `{rewind_hash}` | `{total, unlocked, count, scanned_height, blockchain_height}` | | `POST /get_unspent_outs` | `{rewind_hash}` | `{outputs[], blockchain_height}` | | `POST /submit_raw_tx` | `{tx}` | `{ok}` | -| `GET /height` | — | `{height}` | -| `GET /health` | — | liveness (always 200 if up) | -| `GET /ready` | — | readiness (503 if the node is unreachable) | +| `GET /height` | | `{height}` | +| `GET /health` | | liveness (always 200 if up) | +| `GET /ready` | | readiness (502 if the node is unreachable) | -`get_unspent_outs` outputs — **crucially with the recovered path**: +`get_unspent_outs` outputs carry the recovered path: ```jsonc { "commit": "09…", "value": 1000000000, "height": 1851234, "mmr_index": 987654, "is_coinbase": false, "lock_height": 0, - "key_id": "0300000000…", "n_child": 7 } // ← direct-spend, no client identify + "spendable": true, "key_id": "0300000000…", "n_child": 7 } ``` +`key_id` is the authoritative derivation identifier, so the client spends without +a local identify search. `spendable` is `lock_height <= blockchain_height`, so an +immature coinbase is never presented as spendable. ### Admin API (bearer-gated) | Route | Body | Purpose | |-------|------|---------| -| `POST /list_accounts` | — | enumerate registered `rewind_hash`es | -| `POST /rescan` | `{rewind_hash, height}` | **backwards-only** re-derive from `height` | +| `POST /list_accounts` | | enumerate registered `rewind_hash`es | +| `POST /rescan` | `{rewind_hash, height}` | backwards-only re-derive from `height` | -`/rescan` is backwards-only (mirrors the monero-lws rescan invariant): it clears -stored outputs at/above `height` and lowers `scan_height`, so the scanner -re-derives them. A rescan to a height `>=` the current scan height is rejected as -a no-op. +`/rescan` is backwards-only (it mirrors the monero-lws rescan invariant): it +clears stored outputs at or above `height` and lowers `scan_height`, so the +scanner re-derives them. A rescan to a height at or above the current scan height +is rejected as a no-op. --- ## Database schema -See `migrations/0001_init.sql`. It runs **unchanged on both PostgreSQL and -SQLite** (verified in a test). **Never** stores a private/spend key. +See `migrations/0001_init.sql`. It runs unchanged on both PostgreSQL and SQLite +(verified in a test). It never stores a private or spend key. ``` accounts(rewind_hash PK, start_height, scan_height, created_at) outputs("commit" PK, rewind_hash → accounts, value, height, mmr_index, is_coinbase, lock_height, key_id, n_child, spent, spent_height) INDEX(rewind_hash, spent), INDEX("commit") -chain_cursor(id=1 PK, height, block_hash) -- reorg-safe resume +chain_cursor(id=1 PK, output_mmr_index, height, block_hash) ``` -* `outputs.key_id` / `n_child` are recovered during rewind — the direct-spend - enabler. -* `outputs.spent` / `spent_height` are set when a stored output's commitment - appears as a block input. -* `chain_cursor` is the single-row global resume point; on a reorg the scanner - rolls back rows above the fork and reseeks from it. +Notes: +- `outputs.key_id` and `n_child` are recovered during rewind; they are the + direct-spend enabler. +- `outputs.value` is stored as a decimal string, because sqlx's `Any` driver + truncates i64 to i32 for SQLite. Amounts are summed in Rust. +- `outputs.spent` and `spent_height` are set when a stored commit leaves the + node's unspent set. +- `chain_cursor` holds the scanner's resume point: `output_mmr_index` is how far + it has walked the output PMMR; `height` and `block_hash` are the chain tip it + last saw, the reorg checkpoint. --- ## How a wallet backend proxies to it -Identical wiring to a Monero LWS client — forward the view credential, serve the -stored results: +Wiring identical to a Monero LWS client: forward the view credential, serve the +stored results. ``` wallet ──JWT──▶ wallet backend ──private──▶ grin-lws ──▶ grin node /wallet/grin/scan /get_unspent_outs (Foreign API v2) ``` -1. Add a grin-lws client config (`lws_url`, optional `admin_url` + `admin_key`) +1. Add a grin-lws client config (`url`, optional `admin_url` and `admin_key`) alongside your existing Monero-LWS config. -2. Add an HTTP client mirroring your Monero-LWS client (shared `reqwest::Client`, - timeouts, size-capped hostile-response reads, a `Secret`-wrapped admin key). +2. Add an HTTP client mirroring your Monero-LWS client: a shared `reqwest::Client`, + timeouts, size-capped hostile-response reads, and a redacted admin key. 3. Behind a feature flag, have your grin scan endpoint forward the `rewind_hash` - to grin-lws `/get_unspent_outs` (returning outputs **with paths**) instead of + to grin-lws `/get_unspent_outs` (which returns outputs with paths) instead of driving grin-wallet `scan_rewind_hash` on-demand. 4. Register at balance-fetch time by forwarding `rewind_hash` to grin-lws `/register` (idempotent), exactly as you await LWS registration for Monero. +5. Trust grin-lws only once it is synced (`scanned_height` near + `blockchain_height`); while an account is still backfilling, fall back to the + authoritative grin-wallet scan so a not-yet-scanned account never reports a + false zero. -**Non-custodial guarantee preserved end to end:** the backend forwards only the -`rewind_hash` and stores nothing; grin-lws holds the `rewind_hash` for scanning -just as monero-lws holds the view key. No spend key ever leaves the wallet. +The non-custodial guarantee is preserved end to end: the backend forwards only +the `rewind_hash` and stores nothing; grin-lws holds the `rewind_hash` for +scanning, just as monero-lws holds the view key. No spend key ever leaves the +wallet. -Once grin-lws is wired in, the wallet client can **drop** any client-side -output-identify helper — `/get_unspent_outs` returns paths directly. - ---- - -## Build order - -Ship in stages; the deployed on-demand `/wallet/grin/scan` keeps serving balance -throughout, so nothing here blocks the client refactor. - -1. **Scaffold.** ✅ Routes, DTOs, config, DB layer, migrations, health/readiness. -2. **Rangeproof rewind (the core milestone).** ✅ `grin::rewind_output`: compute - the rewind nonce from `rewind_hash` + commitment, rewind the Bulletproof, and - parse the proof message to recover `(value, key_id, n_child)` — VIEW-ONLY, the - server-side analog of `grin_recover_output`. Cross-validated against grin's - reference `ProofBuilder`; adversarial money-safety review passed. (v3 outputs - only — legacy needs the master secret, same as grin-wallet's view-only scan.) -3. **Grin node reads.** ✅ `get_unspent_outputs` (PMMR-index, with proofs), - `get_outputs` (spend-by-absence), `get_pmmr_indices`, `get_header`, `get_tip`, - `push_transaction` — all against the Foreign API v2, defensively parsed. -4. **Scanner.** ✅ The `tick` loop: page the UTXO set from the cursor, rewind per - caught-up account, `insert_output` on matches, spend-reconcile by absence, - advance the cursor + account scan heights, coinbase-maturity/lock read path. -5. **Reorg + backfill.** ✅ Detect via header-hash mismatch at the checkpoint, - bounded `rollback_to`, reseek; backfill a new account from its birthday index. -6. **Live-node smoke test.** ⏳ Point at a real grin node with a funded wallet and - confirm balance/outputs match grin-wallet's `scan_rewind_hash`. -7. **Backend proxy.** ⏳ Land a `GrinLwsClient` in a wallet backend behind a - feature flag; parity-check against the on-demand path; retire the embedded - grin-wallet coupling. +Once grin-lws is wired in, the wallet client can drop any client-side +output-identify helper: `/get_unspent_outs` returns paths directly. --- @@ -213,7 +197,7 @@ throughout, so nothing here blocks the client refactor. # Postgres: createdb grinlws && psql grinlws < migrations/0001_init.sql # SQLite: sqlite3 grin-lws.db < migrations/0001_init.sql # 2. Configure. -cp .env.example .env # edit DATABASE_URL + GRIN_NODE_URL +cp .env.example .env # edit DATABASE_URL and GRIN_NODE_URL # 3. Run. cargo run # binds GRINLWS_BIND (default 127.0.0.1:3480) curl localhost:3480/health @@ -221,21 +205,25 @@ curl localhost:3480/health `/height`, `/submit_raw_tx`, and `/ready` require a reachable grin node (they return `502` otherwise). `/register`, `/get_balance`, and `/get_unspent_outs` -read from the DB store, which the background scanner fills once it has a grin -node to page the output set from. `/health` always works. +read from the DB store, which the background scanner fills once it has a grin node +to page the output set from. `/health` always works. + +On a fresh database the scanner first walks the whole unspent-output set (a large, +one-time download), then keeps up incrementally. Register accounts with their +wallet birthday (`start_height`), not from genesis, so each account's backfill +stays small. ## Deployment -Run as a private service alongside a grin node and its database. Bind to -loopback / a private network; expose it only to the proxying backend. Add a -readiness probe on `/ready`. Everything is env-configured — no code changes to -deploy. +Run as a private service alongside a grin node and its database. Bind to loopback +or a private network, and expose it only to the proxying backend. Add a readiness +probe on `/ready`. Everything is env-configured, so no code changes to deploy. --- -## License & provenance +## License and provenance -MIT — see [`LICENSE`](LICENSE). +MIT; see [`LICENSE`](LICENSE). grin-lws was extracted from the [Smirk](https://smirk.cash) wallet's backend and generalized into a standalone service that any wallet backend can run.