Add debug logs for each tick + forward-scan summary (accounts, caught-up,
index range, scanned/matched counts) and info logs when a new account's
backfill starts and completes. No per-output or commitment logging (keeps
amounts/commits out of logs). Added while validating against a live grin
node + a funded wallet: grin-lws recovered the wallet's exact balance via
both the forward-scan and backfill paths.
The rewind crypto, PMMR-index node reads, and the scanner are
implemented + tested (unit + end-to-end). Update README status, the
build-order checklist (rewind/reads/scanner/reorg done; live-node smoke
+ backend proxy remain), the main.rs module status, and the startup log.
Building an end-to-end integration test (mock grin node → real tick() →
SQLite store) surfaced the real dual-DB blockers in sqlx::Any — NOT the
placeholder issue the review flagged:
- Any truncates i64 -> i32 for SQLite (12345678900 -> -539222988),
which would corrupt grin amounts. Store outputs.value as a decimal
string, sum in Rust. (Probe-documented; other int columns are < 2^31
for decades.)
- Any can't decode SQLite BOOLEAN; is_coinbase/spent are now INTEGER 0/1.
- DEFAULT now() -> DEFAULT CURRENT_TIMESTAMP (portable).
The integration test now proves the whole pipeline end-to-end on SQLite:
node UTXO listing → view-only rewind (golden output) → store with
value + key_id → correct balance → spend-by-absence reconcile → zero.
One schema + one query set now serve both backends.
The review's premise (sqlx::Any breaks on SQLite $N placeholders) is
wrong for sqlx 0.7: SQLite accepts $N numbered params and binds them
positionally, including reused placeholders. A probe test applies the
real migration + exercises set_cursor's reused-$N upsert, insert_output
(quoted "commit", bool), coinbase-maturity reads, and mark_spent against
SQLite — all pass. The only true incompatibility was DEFAULT now(),
changed to the portable DEFAULT CURRENT_TIMESTAMP. Schema + queries now
run unchanged on both backends. Updated the stale dialect notes.
Replace the stub scanner with the real incremental algorithm:
- discover: page get_unspent_outputs (with proofs) from the cursor to
the tip, rewind each new output against caught-up accounts, store
matches with their derivation path + a derived lock_height (coinbase
maturity = 1440)
- spend reconcile: mark any stored-unspent commit the node no longer
lists (grin has no per-block inputs; spends = absence from the set)
- reorg: detect via header-hash mismatch at the checkpoint height,
roll back a bounded depth, reseek from the fork
- backfill: bring a newly-registered account up from its birthday index
- cursor only advances after discover AND reconcile succeed
DB: chain_cursor gains output_mmr_index (tip height/hash become the
reorg checkpoint); new helpers (list_accounts, all_unspent_commits,
mark_unspent, account_unlocked_totals, scan-height advance/clamp).
Read path: get_balance adds `unlocked` (spendable-now) and
get_unspent_outs flags per-output `spendable`, so immature coinbase /
locked outputs are never presented as spendable.
Grin scans by PMMR output index, not block-by-block, so replace the
block-based stubs with the actual Foreign-API-v2 surface:
- get_unspent_outputs(start,end,max, include_proof=true): page the UTXO
set WITH rangeproofs (the discovery call the scanner rewinds)
- get_outputs(commits): spend detection by absence from the unspent set
- get_pmmr_indices(height): birthday height -> start index for backfill
- get_header(height) + get_tip(): reorg detection + tip hash
- push_transaction: fix the {"Ok":null} success case (the generic rpc
helper mis-read null as an error) via a null-tolerant rpc_ok_unit
- delete get_block/BlockView (grin's get_block returns no proofs)
- reshape ChainOutput to the UTXO-listing fields; permissive,
version-tolerant deserialization; per-output decode failures skip.
Tests for the node JSON parsing + spent/proofless filtering.
Replace the rewind_output stub with a real, view-only implementation:
- src/rewind.rs: derive the v3 rewind nonce (keyed BLAKE2b over the
commitment) and rewind the Bulletproof via upstream grin_secp256k1zkp,
then parse the 20-byte proof message into the authoritative key_id +
value. Faithful minimal port of grin-ext's cross-validated crypto; no
private key is ever involved. v3-only (legacy needs the master secret,
same limit as grin-wallet's view-only scan); paths are advisory.
- src/grin.rs: wire rewind_output with hostile-input-safe hex decoding.
- Cargo.toml: depend on upstream grin_secp256k1zkp + blake2 (not the
vendored fork — the server target needs no wasm patch).
- Golden-vector tests generated by grin's OWN reference ProofBuilder
(depth-3 Grim + depth-4 Smirk [0,0,n,0]); proves byte-compatibility
with the published secp crate. Passed an adversarial money-safety
review (no false-positive/negative, panic-safe).
Make the scaffold standalone and publish-ready:
- Rewrite code + README references to the originating backend as a generic
"wallet backend" so the service reads as a reusable, run-anywhere LWS.
- Add the MIT LICENSE file (matches Cargo.toml) and a short provenance note.
- Add repository/readme/keywords/categories metadata to Cargo.toml.
A standalone, self-hostable Grin light-wallet-server mirroring the Monero LWS:
register a view/rewind credential, run a background chain scanner, rewind outputs
against registered keys, store per-account matches in a DB, and serve balance /
outputs / broadcast / height fast (vs the current on-demand stateless scan proxy).
This is the scaffold — README (architecture + HTTP surface + DB schema + how a
wallet backend proxies to it), Cargo skeleton, migrations sketch, and route/scanner
stubs. Public-clean: no operator-specific infra, hosts, or secrets.
jwinterm