In RPC calls `get_blocks.bin` and `get_transactions`:
1. Skip deserializing then re-serializing tx blobs
2. Provide full unpruned transaction from mempool in `get_transactions` when `!req.prune`
3. Change `on_get_transactions()` response sort time from O(N^2) to O(N)
4. Generally improve memory usage and lookups in `on_get_transactions()`
Reviewed-by: selsta <selsta@sent.at>
Reviewed-by: j-berman <justinberman@protonmail.com>
Reviewed-by: jpk68 <jpk68@tutanota.com>
Simple API to check max blob size before parsing. This is useful
when reading blobs from untrusted sources.
@selsta pointed out that coinbase txs can technically be larger
than get_max_tx_size(), otherwise we could enforce it on all txs.
Never had a caller:
- Blockchain::init taking HardFork*&: added in 8f863e742, no caller in
any revision since
- tx_memory_pool::get_txpool_weight, set_txpool_max_weight: added as
get_txpool_size and set_txpool_max_size in bc61ae69b alongside
--max-txpool-size, which is wired through tx_memory_pool::init
instead; renamed size to weight in 5ffb2ff9b, still uncalled
- get_transaction_version: added in b750fb27b, no caller since
Dead when their last user went:
- core::get_blocks taking vector<pair<blobdata, block>>, both overloads:
last caller removed in 9faef1f83; ed2c81ed9 later converted the
already-dead signatures from std::list to std::vector. The
vector<block> overload is still used by core_tests and is kept
- blocks_container: 9e82b694d removed the in-memory blockchain that
used it
- tx_by_fee_and_receive_time_entry: 445319d3f replaced the sorted
container it keyed with a boost::bimap
- m_store_blockchain_interval: its do_call went with the original
blockchain_storage format in 9e82b694d
- m_fork_moaner: its do_call went with the "We are most likely forked"
message in f0371210e
Also drops two break statements after a return in ver_mixed_rct_semantics
and the doc comments belonging to the removed declarations.
- When there are missing IDs, blocks.size() != arg.blocks.size(), so arg.blocks can't be indexed by `i` - the indices will be wrong, the wrong weight will be returned to some peer and this peer will ban our node.
Use `bl.second` instead of `arg.blocks[i]`. Also it saves one DB query per returned block.
Does the following to speedup the `/is_key_image_spent` RPC endpoint:
- Reads all on-chain key images in one LMDB read transacion
- Uses `cryptonote_core::are_key_images_spent_in_pool()` instead of `cryptonote_core::get_pool_transactions_and_spent_keys_info()` for pool querying. This only does a LMDB read per key image if in the pool.
- Filters known on-chain spent key images before querying for key images spent in pool
This RPC endpoint was causing major daemon slowdowns in the Carrot/FCMP++ Alpha Stressnet when using the `rescan_spent` command, especially for large wallets.
The effect was much worse if the mempool was full.
This replaces `ver_rct_non_semantics_simple_cached()` with an API that offloads
the responsibility of tracking input verification successes to the caller. The
main caller of this function in the codebase, `cryptonote::Blockchain()` instead
keeps track of the verification results for transaction in the mempool by
storing a "verification ID" in the mempool metadata table (with `txpool_tx_meta_t`).
This has several benefits, including:
* When the mempool is large (>8192 txs), we no longer experience cache misses and unnecessarily re-verify ring signatures. This greatly improves block propagation time for FCMP++ blocks under load
* For the same reason, reorg handling can be sped up by storing verification IDs of transactions popped from the chain
* Speeds up re-validating every mempool transaction on fork change (monerod revalidates the whole tx-pool on HFs #10142)
* Caches results for every single type of Monero transaction, not just latest RCT type
* Cache persists over a node restart
* Uses 512KiB less RAM (8192*2*32B)
* No additional storage or DB migration required since `txpool_tx_meta_t` already had padding allocated
* Moves more verification logic out of `cryptonote::Blockchain`
Furthermore, this opens the door to future multi-threaded block verification
speed-ups. Right now, transactions' input proof verification is limited to one
transaction at a time. However, one can imagine a scenario with verification IDs
where input proofs are optimistically multi-threaded in advance of block
processing. Then, even though ring member fetching and verification is
single-threaded inside of `cryptonote::Blockchain::check_tx_inputs()`, the
single thread can skip the CPU-intensive cryptographic code if the verification
ID allows it.
Also changes the default log category in `tx_verification_utils.cpp` from "blockchain" to "verify".