Never had a caller:
- Blockchain::init taking HardFork*&: added in 8f863e742, no caller in
any revision since
- tx_memory_pool::get_txpool_weight, set_txpool_max_weight: added as
get_txpool_size and set_txpool_max_size in bc61ae69b alongside
--max-txpool-size, which is wired through tx_memory_pool::init
instead; renamed size to weight in 5ffb2ff9b, still uncalled
- get_transaction_version: added in b750fb27b, no caller since
Dead when their last user went:
- core::get_blocks taking vector<pair<blobdata, block>>, both overloads:
last caller removed in 9faef1f83; ed2c81ed9 later converted the
already-dead signatures from std::list to std::vector. The
vector<block> overload is still used by core_tests and is kept
- blocks_container: 9e82b694d removed the in-memory blockchain that
used it
- tx_by_fee_and_receive_time_entry: 445319d3f replaced the sorted
container it keyed with a boost::bimap
- m_store_blockchain_interval: its do_call went with the original
blockchain_storage format in 9e82b694d
- m_fork_moaner: its do_call went with the "We are most likely forked"
message in f0371210e
Also drops two break statements after a return in ver_mixed_rct_semantics
and the doc comments belonging to the removed declarations.
/get_blocks.bin is HTTP binary RPC serialized with portable storage, not a
P2P Levin message. The previously added pool limiter used
LEVIN_DEFAULT_MAX_PACKET_SIZE as a response budget, but that constant does
not apply to the RPC path.
Fixes the following circumstance:
1. Wallet submits tx to node.
2. Node adds tx to pool and queues for relay.
3. Node shuts off before the relay executes in the strand.
4. Node restarts.
5. Now, the fix ensures the node will re-relay the tx in the
get_relayable_transactions loop.
Also makes sure that stem txs don't have last_relayed_time set
to max, which would prevent them from ever being re-relayed on
a restart as well.
Context: https://github.com/seraphis-migration/monero/issues/365#issuecomment-4811722029
b5f7330 tx_pool: fix use-after-free in prune() - txid was a reference to an item which was later deleted in remove_tx_from_transient_lists(), and txid was used after that (SChernykh)
Replace the plain time_t map value with an added_tx_info struct that
stores both receive_time and a sensitive flag. Thread the flag through
all add_tx_to_transient_lists call sites so incremental RPC queries
can skip sensitive transactions.
Reported-by: selsta
Add an include_sensitive parameter to tx_memory_pool::get_pool_for_rpc
(and its core passthrough), mirroring the include_sensitive_data
parameter on the HTTP analog get_transactions_and_spent_keys_info.
When false, receive_time and last_relayed_time are zeroed using the
same masking already applied on the HTTP path.
The ZMQ handler in daemon_handler.cpp passes !m_restricted, so
--restricted-zmq-rpc callers now receive the same privacy-filtered view
as restricted HTTP callers instead of the unfiltered timing metadata
they previously got. Stem-phase txs continue to be excluded regardless
(relay_category::broadcasted filter unchanged).
Refs #10529.
This replaces `ver_rct_non_semantics_simple_cached()` with an API that offloads
the responsibility of tracking input verification successes to the caller. The
main caller of this function in the codebase, `cryptonote::Blockchain()` instead
keeps track of the verification results for transaction in the mempool by
storing a "verification ID" in the mempool metadata table (with `txpool_tx_meta_t`).
This has several benefits, including:
* When the mempool is large (>8192 txs), we no longer experience cache misses and unnecessarily re-verify ring signatures. This greatly improves block propagation time for FCMP++ blocks under load
* For the same reason, reorg handling can be sped up by storing verification IDs of transactions popped from the chain
* Speeds up re-validating every mempool transaction on fork change (monerod revalidates the whole tx-pool on HFs #10142)
* Caches results for every single type of Monero transaction, not just latest RCT type
* Cache persists over a node restart
* Uses 512KiB less RAM (8192*2*32B)
* No additional storage or DB migration required since `txpool_tx_meta_t` already had padding allocated
* Moves more verification logic out of `cryptonote::Blockchain`
Furthermore, this opens the door to future multi-threaded block verification
speed-ups. Right now, transactions' input proof verification is limited to one
transaction at a time. However, one can imagine a scenario with verification IDs
where input proofs are optimistically multi-threaded in advance of block
processing. Then, even though ring member fetching and verification is
single-threaded inside of `cryptonote::Blockchain::check_tx_inputs()`, the
single thread can skip the CPU-intensive cryptographic code if the verification
ID allows it.
Also changes the default log category in `tx_verification_utils.cpp` from "blockchain" to "verify".
Changes complexity from M*N to (2*N+M)*log2(M). The FCMP++ stressnet recently hit mempool sizes of ~55k txs.
If the requesting node's mempool is populated, this results in an average of (55000*55000)/2
(about 1.5 billion) comparisons for the responding node. Under this commit, this would be reduced to
(55000+55000)*log2(55000) comparisons (about 2.6 million), a 99.83% reduction.
Otherwise we can end up double counting txs towards the weight,
which can over-state the pool weight. E.g. relay tx to node in
stem phase, add its weight to pool weight, then receive tx
from another node, then bump the pool weight again. That double
counts the tx towards the pool weight.
If the weight exceeds the max, the node will "prune" txs from the
pool. Thus, over-counting is probably a cause of, but perhaps
not the only cause of:
https://github.com/seraphis-migration/monero/issues/148
The commit kills support for deprecated ephemeral Boost messages: signed/unsigned transaction sets, pending transactions, reserve proofs, MMS messages, etc.
It does NOT kill support for loading very old wallets in Boost format, that should be supported indefinitely. These messages were deprecated 5 years ago. Since
then, we have had a hard fork to enable a new non-compatible transaction type (w/ view tags), and disable the old transaction type. This renders basically all
of the aforementioned messages before that HF useless, with the possible exception of reserve proofs.
This commit also cleans up dead inclusions of boost serialization headers.
This commit is part of upstreaming Carrot/FCMP++. Killing support for Boost messages now means less boilerplate Boost serialization review for Carrot/FCMP++.
- Fixed undefined behavior after a call to `remove_tx_from_transient_lists` (it used an invalid iterator)
- Fixed `txCompare` (it wasn't strictly weak ordered)
std::sort is unstable, so it can return random sets of transactions when mempool has many transactions with the same fee/byte. It can result in p2pool mining empty blocks sometimes because it doesn't pick up "new" transactions immediately.
Related to https://github.com/monero-project/research-lab/issues/78
Added a relay rule that enforces the `unlock_time` field is equal to 0 for non-coinbase transactions.
UIs changed:
* Removed `locked_transfer` and `locked_sweep_all` commands from `monero-wallet-cli`
APIs changed:
* Removed `unlock_time` parameters from `wallet2` transfer methods
* Wallet RPC transfer endpoints send error codes when requested unlock time is not 0
* Removed `unlock_time` parameters from `construct_tx*` cryptonote core functions
c4af33e Enforce restricted # pool txs served via RPC + optimize chunked reqs (j-berman)
9752116 wallet2, RPC: Optimize RPC calls for periodic refresh from 3 down to 1 call (rbrunner7)
- `/getblocks.bin` respects the `RESTRICTED_TX_COUNT` (=100) when
returning pool txs via a restricted RPC daemon.
- A restricted RPC daemon includes a max of `RESTRICTED_TX_COUNT` txs
in the `added_pool_txs` field, and returns any remaining pool hashes
in the `remaining_added_pool_txids` field. The client then requests
the remaining txs via `/gettransactions` in chunks.
- `/gettransactions` no longer does expensive no-ops for ALL pool txs
if the client requests a subset of pool txs. Instead it searches for
the txs the client explicitly requests.
- Reset `m_pool_info_query_time` when a user:
(1) rescans the chain (so the wallet re-requests the whole pool)
(2) changes the daemon their wallets points to (a new daemon would
have a different view of the pool)
- `/getblocks.bin` respects the `req.prune` field when returning
pool txs.
- Pool extension fields in response to `/getblocks.bin` are optional
with default 0'd values.
Before the fix, it processed all transactions in the mempool which could be very slow when mempool grows to several MBs in size. I observed `get_block_template_backlog` taking up to 15 seconds of CPU time under high mempool load.
After the fix, only transactions that can potentially be mined in the next block will be processed (a bit more than the current block median weight).
tobtoht