Update rapidjson

Saves ~20 MB of memory

README.md

@@ -2,9 +2,7 @@
 
 Decentralized pool for Monero mining.
 
-**NOTE** This is a highly experimental and untested software. I did some extensive testing locally, but there's zero guarantee it will work for you! It requires a custom monerod version (for now). No binaries are provided yet. Testing on mainnet has started! You can check the pool status at https://p2pool.io/
+Mining on the mainnet has started! You can check the pool status at https://p2pool.io/
-
-No ETA on the official release date, but hopefully before the end of September 2021.
 
 ### Build status
 
@@ -126,6 +124,7 @@ make -j$(sysctl -n hw.logicalcpu)
 ### Windows
 
 p2pool binary (Visual Studio Community 2019 build):
+*NOTE: You need to have the "Desktop Development with C++" module installed.*
 ```
 git clone --recursive https://github.com/SChernykh/p2pool
 cd p2pool
@@ -135,6 +134,8 @@ cmake .. -G "Visual Studio 16 2019"
 ```
 then open generated build\p2pool.sln in Visual Studio and build it there
 
+Alternatively, you can select "Clone a repository" within the GUI, then select "Build" from the menu. 
+
 monerod binary compatible with p2pool:
 ```
 git clone --recursive https://github.com/SChernykh/monero
@@ -144,28 +145,35 @@ git submodule sync && git submodule update --init --force --recursive
 ```
 then follow the instructions from https://github.com/monero-project/monero/#on-windows
 
-## How to test
+## How to mine on P2Pool
 
-Mainnet test has started! **PPLNS window = 2160 blocks, block time = 10 seconds**. This guide assumes that you run everything on the same machine. If it's not the case, change `127.0.0.1` to appropriate IP addresses for your setup. It's highly recommended to create a new mainnet wallet for testing because **wallet addresses are public on p2pool**.
+This guide assumes that you run everything on the same machine. If it's not the case, change `127.0.0.1` to appropriate IP addresses for your setup. It's highly recommended to create a new mainnet wallet for mining because **wallet addresses are public on p2pool**.
 
 **Wallet software compatible with p2pool payouts**
 - Official Monero CLI and GUI v0.17.2.3 and newer
 - Monerujo v2.1.0 "Vertant" and newer
 - Cake Wallet v4.2.7 and newer
+- MyMonero
 
-The purpose of this test is to bring as much hashrate as possible and check if stratum server works fine!
+**General Considerations**
 
-- Grab the latest source code for both p2pool and monerod and build them (see above, also notice that the branch name for monerod changed, you'll need to checkout p2pool-api-v0.17)
+- Create a separate restricted user account for mining. p2pool is relatively new and may still have serious bugs/vulnerabilities!
+- You have to use the primary wallet address for mining. Subaddresses and integrated addresses are not supported, just like with monerod solo mining.
+- Check that ports 18080 (Monero p2p port) and 37889 (p2pool p2p port) are open in your firewall to ensure better connectivity. If you're mining from a computer behind NAT (like a router) you could consider forwarding the ports to your local machine.
+- You can connect multiple miners to the same p2pool node. The more the better!
+
+Step-by-step guide:
+
+### GNU/Linux
+
+- Download binaries from https://github.com/SChernykh/p2pool/releases/latest
+- Alternatively, grab the latest source code for both p2pool and monerod and build them (see above, also notice that the branch name for monerod changed, you'll need to checkout p2pool-api-v0.17)
 - Prepare enough huge pages (each of monerod/p2pool/xmrig needs them): `sudo sysctl vm.nr_hugepages=3072`
-- Create a separate restricted user account for testing. p2pool is untested and can have serious bugs/vulnerabilities!
 - Get xmrig (linux-static-x64) binary from https://github.com/xmrig/xmrig/releases/latest
 - Check that ports 18080 (Monero p2p port) and 37889 (p2pool p2p port) are open in your firewall to ensure better connectivity
-- Create a new mainnet wallet
+- Run `./monerod --zmq-pub tcp://127.0.0.1:18083 --disable-dns-checkpoints --enable-dns-blocklist` **don't forget --zmq-pub parameter in the command line**
-- You have to use the primary wallet address for mining. Subaddresses and integrated addresses are not supported, just like with monerod solo mining
+- Double check that it shows **Monero 'Oxygen Orion' (v0.17.2.3-7dbb0d1fc)** on startup. Wait until it's synchronized.
-- Run `./monerod --zmq-pub tcp://127.0.0.1:18083` **don't forget --zmq-pub parameter in the command line**
-- Double check that it shows **Monero 'Oxygen Orion' (v0.17.2.3-04bfd948a)** on startup. Wait until it's synchronized.
 - Run `./p2pool --host 127.0.0.1 --wallet YOUR_WALLET_ADDRESS`
-- Keep both monerod and p2pool running for the whole duration of your test
 - p2pool has _very_ verbose logging by default, it will spam a lot, no I mean A LOT in both console and in p2pool.log. Logs help testing immensely!
 - You can use `logrotate` with a config like this to control logfile growth:
     ```
@@ -178,15 +186,43 @@ The purpose of this test is to bring as much hashrate as possible and check if s
 	    nocreate
     }
     ```
-- Wait until initial p2pool sync is finished, it shouldn't take more than 5-10 minutes. Of course it depends on your connection speed!
+- Wait until initial p2pool sync is finished, it shouldn't take more than 5-10 minutes, once completed xmrig should be able to connect to the stratum server on port 3333.
-- p2pool has a stratum server listening on port 3333, you can connect xmrig to it now
 - Run `./xmrig -o 127.0.0.1:3333`. Note that you don't need to specify wallet address for xmrig. **Wallet address set in xmrig config will be ignored!**
 - To set custom fixed difficulty for your miner (for example, 10000), run `./xmrig -u x+10000 -o 127.0.0.1:3333`
 - xmrig should connect and start mining
-- you can connect multiple miners to the same p2pool node. The more the better!
-- From now on, watch your wallet to see if it gets anything
 - Also check p2pool.log for any warnings and errors: `grep -E 'WARNING|ERROR' p2pool.log`
 
+### Windows 
+
+*NOTE: Windows SmartScreen may block incoming connections by files that are "Downloaded from the Internet". You can allow 'p2pool.exe' and 'monerod.exe' by double-clicking them, clicking "More Info", then click "Run Anyway" and then closing them immediately so you can run them from the command line. Advanced users can use the PowerShell cmdlet `Unblock-File` to remove this flag.*
+
+- Download p2pool binaries from https://github.com/SChernykh/p2pool/releases/latest
+- Download xmrig binary from https://github.com/xmrig/xmrig/releases/latest *(xmrig-6.15.0-gcc-win64.zip is the current version)*
+- Expand the p2pool binaries into an appropriate location (`%USERPROFILE%/bin` or `C:/bin/` are good options)
+- Expand xmrig binary into appropriate location (same folder as p2pool is fine)
+- Prepare huge pages (each of monerod/p2pool/xmrig needs them): 
+  - On Windows 10 or above, run xmrig at least once as Administrator (right-click Run As Administrator)
+  - On earlier versions of Windows, you'll need to run it as admin at least once per login.
+- Open a command prompt and navigate to the folder where you extracted p2pool.
+- *When running these commands, Windows Firewall may prompt to allow connections, click "Allow"*
+- Run `.\Monero\monerod.exe --zmq-pub tcp://127.0.0.1:18083 --disable-dns-checkpoints --enable-dns-blocklist` *NOTE: don't forget --zmq-pub parameter in the command line*
+- Double check that it shows **Monero 'Oxygen Orion' (v0.17.2.3-7dbb0d1fc)** on startup. Wait until it's synchronized.
+- Run `.\p2pool.exe --host 127.0.0.1 --wallet YOUR_WALLET_ADDRESS`
+- Wait until initial p2pool sync is finished, it shouldn't take more than 5-10 minutes, once completed xmrig should be able to connect to the stratum server on port 3333.
+- Run `.\xmrig.exe -o 127.0.0.1:3333`. Note that you don't need to specify wallet address for xmrig. **Wallet address set in xmrig config will be ignored!**
+- To set custom fixed difficulty for your miner (for example, 10000), run `xmrig.exe -u x+10000 -o 127.0.0.1:3333`
+- Windows Quickstart: Create a batch (.bat) file with the following contents and place it in your p2pool directory along with xmrig.exe.
+```
+@ECHO OFF
+start cmd /k %~dp0\Monero\monerod.exe --zmq-pub tcp://127.0.0.1:18083 --disable-dns-checkpoints --enable-dns-blocklist
+ECHO Wait until the Monero daemon shows fully synced before continuing. This can take some time. Type 'status' in other window to check progress.
+PAUSE
+start cmd /k %~dp0\p2pool.exe --wallet YOUR_WALLET_ADDRESS
+ECHO Wait until the daemon shows fully synced before continuing. This can take some time.
+PAUSE
+%~dp0\xmrig.exe -u x+30000 -o 127.0.0.1
+```
+
 ## Donations
 
 If you'd like to support further development of Monero P2Pool, you're welcome to send any amount of XMR to the following address:

external/src/llhttp/llhttp.c

@@ -2,6 +2,7 @@
 #pragma warning(disable : 4061 4065 4100 4152 4242 4244 4668 4702 4710 4711 4820 5045)
 #elif defined __GNUC__
 #pragma GCC diagnostic ignored "-Wunused-parameter"
+#pragma GCC diagnostic ignored "-Wunused-variable"
 #endif
 
 #if LLHTTP_STRICT_MODE

src/crypto.cpp

@@ -227,22 +227,50 @@ public:
 		return true;
 	}
 
+	void clear()
+	{
+		MutexLock lock(m);
+
+		derivations.clear();
+		public_keys.clear();
+	}
+
 private:
 	uv_mutex_t m;
 	std::unordered_map<std::array<uint8_t, HASH_SIZE * 2>, hash> derivations;
 	std::unordered_map<std::array<uint8_t, HASH_SIZE * 2 + sizeof(size_t)>, hash> public_keys;
 };
 
-static Cache cache;
+static Cache* cache = nullptr;
 
 bool generate_key_derivation(const hash& key1, const hash& key2, hash& derivation)
 {
-	return cache.get_derivation(key1, key2, derivation);
+	return cache->get_derivation(key1, key2, derivation);
 }
 
 bool derive_public_key(const hash& derivation, size_t output_index, const hash& base, hash& derived_key)
 {
-	return cache.get_public_key(derivation, output_index, base, derived_key);
+	return cache->get_public_key(derivation, output_index, base, derived_key);
+}
+
+void init_crypto_cache()
+{
+	if (!cache) {
+		cache = new Cache();
+	}
+}
+
+void destroy_crypto_cache()
+{
+	if (cache) {
+		delete cache;
+		cache = nullptr;
+	}
+}
+
+void clear_crypto_cache()
+{
+	cache->clear();
 }
 
 } // namespace p2pool

src/crypto.h

@@ -24,4 +24,8 @@ bool check_keys(const hash& pub, const hash& sec);
 bool generate_key_derivation(const hash& key1, const hash& key2, hash& derivation);
 bool derive_public_key(const hash& derivation, size_t output_index, const hash& base, hash& derived_key);
 
+void init_crypto_cache();
+void destroy_crypto_cache();
+void clear_crypto_cache();
+
 } // namespace p2pool

src/main.cpp

@@ -16,13 +16,14 @@
  */
 
 #include "common.h"
+#include "crypto.h"
 #include "p2pool.h"
 #include "stratum_server.h"
 #include "p2p_server.h"
 
 static void usage()
 {
-	printf(
+	printf("P2Pool %s\n"
 		"\nUsage:\n\n" \
 		"--wallet             Wallet address to mine to. Subaddresses and integrated addresses are not supported!\n"
 		"--host               IP address of your Monero node, default is 127.0.0.1\n"
@@ -38,6 +39,7 @@ static void usage()
 		"--help               Show this help message\n\n"
 		"Example command line:\n\n"
 		"%s --host 127.0.0.1 --rpc-port 18081 --zmq-port 18083 --wallet YOUR_WALLET_ADDRESS --stratum 0.0.0.0:%d --p2p 0.0.0.0:%d\n\n",
+		p2pool::VERSION,
 #ifdef _WIN32
 		"p2pool.exe"
 #else
@@ -68,10 +70,14 @@ int main(int argc, char* argv[])
 	int result;
 
 	memory_tracking_start();
+
+	p2pool::init_crypto_cache();
 	{
 		p2pool::p2pool pool(argc, argv);
 		result = pool.run();
 	}
+	p2pool::destroy_crypto_cache();
+
 	memory_tracking_stop();
 
 	return result;

src/p2p_server.cpp

@@ -102,14 +102,11 @@ P2PServer::~P2PServer()
 	uv_mutex_destroy(&m_peerListLock);
 	uv_mutex_destroy(&m_broadcastLock);
 	uv_mutex_destroy(&m_missingBlockRequestsLock);
+
+	clear_cached_blocks();
 	uv_rwlock_destroy(&m_cachedBlocksLock);
 
 	delete m_block;
-
-	for (auto it : m_cachedBlocks) {
-		delete it.second;
-	}
-
 	delete m_cache;
 }
 
@@ -124,6 +121,16 @@ void P2PServer::add_cached_block(const PoolBlock& block)
 	m_cachedBlocks.insert({ new_block->m_sidechainId, new_block });
 }
 
+void P2PServer::clear_cached_blocks()
+{
+	WriteLock lock(m_cachedBlocksLock);
+
+	for (auto it : m_cachedBlocks) {
+		delete it.second;
+	}
+	m_cachedBlocks.clear();
+}
+
 void P2PServer::store_in_cache(const PoolBlock& block)
 {
 	if (m_cache && block.m_verified && !block.m_invalid) {

src/p2p_server.h

@@ -49,6 +49,7 @@ public:
 	~P2PServer();
 
 	void add_cached_block(const PoolBlock& block);
+	void clear_cached_blocks();
 	void store_in_cache(const PoolBlock& block);
 
 	void connect_to_peers(const std::string& peer_list);

src/p2pool.cpp

@@ -50,6 +50,8 @@ p2pool::p2pool(int argc, char* argv[])
 	, m_updateSeed(true)
 	, m_submitBlockData{}
 {
+	LOGINFO(1, log::LightCyan() << VERSION);
+
 	if (!m_params->m_wallet.valid()) {
 		LOGERR(1, "Invalid wallet address. Try \"p2pool --help\".");
 		panic();
@@ -370,6 +372,10 @@ void p2pool::submit_block() const
 
 	request = "{\"jsonrpc\":\"2.0\",\"id\":\"0\",\"method\":\"submit_block\",\"params\":[\"";
 
+	const uint32_t template_id = submit_data.template_id;
+	const uint32_t nonce = submit_data.nonce;
+	const uint32_t extra_nonce = submit_data.extra_nonce;
+
 	for (size_t i = 0; i < submit_data.blob.size(); ++i) {
 		char buf[16];
 
@@ -389,10 +395,6 @@ void p2pool::submit_block() const
 	}
 	request.append("\"]}");
 
-	const uint32_t template_id = submit_data.template_id;
-	const uint32_t nonce = submit_data.nonce;
-	const uint32_t extra_nonce = submit_data.extra_nonce;
-
 	JSONRPCRequest::call(m_params->m_host.c_str(), m_params->m_rpcPort, request.c_str(),
 		[height, diff, template_id, nonce, extra_nonce, is_external](const char* data, size_t size)
 		{
@@ -1055,6 +1057,8 @@ void p2pool::api_update_stats_mod()
 
 void p2pool::api_update_block_found(const ChainMain* data)
 {
+	clear_crypto_cache();
+
 	if (!m_api) {
 		return;
 	}

src/side_chain.cpp

@@ -1552,6 +1552,10 @@ void SideChain::prune_old_blocks()
 
 	if (num_blocks_pruned) {
 		LOGINFO(4, "pruned " << num_blocks_pruned << " old blocks at heights <= " << h);
+
+		// If side-chain started pruning blocks it means the initial sync is complete
+		// It's now safe to delete cached blocks
+		m_pool->p2p_server()->clear_cached_blocks();
 	}
 }
 

src/util.cpp

@@ -29,6 +29,22 @@ static constexpr char log_category_prefix[] = "Util ";
 
 namespace p2pool {
 
+#define STR2(X) STR(X)
+#define STR(X) #X
+
+const char* VERSION = "v1.1 (built"
+#if defined(__clang__)
+	" with clang/" __clang_version__
+#elif defined(__GNUC__)
+	" with GCC/" STR2(__GNUC__) "." STR2(__GNUC_MINOR__) "." STR2(__GNUC_PATCHLEVEL__)
+#elif defined(_MSC_VER)
+	" with MSVC/" STR2(_MSC_VER)
+#endif
+" on " __DATE__ ")";
+
+#undef STR2
+#undef STR
+
 MinerCallbackHandler::~MinerCallbackHandler() {}
 
 void panic()
@@ -40,7 +56,7 @@ void panic()
 			__debugbreak();
 		}
 #endif
-		exit(1);
+		abort();
 	} while (true);
 }
 

src/util.h

@@ -19,6 +19,8 @@
 
 namespace p2pool {
 
+extern const char* VERSION;
+
 template<typename T> struct not_implemented { enum { value = 0 }; };
 
 struct nocopy_nomove
@@ -126,7 +128,7 @@ namespace std {
 template<>
 struct hash<p2pool::hash>
 {
-	FORCEINLINE size_t operator()(const p2pool::hash& value) const
+	FORCEINLINE size_t operator()(const p2pool::hash& value) const noexcept
 	{
 		uint64_t result = 0xcbf29ce484222325ull;
 		for (size_t i = 0; i < p2pool::HASH_SIZE; ++i) {
@@ -139,7 +141,7 @@ struct hash<p2pool::hash>
 template<size_t N>
 struct hash<std::array<uint8_t, N>>
 {
-	FORCEINLINE size_t operator()(const std::array<uint8_t, N>& value) const
+	FORCEINLINE size_t operator()(const std::array<uint8_t, N>& value) const noexcept
 	{
 		uint64_t result = 0xcbf29ce484222325ull;
 		for (size_t i = 0; i < N; ++i) {

src/zmq_reader.cpp

@@ -82,10 +82,14 @@ void ZMQReader::run()
 		char addr[32];
 
 		snprintf(addr, sizeof(addr), "tcp://%s:%u", m_address, m_zmqPort);
-		m_subscriber.connect(addr);
+		if (!connect(addr, m_zmqPort)) {
+			throw zmq::error_t();
+		}
 
 		snprintf(addr, sizeof(addr), "tcp://127.0.0.1:%u", m_publisherPort);
-		m_subscriber.connect(addr);
+		if (!connect(addr, m_publisherPort)) {
+			throw zmq::error_t();
+		}
 
 		m_subscriber.set(zmq::sockopt::subscribe, "json-full-chain_main");
 		m_subscriber.set(zmq::sockopt::subscribe, "json-full-miner_data");
@@ -121,6 +125,38 @@ void ZMQReader::run()
 	LOGINFO(1, "worker thread stopped");
 }
 
+bool ZMQReader::connect(const char* address, uint32_t id)
+{
+	struct ConnectMonitor : public zmq::monitor_t
+	{
+		void on_event_connected(const zmq_event_t&, const char* address) ZMQ_OVERRIDE
+		{
+			LOGINFO(1, "connected to " << address);
+			connected = true;
+		}
+
+		bool connected = false;
+	} monitor;
+
+	char buf[32];
+	snprintf(buf, sizeof(buf), "inproc://connect-mon-%u", id);
+	monitor.init(m_subscriber, buf);
+	m_subscriber.connect(address);
+
+	using namespace std::chrono;
+	const system_clock::time_point start_time = system_clock::now();
+
+	while (!monitor.connected && monitor.check_event(-1)) {
+		const int64_t elapsed_time = duration_cast<milliseconds>(system_clock::now() - start_time).count();
+		if (elapsed_time >= 3000) {
+			LOGERR(1, "failed to connect to " << address);
+			return false;
+		}
+	}
+
+	return true;
+}
+
 void ZMQReader::parse(char* data, size_t size)
 {
 	char* value = data;

src/zmq_reader.h

@@ -30,6 +30,7 @@ public:
 private:
 	static void run_wrapper(void* arg) { reinterpret_cast<ZMQReader*>(arg)->run(); }
 	void run();
+	bool connect(const char* address, uint32_t id);
 
 	void parse(char* data, size_t size);
 

tests/src/crypto_tests.cpp

@@ -25,6 +25,8 @@ namespace p2pool {
 
 TEST(crypto, derivation)
 {
+	init_crypto_cache();
+
 	// Run the tests twice to check how crypto cache works
 	for (int i = 0; i < 2; ++i) {
 		std::ifstream f("crypto_tests.txt");
@@ -61,6 +63,8 @@ TEST(crypto, derivation)
 			}
 		} while (!f.eof());
 	}
+
+	destroy_crypto_cache();
 }
 
 }