Merge #6825: feat: add special transaction support to compact block filters (BIP-157/158)

4b90441 docs: indicate that blockfilter will be re-created (PastaPastaPasta)
b2522a2 chore: make clang format happy (UdjinM6)
09490b7 chore: drop useless feature_blockfilter_version.py (UdjinM6)
4750246 doc: shrink release notes (UdjinM6)
f28bf6d feat: start blockfilter sync from scratch on db version upgrades (UdjinM6)
d3f1b9e test: remove wallet dependency check from blockfilter tests (pasta)
492c109 feat: add versioning to blockfilter index to detect incompatible format (pasta)
bf352e7 evo: use Span<const unsigned char> in specialtx filter extraction callback to avoid temporary allocations; adjust helpers and call site (pasta)
c734b64 tests: p2p_blockfilters: use assert_greater_than for clearer failures (pasta)
a5bab55 test: remove wallet dependency from p2p_blockfilters.py and refactor duplicated code (Konstantin Akimov)
a1ecf5d tests: p2p_blockfilters: skip when wallet module is not available (pasta)
9266b9a refactor: break circular dependency over evo/assetlock and llmq/signing (Konstantin Akimov)
1c9809f filters: add cross-reference comments to keep bloom and compact filter extraction in sync (pasta)
7a33c14 tests: compact filters: fix F841, verify tx inclusion, and cleanups (pasta)
3fb9183 test: improve p2p_blockfilters special transaction tests (pasta)
8d08a54 Apply suggestions from code review (PastaPastaPasta)
0ea85f7 fix: remove unused variable in blockfilter_tests (pasta)
773fe9b lint: add specialtx_filter circular dependencies to allowed list (pasta)
8ae2280 feat: add special transaction support to compact block filters (BIP-157/158) (pasta)

Pull request description:

  ## Summary
  - Implements extraction of special transaction fields for compact block filters
  - Achieves feature parity between bloom filters and compact block filters for special transactions
  - Enables SPV clients to detect special transactions using either filtering mechanism

  ## What this PR does

  This PR adds support for Dash special transactions to the compact block filter implementation (BIP-157/158). Previously, only bloom filters could detect special transaction fields, leaving compact block filter users unable to track masternode-related transactions and platform operations.

  ### Implementation Details

  The implementation uses a delegation pattern to extract special transaction fields without creating circular dependencies:
  - `ExtractSpecialTxFilterElements()` function extracts relevant fields from each special transaction type
  - Fields extracted match exactly those in the bloom filter implementation (`CheckSpecialTransactionMatchesAndUpdate`)
  - All 5 special transaction types are supported:
    - **ProRegTx**: collateral outpoint, owner/voting keys, payout script
    - **ProUpServTx**: ProTx hash, operator payout script
    - **ProUpRegTx**: ProTx hash, voting key, payout script
    - **ProUpRevTx**: ProTx hash
    - **AssetLockTx**: credit output scripts (excluding OP_RETURN)

  ### Testing

  Comprehensive test coverage includes:
  - **Unit tests**: Test all 5 special transaction types with realistic payloads
  - **Functional tests**: Create actual AssetLockTx transactions with proper signatures
  - **Edge cases**: Empty scripts, OP_RETURN exclusion, multiple outputs

  ## Test plan
  - [x] Unit tests pass (`make check`)
  - [x] Functional test passes (`test/functional/p2p_blockfilters.py`)
  - [x] All special transaction types covered in tests
  - [x] Verified special fields are added to filters (via debug logging)

  🤖 Generated with [Claude Code](https://claude.ai/code)

ACKs for top commit:
  knst:
    utACK 4b90441
  UdjinM6:
    utACK 4b90441

Tree-SHA512: 673b47ef8626761cdffc659c6ca0ffd214b6d3e7ea6086976528ec79967f5a6a6d864b63cc1e5bac38f079fa092c49637f334c058231b4f6c687f9dae585db27

Author: PastaPastaPasta

doc/release-notes-6825.md

@@ -0,0 +1,19 @@
+# Breaking Change: Block Filter Index Format Update
+
+## Summary
+The compact block filter index format has been updated to include Dash special transaction data, providing feature parity with bloom filters for SPV client support. This change is **incompatible** with existing blockfilter indexes. Existing blockfilter indexes will automatically be re-created with the new version.
+
+## Technical Details
+- The blockfilter index now includes fields from Dash special transactions:
+  - ProRegTx (masternode registration)
+  - ProUpServTx (masternode service updates)
+  - ProUpRegTx (masternode operator updates)
+  - ProUpRevTx (masternode revocation)
+  - AssetLockTx (platform credit outputs)
+- A versioning system has been added to detect incompatible indexes on startup
+- The index version is now 2 (previously unversioned)
+
+## Benefits
+- SPV clients can now detect and track Dash-specific transactions
+- Feature parity between bloom filters and compact block filters
+- Protection against serving incorrect filter data to light clients

src/Makefile.am

@@ -202,6 +202,7 @@ BITCOIN_CORE_H = \
   evo/simplifiedmns.h \
   evo/smldiff.h \
   evo/specialtx.h \
+  evo/specialtx_filter.h \
   evo/specialtxman.h \
   dsnotificationinterface.h \
   governance/governance.h \
@@ -481,6 +482,7 @@ libbitcoin_node_a_SOURCES = \
   evo/simplifiedmns.cpp \
   evo/smldiff.cpp \
   evo/specialtx.cpp \
+  evo/specialtx_filter.cpp \
   evo/specialtxman.cpp \
   flatfile.cpp \
   governance/classes.cpp \

src/blockfilter.cpp

@@ -7,6 +7,7 @@
 
 #include <blockfilter.h>
 #include <crypto/siphash.h>
+#include <evo/specialtx_filter.h>
 #include <hash.h>
 #include <primitives/transaction.h>
 #include <script/script.h>
@@ -195,6 +196,11 @@ static GCSFilter::ElementSet BasicFilterElements(const CBlock& block,
             if (script.empty() || script[0] == OP_RETURN) continue;
             elements.emplace(script.begin(), script.end());
         }
+
+        // Extract special transaction elements using delegation pattern
+        ExtractSpecialTxFilterElements(*tx, [&elements](Span<const unsigned char> data) {
+            elements.emplace(data.begin(), data.end());
+        });
     }
 
     for (const CTxUndo& tx_undo : block_undo.vtxundo) {

src/common/bloom.cpp

@@ -119,6 +119,11 @@ bool CBloomFilter::CheckScript(const CScript &script) const
 // Filter is updated only if it has BLOOM_UPDATE_ALL flag to be able to have
 // simple SPV wallets that doesn't work with DIP2 transactions (multicoin
 // wallets, etc.)
+// NOTE(maintenance): Keep this implementation in sync with
+// ExtractSpecialTxFilterElements in src/evo/specialtx_filter.cpp.
+// Both routines must handle the same set of special-transaction fields.
+// If you modify one, update the other to prevent mismatches between
+// bloom filter relevance and compact filter element extraction.
 bool CBloomFilter::CheckSpecialTransactionMatchesAndUpdate(const CTransaction &tx)
 {
     if (!tx.HasExtraPayloadField()) {

src/evo/assetlocktx.cpp

@@ -6,7 +6,6 @@
 #include <evo/specialtx.h>
 
 #include <llmq/commitment.h>
-#include <llmq/signing.h>
 #include <llmq/quorums.h>
 
 #include <chainparams.h>
@@ -22,6 +21,11 @@
 
 using node::BlockManager;
 
+namespace llmq {
+// forward declaration to avoid circular dependency
+uint256 BuildSignHash(Consensus::LLMQType llmqType, const uint256& quorumHash, const uint256& id, const uint256& msgHash);
+} // namespace llmq
+
 /**
  *  Common code for Asset Lock and Asset Unlock
  */

src/evo/specialtx_filter.cpp

@@ -0,0 +1,132 @@
+// Copyright (c) 2025 The Dash Core developers
+// Distributed under the MIT software license, see the accompanying
+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+#include <evo/specialtx_filter.h>
+
+#include <evo/assetlocktx.h>
+#include <evo/providertx.h>
+#include <evo/specialtx.h>
+#include <primitives/transaction.h>
+#include <script/script.h>
+#include <span.h>
+#include <streams.h>
+
+/**
+ * Rationale for Special Transaction Field Extraction:
+ *
+ * This implementation extracts specific fields from Dash special transactions
+ * to maintain parity with the bloom filter implementation (CBloomFilter::CheckSpecialTransactionMatchesAndUpdate).
+ *
+ * The fields extracted are those that SPV clients might need to detect:
+ * - Owner/Voting keys: To track masternode ownership and voting rights
+ * - Payout scripts: To detect payments to specific addresses
+ * - ProTx hashes: To track masternode lifecycle and updates
+ * - Collateral outpoints: To track masternode collateral
+ * - Credit outputs: To track platform-related transactions
+ *
+ * Each transaction type has different fields based on its purpose:
+ * - ProRegTx: All identity and payout fields (initial registration)
+ * - ProUpServTx: ProTx hash and operator payout (service updates)
+ * - ProUpRegTx: ProTx hash, voting key, and payout (ownership updates)
+ * - ProUpRevTx: ProTx hash only (revocation tracking)
+ * - AssetLockTx: Credit output scripts (platform credits)
+ */
+// Helper function to add a script to the filter if it's not empty
+static void AddScriptElement(const CScript& script, const std::function<void(Span<const unsigned char>)>& addElement)
+{
+    if (!script.empty()) {
+        addElement(MakeUCharSpan(script));
+    }
+}
+
+// Helper function to add a hash/key to the filter
+template <typename T>
+static void AddHashElement(const T& hash, const std::function<void(Span<const unsigned char>)>& addElement)
+{
+    addElement(MakeUCharSpan(hash));
+}
+
+// NOTE(maintenance): Keep this in sync with
+// CBloomFilter::CheckSpecialTransactionMatchesAndUpdate in
+// src/common/bloom.cpp. If you add or remove fields for a special
+// transaction type here, update the bloom filter routine accordingly
+// (and vice versa) to avoid compact-filter vs bloom-filter divergence.
+void ExtractSpecialTxFilterElements(const CTransaction& tx, const std::function<void(Span<const unsigned char>)>& addElement)
+{
+    if (!tx.HasExtraPayloadField()) {
+        return; // not a special transaction
+    }
+
+    switch (tx.nType) {
+    case TRANSACTION_PROVIDER_REGISTER: {
+        if (const auto opt_proTx = GetTxPayload<CProRegTx>(tx)) {
+            // Add collateral outpoint
+            CDataStream stream(SER_NETWORK, PROTOCOL_VERSION);
+            stream << opt_proTx->collateralOutpoint;
+            addElement(MakeUCharSpan(stream));
+
+            // Add owner key ID
+            AddHashElement(opt_proTx->keyIDOwner, addElement);
+
+            // Add voting key ID
+            AddHashElement(opt_proTx->keyIDVoting, addElement);
+
+            // Add payout script
+            AddScriptElement(opt_proTx->scriptPayout, addElement);
+        }
+        break;
+    }
+    case TRANSACTION_PROVIDER_UPDATE_SERVICE: {
+        if (const auto opt_proTx = GetTxPayload<CProUpServTx>(tx)) {
+            // Add ProTx hash
+            AddHashElement(opt_proTx->proTxHash, addElement);
+
+            // Add operator payout script
+            AddScriptElement(opt_proTx->scriptOperatorPayout, addElement);
+        }
+        break;
+    }
+    case TRANSACTION_PROVIDER_UPDATE_REGISTRAR: {
+        if (const auto opt_proTx = GetTxPayload<CProUpRegTx>(tx)) {
+            // Add ProTx hash
+            AddHashElement(opt_proTx->proTxHash, addElement);
+
+            // Add voting key ID
+            AddHashElement(opt_proTx->keyIDVoting, addElement);
+
+            // Add payout script
+            AddScriptElement(opt_proTx->scriptPayout, addElement);
+        }
+        break;
+    }
+    case TRANSACTION_PROVIDER_UPDATE_REVOKE: {
+        if (const auto opt_proTx = GetTxPayload<CProUpRevTx>(tx)) {
+            // Add ProTx hash
+            AddHashElement(opt_proTx->proTxHash, addElement);
+        }
+        break;
+    }
+    case TRANSACTION_ASSET_LOCK: {
+        // Asset Lock transactions have special outputs (creditOutputs) that should be included
+        if (const auto opt_assetlockTx = GetTxPayload<CAssetLockPayload>(tx)) {
+            const auto& extraOuts = opt_assetlockTx->getCreditOutputs();
+            for (const CTxOut& txout : extraOuts) {
+                const CScript& script = txout.scriptPubKey;
+                // Exclude OP_RETURN outputs as they are not spendable
+                if (!script.empty() && script[0] != OP_RETURN) {
+                    AddScriptElement(script, addElement);
+                }
+            }
+        }
+        break;
+    }
+    case TRANSACTION_ASSET_UNLOCK:
+    case TRANSACTION_COINBASE:
+    case TRANSACTION_QUORUM_COMMITMENT:
+    case TRANSACTION_MNHF_SIGNAL:
+        // No additional special fields needed for these transaction types
+        // Their standard outputs are already included in the base filter
+        break;
+    } // no default case, so the compiler can warn about missing cases
+}

src/evo/specialtx_filter.h

@@ -0,0 +1,26 @@
+// Copyright (c) 2025 The Dash Core developers
+// Distributed under the MIT software license, see the accompanying
+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+#ifndef BITCOIN_EVO_SPECIALTX_FILTER_H
+#define BITCOIN_EVO_SPECIALTX_FILTER_H
+
+#include <functional>
+#include <span.h>
+
+class CTransaction;
+
+/**
+ * Extract filterable elements from special transactions for use in compact block filters.
+ * This function extracts the same fields that are included in bloom filters to ensure
+ * SPV clients can detect special transactions using either filtering mechanism.
+ *
+ * @param tx The transaction to extract elements from
+ * @param addElement Callback to add extracted elements to the filter. Uses
+ *                   Span<const unsigned char> to avoid intermediate
+ *                   allocations.
+ */
+void ExtractSpecialTxFilterElements(const CTransaction& tx,
+                                    const std::function<void(Span<const unsigned char>)>& addElement);
+
+#endif // BITCOIN_EVO_SPECIALTX_FILTER_H

src/index/blockfilterindex.cpp

@@ -30,6 +30,7 @@ using node::UndoReadFromDisk;
 constexpr uint8_t DB_BLOCK_HASH{'s'};
 constexpr uint8_t DB_BLOCK_HEIGHT{'t'};
 constexpr uint8_t DB_FILTER_POS{'P'};
+constexpr uint8_t DB_VERSION{'V'};
 
 constexpr unsigned int MAX_FLTR_FILE_SIZE = 0x1000000; // 16 MiB
 /** The pre-allocation chunk size for fltr?????.dat files */
@@ -111,11 +112,34 @@ BlockFilterIndex::BlockFilterIndex(BlockFilterType filter_type,
 
     m_name = filter_name + " block filter index";
     m_db = std::make_unique<BaseIndex::DB>(path / "db", n_cache_size, f_memory, f_wipe);
+
+    // Check version
+    int version = 0;
+    if (!m_db->Read(DB_VERSION, version) || version < CURRENT_VERSION) {
+        // No version or too old version means we need to start from scratch
+        LogPrintf("%s: Outdated or no version blockfilter, starting from scratch\n", __func__);
+        m_db.reset();
+        m_db = std::make_unique<BaseIndex::DB>(path / "db", n_cache_size, f_memory, /*f_wipe=*/true);
+        m_db->Write(DB_VERSION, CURRENT_VERSION);
+    }
+
     m_filter_fileseq = std::make_unique<FlatFileSeq>(std::move(path), "fltr", FLTR_FILE_CHUNK_SIZE);
 }
 
 bool BlockFilterIndex::Init()
 {
+    // Check version compatibility first
+    int version = 0;
+    if (m_db->Exists(DB_VERSION)) {
+        if (!m_db->Read(DB_VERSION, version)) {
+            return error("%s: Failed to read %s index version from database", __func__, GetName());
+        }
+        if (version > CURRENT_VERSION) {
+            return error("%s: %s index version %d is too high (expected <= %d)",
+                        __func__, GetName(), version, CURRENT_VERSION);
+        }
+    }
+
     if (!m_db->Read(DB_FILTER_POS, m_next_filter_pos)) {
         // Check that the cause of the read failure is that the key does not exist. Any other errors
         // indicate database corruption or a disk failure, and starting the index would cause
@@ -136,6 +160,11 @@ bool BlockFilterIndex::CommitInternal(CDBBatch& batch)
 {
     const FlatFilePos& pos = m_next_filter_pos;
 
+    // Write the current version if this is a new index
+    if (!m_db->Exists(DB_VERSION)) {
+        batch.Write(DB_VERSION, CURRENT_VERSION);
+    }
+
     // Flush current filter file to disk.
     CAutoFile file(m_filter_fileseq->Open(pos), SER_DISK, CLIENT_VERSION);
     if (file.IsNull()) {

src/index/blockfilterindex.h

@@ -25,6 +25,9 @@ static constexpr int CFCHECKPT_INTERVAL = 1000;
 class BlockFilterIndex final : public BaseIndex
 {
 private:
+    /** Version of the blockfilter index format. Increment this when breaking changes are made. */
+    static constexpr int CURRENT_VERSION = 2;
+
     BlockFilterType m_filter_type;
     std::string m_name;
     std::unique_ptr<BaseIndex::DB> m_db;