This week’s newsletter notes a vulnerability in Bitcoin Core versions that are already past end-of-life, asks for help testing release candidates of the next major version of Bitcoin Core, provides an update on the Bitcoin-Dev mailing list, describes recent discussions from the list, and links to a chapter about payment batching in Optech’s work-in-progress scaling techniques book. Also included are descriptions of several notable commits in popular Bitcoin infrastructure projects.

Action items

  • Ensure you aren’t running old Bitcoin Core versions: Suhas Daftuar disclosed a vulnerability affecting Bitcoin Core 0.13.0 to 0.13.2. (Note: those releases have been past end-of-life for several months.) The vulnerability would allow an attacker to convince your node that a valid block was invalid, forking you off the consensus block chain and making it possible to trick you into believing that you received confirmed bitcoins you wouldn’t actually control. In addition to checking for old versions of Bitcoin Core in your infrastructure, it’s also recommended that you check for altcoins whose nodes are based on the affected Bitcoin Core versions. See the disclosure details below for more information.

  • Help test Bitcoin Core 0.18.0 RC1: The first Release Candidate (RC) for the next major version of Bitcoin Core has been released. Organizations and experienced users who depend upon Bitcoin Core are highly encouraged to test it for regressions and other problems that could affect your use of it in production. Any testing is appreciated, but if you have some extra time after testing for your specific use cases, please consider helping test 0.18’s changes to the GUI. This interface is primarily used by less experienced users who are unlikely to test RCs themselves but who would be especially affected by any problems that slip through.

News

  • Bitcoin-Dev mailing list status update: the service outage reported in last week’s newsletter has been resolved but list administrators are planning to migrate to another solution. Many posts sent in the past two weeks have been relayed to list subscribers, but some have been lost. If you don’t see your post in the February or March archives, please resend it. Future Optech newsletters will mention any actions list subscribers need to take in order to continue to receive protocol discussion.

  • Bitcoin Core vulnerability disclosure: Suhas Daftuar disclosed a novel method for tricking earlier Bitcoin Core versions into rejecting valid blocks. If an attacker created a block with two transactions whose 32-byte hashes (txids), when concatenated together, appear to be a 64-byte transaction, it’s possible to create two different interpretations of the merkle tree rooted in the block header—one where the tree points to a single invalid 64-byte transaction and one where it points to two valid transactions. (Similar conflicting versions can also be created with more than two transactions.)

    Diagram of two identical merkle roots derived from different block data

    This can create a problem for Bitcoin Core as, normally, if it rejects a block as being invalid, it will add the header hash of that block to a cache so that it doesn’t waste resources requesting or re-processing that block again. This allowed an attacker to send your node the invalid form of the block to subsequently prevent your node from processing its valid form or any blocks that descend from it, forking you off the chain.

    A similar vulnerability was disclosed in 2012 as CVE-2012-2459 and Bitcoin Core was adapted then to not cache invalidity for blocks whose merkle tree contains ambiguities. However, an optimization implemented in Bitcoin Core 0.13.0 reintroduced this caching problem and necessitated a fix, which was included in Bitcoin Core 0.14.0. Daftuar’s email includes a very informative PDF that not only describes this specific problem in detail and shows its cost to be a very small 30 bits of brute force work (although you also need to mine a custom block) but which also describes other known vulnerabilities possible with Bitcoin’s merkle trees and calculates the average amount of brute force work to exploit them. Daftuar did not find any instances of the attack to date in the current consensus block chain.

  • Cleanup soft fork proposal discussion: this week saw discussion about the consensus cleanup soft fork proposal described in last week’s newsletter. Russell O’Connor raised the concern that invalidation of the OP_CODESEPARATOR opcode could prevent existing UTXOs using the opcode from being spent. It’s not possible to detect this because people could’ve paid money to a P2SH address whose not-yet-revealed redeemScript uses the discouraged opcode. O’Connor proposes to mitigate the problem of OP_CODESEPARATOR being used to increase worse-case block verification time by instead increasing the weight (vbytes) of transactions whose evaluated scripts contain the opcode. This would reduce the maximum number of code separators that could be contained within a block while also likely reducing the overall size and total number of operations in the block to the point where it could be verified reasonably quickly.

    O’Connor also raised a similar concern regarding the soft fork’s proposal to invalidate the unallocated sighash type bytes. It’s also not possible to entirely detect this because Bitcoin users may have created pre-signed locktimed transactions for which they’ve lost or destroyed the signing keys, preventing them from creating new signatures. Instead of increasing the weight of the unallocated sighash bytes to restrict their use, he recommends use of a more complex sighash cache (as previously described as an option in the proposed BIP).

    Matt Corallo replied to both of O’Connor’s concerns by pointing out that, although we can’t detect usage of these features for spends that haven’t been broadcast, we can detect them for any transactions in the existing chain—and that usage doesn’t exist. “I’m seriously skeptical that someone is using a highly esoteric scheme and has just been pouring money into it without ever having tested it or having withdrawn any money from it whatsoever,” said Corallo before also discussing the amount of extra complexity for calculating fees and caching sighashes if these features aren’t disabled. His rebuttal also included a plea for anyone using transaction features that are not relayed or mined by default (“non-standard”) to contact Bitcoin Core developers and let them know about the situation so that policies can be reconsidered.

  • Feedback requested on signet: Karl-Johan Alm has been working on an alternative to Bitcoin’s testnet that uses centrally-signed blocks instead of proof of work. Although this doesn’t allow testing the decentralized nature of Bitcoin, it could make the testing network much more convenient for application developers by providing regular block production most of the time plus scheduled tests of adverse events such as block chain reorganizations or fee spikes. It would also ensure the central signing authority always had test coins to distribute via their faucet. By comparison, testnet block production is sometimes too fast for peers to keep up or so slow that it’s useless for testing, faucets are often empty, and griefers can create reorg scenarios that would be extremely unlikely to exist on a network with actual value at stake. Alm is seeking feedback and would like to eventually incorporate his code into Bitcoin Core (and, probably, have other node implementations support it too).

  • Removal of BIP61 P2P reject messages: Marco Falke started a thread seeking feedback about his desire to remove BIP61 reject messages from Bitcoin Core. When your node receives a message (such as a transaction) that has some problem, your node will return a reject message that contains a description of the problem. BIP61 messages are not trustless (your node could lie) and the same information about problems can be extracted from the rejecting node’s logs, which allows developers to investigate problems with messages sent to their own nodes. See Newsletter #13 for our description of Falke’s PR that disabled reject messages by default in Bitcoin Core.

    Andreas Schildbach, a wallet author and lead maintainer of the popular BitcoinJ library, requested keeping the messages and re-enabling them by default. His users email him logfiles containing reject messages when their transactions don’t go through, helping him to debug problems. In response, Gregory Maxwell pointed out that even when an honest node accepts a transaction, that doesn’t mean it’ll be also accepted by that node’s peers. That means clients still need to monitor for transaction propagation without using BIP61, making BIP61 redundant for that purpose. Similarly, BIP61 can’t reasonably be used to detect transactions with too-low feerates because an accepted transaction paying a minimum feerate may take weeks longer to confirm than the user desired when default-sized mempools are full. Finally, verification nodes are designed to maximize performance, which often conflicts with the ability to provide maximally-useful debugging information to random untrusted peers.

  • Extension fields to Partially Signed Bitcoin Transactions (PSBTs): Andrew Poelstra proposed the addition of several fields to PSBTs to help support several new features. He also proposed making one currently-required field optional. These new fields can help clients determine whether an OP_CHECKSEQUENCEVERIFY (CSV) condition is satisfied, support the full range of scripts it’s possible to generate with miniscript, and include extra data for use with the MuSig, pay-to-contract, and sign-to-contract protocols. BIP174 author Andrew Chow appeared receptive to most of the suggestions.

  • Review of Bitcoin privacy literature published: Chris Belcher published an extended summary of various privacy concerns present in Bitcoin. That page and the Wiki’s related Privacy category provides an excellent starting point for anyone researching Bitcoin privacy concerns.

  • Version 2 addr message proposed: Wladimir van der Laan has proposed creating a BIP for a new version of the P2P protocol addr message. The existing message communicates the IP address or OnionCat encoded Tor hidden service (.onion) name of a node, its port, and a bitmap of the services the node provides. However, since the release of the original Bitcoin codebase, Tor has upgraded their hidden service addresses to use 256 bits, preventing them from being used in Bitcoin’s existing addr messages. There are also other network overlay protocols, such as I2P, that also use longer addresses. The proposed BIP, if implemented, will provide support for these protocols.

  • Optech publishes book chapter about payment batching: paying multiple people in the same transaction can reduce the average transaction fee cost per payment by more than 70%. The technique is especially convenient for high-frequency spenders such as exchanges. As part of Optech’s ongoing work to create a guide to individually-deployable scaling techniques, we’re publishing our draft chapter that describes this technique and its tradeoffs in detail.

Notable code and documentation changes

Notable changes this week in Bitcoin Core, LND, C-Lightning, Eclair, libsecp256k1, and Bitcoin Improvement Proposals (BIPs). Note that Bitcoin Core currently has work being performed on both its master development branch and the branch for the upcoming 0.18 release, so we’ve noted which branch each Bitcoin Core merge affected.

  • Bitcoin Core #15118 generalizes how Bitcoin Core stores and retrieves data associated with blocks and UTXO changes in order to make it easier for new methods to store and retrieve other information in the same way. This was done to allow reusing that mechanism for storing BIP157 compact block filters on disk. This is currently part of the master development branch only.

  • Bitcoin Core #15492 removes the deprecated generate RPC used for creating blocks in regtest mode. This RPC was previously superseded by the generatetoaddress RPC which doesn’t require the node to be built or run with wallet support. This is part of the master development branch only.

  • Bitcoin Core #15497 changes the use of output script descriptors in multiple RPCs to use consistent range notation for deriving multiple addresses from a descriptor with a BIP32 HD wallet path. This is part of the 0.18 branch and 0.18.0RC1 release.

  • LND #2690 puts more gossip traffic in a queue (rather than sending it immediately) so that higher-priority information is more likely to be handled quickly. Gossip traffic is used for communicating which peers are on the network and what channels they have available.

  • C-Lightning #2391 deprecates the address field in the newaddr RPC, replacing it with either a bech32 field or a p2sh-segwit field depending on the address type requested (or both fields if an optional all parameter is passed to the RPC). The address type in each field is consistent with its name.