This week’s newsletter includes the usual dashboard and action items, a feature article by developer Anthony Towns about the consolidation of four million UTXOs at Xapo, news about possible upgrades to Bitcoin’s script system, links to a few highly-voted questions and answers on the Bitcoin StackExchange, and some notable commits in the development branches of the Bitcoin Core, Lightning Network Daemon (LND), and C-lightning projects.

Action items

  • Bitcoin Core 0.16.2 released: a minor release that brings bug fixes and minor improvements. If you use the abandontransaction or verifytxoutproof RPCs, you should particularly consider upgrading. Otherwise, we recommend you review the release notes for other changes that may affect your operation and upgrade when convenient.

Dashboard items

  • Fees still low: hash rate increased difficulty by more than 14% in the 2,016-block retarget period ending Sunday, giving an average time between blocks of 8 minutes and 41 seconds. This helped to absorb the increased transaction load from the past week’s price speculation. Immediately following a difficulty retarget, the average time between blocks is restored to 10 minutes.

    As we transition away from the normal weekend transaction lull into the new week, there’s the possibility for a rapid increase in estimated transaction fees. We recommend being careful sending large low-fee transactions such as consolidations until closer to the weekend when transaction volume begins to taper off again.

Field Report: Consolidation of 4 Million UTXOs at Xapo

by Anthony Towns, Developer on Bitcoin Core at Xapo

As mentioned in newsletter #3, the past few months of low transaction fees makes it a great time to do UTXO consolidation! Consolidation has been one of a variety of activities Xapo has been undertaking to be prepared for the next time fees spike like they did in the last few months of 2017.

Plot of total Bitcoin UXTOs, January - July 2018
Plot of total Bitcoin UXTOs, January - July 2018, source: Statoshi

The idea behind UTXO consolidation is essentially this: when your average outgoing payment is larger than your average incoming payment (or when they’re the same, but you’re batching outgoing payments), you’ll often have to combine many UTXOs in order to fund an outgoing transaction, which increases the size of your transactions and hence the fees you pay. By consolidating UTXOs in advance, you can combine inputs ahead of time, giving you more control over when most of those costs are incurred. If you can do it when fees are low, that lets you reduce those costs pretty substantially.

For example, if you would have spent a dozen 2-of-3 multisig inputs at 100 s/B (satoshis per byte), that would cost around 360,000 satoshis; while if you consolidated those inputs beforehand at 2 s/B, and then spent the single consolidated input later at 100 s/B, your total cost for the two transactions is only about 41,000 satoshis: i.e. 87% less paid in fees. And if fees don’t rise the risk isn’t huge: if fees just sat at 2 s/B, you’d be spending 7,900 satoshis across two transactions if you consolidated, rather than spending 7,200 satoshis in a single transaction if you did nothing.

Consolidation also gives an opportunity to update the addresses you use for your UTXOs, for example to roll keys over, switch to multisig, or switch to segwit or bech32 addresses. And reducing the number of UTXOs makes it easier to run a full node too, marginally improving Bitcoin’s decentralisation and overall security, which is always nice.

Of course, one thing you really don’t want to have happen is for your consolidation transactions to somehow fill up the blockchain and cause fees to immediately start rising! There are two metrics to watch to avoid this risk: one is whether the mempool is full (which causes the minimum acceptable fee to rise), and the other is how much empty space there has been in recent blocks (which gives an indication of whether miners will accept more transactions at the minimum fee). Both these metrics have been very promising most of the time over the past few months: the mempool has regularly been close to empty, meaning the transactions paying as little as 1 s/B have been propagated to miners; and many blocks have not been full, meaning cheap consolidation transactions will get mined reasonably quickly rather than creating a backlog that will cause fees to rise.

The approach we took to actually doing the consolidation was to have a script that would select groups of small UTXOs and create a consolidation transaction spending them to a single pool address at a fee rate of 1.01 satoshis per byte. The script gradually feeds consolidation transactions into the network, so it doesn’t cause too large a spike in the mempool, and perhaps more importantly so we don’t risk having our transactions get dropped because they have low fees and the mempool has filled up. We triggered this manually when we were comfortable it wouldn’t interfere with our operations, and when there didn’t seem to be much load on the Bitcoin network in general.

All in all, this has worked out pretty well; we’ve reduced our UTXO count by something like 4 million UTXOs this year, and aside from some concerned redditors, the cost to the network as a whole has been minimal, as has the cost to us.

Additional resources

News

  • “Improvements in the Bitcoin Scripting Language” by Pieter Wuille: a talk last week giving a high-level overview of several possible near-term improvements to Bitcoin. We highly recommend watching the video, viewing the slides, or reading the transcript (with references) by Bryan Bishop—but if you’re too busy, here’s Wuille’s conclusion: “my initial focus here is Schnorr signatures and taproot. The reason for this focus is that the ability to make any input and output in the cooperative case look identical is an enormous win for how script execution works.

    “Schnorr is necessary for this because without it we cannot encode multiple parties into a single key. Having multiple branches in there is a relatively simple change.

    “If you look at the consensus changes necessary for these things, it’s really remarkably small, dozens of lines of code. It looks like a lot of the complexity is in explaining why these things are useful and how to use them and not so much in the impact on the consensus rules.

    “Things like aggregation, I think, are something that can be done after we have explored various options for structural improvements to the scripting language, once it’s clear around what the structuring should be, because we will probably learn from the deployments how these things get used in practice. That’s what I’m working on with a number of collaborators and we’ll hopefully be proposing something soon, and that’s the end of my talk.”

Selected Q&A from Bitcoin StackExchange

Bitcoin StackExchange is one of the first places Optech contributors look for answers to their questions—or when we have a few spare moments of time to help answer other people’s questions. In this new monthly feature, we highlight some of the top voted questions and answers made there in the past month.

  • Schnorr versus ECDSA: in this answer, Bitcoin protocol developer Pieter Wuille explains some of the chief advantages of the Schnorr signature scheme over Bitcoin’s current ECDSA signature scheme.

  • Why does HD key derivation stop working after a certain index in BIP44 wallets?: a developer testing his wallet finds that a payment sent to low-numbered key indexes works as expected, but payments sent to high-numbered indexes never appear in his wallets. An answer from Viktor T. reveals why.

  • The maximum size of a Bitcoin DER-encoded signature is…: in this answer, Pieter Wuille provides the math for calculating the size of a Bitcoin signature. As mentioned in Newsletter #3, the maximum size using a regular wallet is 72 bytes—but Wuille explains how you can create a non-standard transaction with a 73 byte signature and why you might think you saw a signature that was 74 or even 75 bytes.

  • If you can use almost any opcode in P2SH, why can’t you use them in scriptPubKeys?: in this answer, Bitcoin technical writer David A. Harding explains why early versions of Bitcoin restricted the types of transactions that could be sent to “standard transactions” and why most of those restrictions are still in place even though almost all opcodes are available for standard use now via P2SH and segwit P2WSH.

Notable commits

A quick look at recent merges and commits made in various open source Bitcoin projects.

  • Bitcoin Core #12257: if you start Bitcoin Core with the optional flag -avoidpartialspends, the wallet will by default spend all outputs received to the same address whenever any one of them would be spent. This prevents two outputs to the same address from being spent in separate transactions, which is a common way address reuse reduces privacy. The downside is that it may make transactions larger than the smallest they need to be. Bitcoin businesses using Bitcoin Core’s built-in wallet who don’t need the extra privacy may still want to toggle this flag on when fees are low for automatic consolidation of related inputs.

  • LND #1617: updates estimates for the size of on-chain transactions to prevent transactions from accidentally paying too low of a fee and getting stuck. This commit may be interesting for anyone wondering about the size of the transactions (and parts of transactions) produced in the current protocol.

  • LND #1531: the daemon no longer looks for spends in the memory pool—it waits for them to be a confirmed part of a block first. This allows the same code to work on full nodes like Bitcoin Core and btcd as well as on BIP157-based lightweight clients that don’t have access to unconfirmed transactions. This is part of the ongoing effort to help people without full nodes use LN.

  • In several commits, C-lightning developers have mostly completed the transition from handling peer-related functions in gossipd to handling them in channeld or connectd as appropriate.

  • C-lightning has improved its secret handling so that secrets and signatures are always generated and stored by a separate daemon than the parts of the system directly connected to the network.