Rusty Russell's Coding Blog | Stealing From Smart People

Aug/15

4

The Bitcoin Blocksize: A Summary

There’s a significant debate going on at the moment in the Bitcoin world; there’s a great deal of information and misinformation, and it’s hard to find a cogent summary in one place.  This post is my attempt, though I already know that it will cause me even more trouble than that time I foolishly entitled a post “If you didn’t run code written by assholes, your machine wouldn’t boot”.

The Technical Background: 1MB Block Limit

The bitcoin protocol is powered by miners, who gather transactions into blocks, producing a block every 10 minutes (but it varies a lot).  They get a 25 bitcoin subsidy for this, plus whatever fees are paid by those transactions.  This subsidy halves every 4 years: in about 12 months it will drop to 12.5.

Full nodes on the network check transactions and blocks, and relay them to others.  There are also lightweight nodes which simply listen for transactions which affect them, and trust that blocks from miners are generally OK.

A normal transaction is 250 bytes, and there’s a hard-coded 1 megabyte limit on the block size.  This limit was introduced years ago as a quick way of avoiding a miner flooding the young network, though the original code could only produce 200kb blocks, and the default reference code still defaults to a 750kb limit.

In the last few months there have been increasing runs of full blocks, causing backlogs for a few hours.  More recently, someone deliberately flooded the network with normal-fee transactions for several days; any transactions paying less fees than those had to wait for hours to be processed.

There are 5 people who have commit access to the bitcoin reference implementation (aka. “bitcoin-core”), and they vary significantly in their concerns on the issue.

The Bitcoin Users’ Perspective

From the bitcoin users perspective, blocks should be infinite, and fees zero or minimal.  This is the basic position of respected (but non-bitcoin-core) developer Mike Hearn, and has support from bitcoin-core ex-lead Gavin Andresen.  They work on the wallet and end-user side of bitcoin, and they see the issue as the most urgent.  In an excellent post arguing why growth is so important, Mike raises the following points, which I’ve paraphrased:

  1. Currencies have network effects. A currency that has few users is simply not competitive with currencies that have many.
  2. A decentralised currency that the vast majority can’t use doesn’t change the amount of centralisation in the world. Most people will still end up using banks, with all the normal problems.
  3. Growth is a part of the social contract. It always has been.
  4. Businesses will only continue to invest in bitcoin and build infrastructure if they are assured that the market will grow significantly.
  5. Bitcoin needs users, lots of them, for its political survival. There are many people out there who would like to see digital cash disappear, or be regulated out of existence.

At this point, it’s worth mentioning another bitcoin-core developer: Jeff Garzik.  He believes that the bitcoin userbase has been promised that transactions will continue to be almost free.  When a request to change the default mining limit from 750kb to 1M was closed by the bitcoin lead developer Wladimir van der Laan as unimportant, Jeff saw this as a symbolic moment:

What Happens If We Don’t Increase Soon?

Mike Hearn has a fairly apocalyptic view of what would happen if blocks fill.  That was certainly looking likely when the post was written, but due to episodes where the blocks were full for days, wallet designers are (finally) starting to estimate fees for timely processing (miners process larger fee transactions first).  Some wallets and services didn’t even have a way to change the setting, leaving users stranded during high-volume events.

It now seems that the bursts of full blocks will arrive with increasing frequency; proposals are fairly mature now to allow users to post-increase fees if required, which (if all goes well) could make for a fairly smooth transition from the current “fees are tiny and optional” mode of operation to a “there will be a small fee”.

But even if this rosy scenario is true, this begsavoids the bigger question of how high fees can become before bitcoin becomes useless.  1c?  5c?  20c? $1?

So What Are The Problems With Increasing The Blocksize?

In a word, the problem is miners.  As mining has transitioned from a geek pastime, semi-hobbyist, then to large operations with cheap access to power, it has become more concentrated.

The only difference between bitcoin and previous cryptocurrencies is that instead of a centralized “broker” to ensure honesty, bitcoin uses an open competition of miners. Given bitcoin’s endurance, it’s fair to count this a vital property of bitcoin.  Mining centralization is the long-term concern of another bitcoin-core developer (and my coworker at Blockstream), Gregory Maxwell.

Control over half the block-producing power and you control who can use bitcoin and cheat anyone not using a full node themselves.  Control over 2/3, and you can force a rule change on the rest of the network by stalling it until enough people give in.  Central control is also a single point to shut the network down; that lets others apply legal or extra-legal pressure to restrict the network.

What Drives Centralization?

Bitcoin mining is more efficient at scale. That was to be expected[7]. However, the concentration has come much faster than expected because of the invention of mining pools.  These pools tell miners what to mine, in return for a small (or in some cases, zero) share of profits.  It saves setup costs, they’re easy to use, and miners get more regular payouts.  This has caused bitcoin to reel from one centralization crisis to another over the last few years; the decline in full nodes has been precipitous by some measures[5] and continues to decline[6].

Consider the plight of a miner whose network is further away from most other miners.  They find out about new blocks later, and their blocks get built on later.  Both these effects cause them to create blocks which the network ignores, called orphans.  Some orphans are the inevitable consequence of miners racing for the same prize, but the orphan problem is not symmetrical.  Being well connected to the other miners helps, but there’s a second effect: if you discover the previous block, you’ve a head-start on the next one.  This means a pool which has 20% of the hashing power doesn’t have to worry about delays at all 20% of the time.

If the orphan rate is very low (say, 0.1%), the effect can be ignored.  But as it climbs, the pressure to join a pool (the largest pool) becomes economically irresistible, until only one pool remains.

Larger Blocks Are Driving Up Orphan Rates

Large blocks take longer to propagate, increasing the rate of orphans.  This has been happening as blocks increase.  Blocks with no transactions at all are smallest, and so propagate fastest: they still get a 25 bitcoin subsidy, though they don’t help bitcoin users much.

Many people assumed that miners wouldn’t overly centralize, lest they cause a clear decentralization failure and drive the bitcoin price into the ground.  That assumption has proven weak in the face of climbing orphan rates.

And miners have been behaving very badly.  Mining pools orchestrate attacks on each other with surprising regularity; DDOS and block withholding attacks are both well documented[1][2].  A large mining pool used their power to double spend and steal thousands of bitcoin from a gambling service[3].  When it was noticed, they blamed a rogue employee.  No money was returned, nor any legal action taken.  It was hoped that miners would leave for another pool as they approached majority share, but that didn’t happen.

If large blocks can be used as a weapon by larger miners against small ones[8], it’s expected that they will be.

More recently (and quite by accident) it was discovered that over half the mining power aren’t verifying transactions in blocks they build upon[4].  They did this in order to reduce orphans, and one large pool is still doing so.  This is a problem because lightweight bitcoin clients work by assuming anything in the longest chain of blocks is good; this was how the original bitcoin paper anticipated that most users would interact with the system.

The Third Side Of The Debate: Long Term Network Funding

Before I summarize, it’s worth mentioning the debate beyond the current debate: long term network support.  The minting of new coins decreases with time; the plan of record (as suggested in the original paper) is that total transaction fees will rise to replace the current mining subsidy.  The schedule of this is unknown and generally this transition has not happened: free transactions still work.

The block subsidy as I write this is about $7000.  If nothing else changes, miners would want $3500 in fees in 12 months when the block subsidy halves, or about $2 per transaction.  That won’t happen; miners will simply lose half their income.  (Perhaps eventually they form a cartel to enforce a minimum fee, causing another centralization crisis? I don’t know.)

It’s natural for users to try to defer the transition as long as possible, and the practice in bitcoin-core has been to aggressively reduce the default fees as the bitcoin price rises.  Core developers Gregory Maxwell and Pieter Wuille feel that signal was a mistake; that fees will have to rise eventually and users should not be lulled into thinking otherwise.

Mike Hearn in particular has been holding out the promise that it may not be necessary.  On this he is not widely supported: that some users would offer to pay more so other users can continue to pay less.

It’s worth noting that some bitcoin businesses rely on the current very low fees and don’t want to change; I suspect this adds bitterness and vitriol to many online debates.

Summary

The bitcoin-core developers who deal with users most feel that bitcoin needs to expand quickly or die, that letting fees emerge now will kill expansion, and that the infrastructure will improve over time if it has to.

Other bitcoin-core developers feel that bitcoin’s infrastructure is dangerously creaking, that fees need to emerge anyway, and that if there is a real emergency a blocksize change could be rolled out within a few weeks.

At least until this is resolved, don’t count on future bitcoin fees being insignificant, nor promise others that bitcoin has “free transactions”.


[1] http://www.coindesk.com/bitcoin-mining-pools-ddos-attacks/ “Bitcoin Mining Pools Targeted in Wave of DDOS Attacks” Coinbase 2015

[2] http://blog.bettercrypto.com/?p=1131 “Block Withholding Attacks – Recent Research” N T Courtois 2014

[3] https://bitcointalk.org/index.php?topic=327767.0 “GHash.IO and double-spending against BetCoin Dice” mmtech et. al 2013

[4] https://www.reddit.com/r/Bitcoin/comments/3c8tq2/questions_about_the_july_4th_bip66_fork/cstgajp “Questions about the July 4th BIP66 fork”

[5] https://twitter.com/petertoddbtc/status/608475414449778688 “350,000 full nodes to 6,000 in two years…” P Todd 2015

[6] https://getaddr.bitnodes.io/dashboard/?days=365 “Reachable nodes during the last 365 days.” Bitnodes.io

[7] https://bitcointalk.org/index.php?topic=532.msg6306#msg6306 “Re: Scalability and transaction rate” Satoshi 2010

[8] http://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg08161.html “[Bitcoin-development] Mining centralization pressure from non-uniform propagation speed” Pieter Wuille 2015

RSS Feed

11 Comments for The Bitcoin Blocksize: A Summary

Edmund Edgar | August 6, 2015 at 10:38 am

Are large blocks increasing orphan rates? I’m looking at the chart at blockchain.info and I’m not particularly seeing that trend, apart from the spike around the BIP 66 fork.

https://blockchain.info/charts/n-orphaned-blocks?timespan=2year&showDataPoints=true&daysAverageString=7&show_header=true&scale=0&address=

Author comment by rusty | August 6, 2015 at 11:22 am

blockchain.info is less reliable than I’d like :(

There have been previous papers on propagation time, but Tradeblock have a nice graph here: https://research.tradeblock.com/wp-content/uploads/2015/06/Time-to-3k.png

Edmund Edgar | August 6, 2015 at 1:33 pm

Overall network propagation times are a bit different from orphan rates, because you won’t be able to see the full effects of things miners can do to keep their orphan rates low, like using the relay network, or mining on headers until you have a chance to download and validate a block.

Maybe worth chasing up your source for the orphan rate claim, since it’s quite pivotal to the argument and written in big letters?

Author comment by rusty | August 6, 2015 at 4:10 pm

It’s pretty self-evident, but see https://tradeblock.com/blog/bitcoin-network-capacity-analysis-part-6-data-propagation for their conclusion using real network data.

Non-validation is happening now, because of orphaning; that’s making things worse (network wise), not better. BlueMatt indicated that most miners are already using the Relay network, so I don’t expect significant gains there :(

Edmund Edgar | August 6, 2015 at 8:15 pm

Maybe I’m missing it but the only orphan rate information I could see there was a three-month graph, which was dead flat.

They’re showing that larger blocks take longer to propagate (as you’d expect) but they’re not showing that the network is failing to keep up by improving as fast as demand is growing, which is what you need to establish to say that larger blocks are driving up orphan rates.

I’d agree that I wouldn’t like to bet the house on blockchain.info’s data, although showing little graphics of orphaned blocks is pretty much their core competency. But it seems fair to ask what the actual picture looks like.

On mining on headers, non-validation is indeed very bad for the network, but mining first and validating later is fine within reasonable bounds, as long as you actually stop mining once you discover that a block is invalid. This is apparently where the Chinese miners tripped up on the BIP 66 fork – apparently their own bitcoind’s were telling them the block was invalid, but they didn’t have the code in place to act on that knowledge – and it ended up costing them money that they wouldn’t have lost with systems better approximating what a rational miner would do.

Author comment by rusty | August 15, 2015 at 12:56 pm

https://research.tradeblock.com/wp-content/uploads/2015/06/Orphan-vs-Non-Orphan.png

Frankly, you seem determined not to see this? We’ve established that larger blocks are more likely to be orphaned, and that miners are taking bitcoin-harming measures to avoid orphaning. The conclusion that larger blocks will make this worse seems so reasonable that you’d need to provide evidence that it wasn’t so.

> and it ended up costing them money that they wouldn’t have lost with systems better approximating what a rational miner would do.

Oh I agree, they should never have done it. They did. And they shouldn’t still be doing it. They are. We can argue that’s not rational, but unfortunately that doesn’t make it less of a problem in practice (which is where we need to solve it).

Though it does lead to two theories: the first is that all the rational miners stopped long ago, and we’re left with fanatics who are bad at math ;) More useful is the theory that miners are using voodoo optimizations (the consequences of which they understand poorly) to overcome limitations in bitcoin core. This latter one seems to present an avenue for a solution…

stefanha | August 12, 2015 at 11:47 pm

How about decreasing the average block generation time from 10 minutes to, say, 5 minutes?

It would allow the network to generate more blocks without requiring larger sizes.

Matt Odell | August 13, 2015 at 2:37 am

“A large mining pool used their power to double spend and steal thousands of bitcoin from a gambling service[3]. When it was noticed, they blamed a rogue employee. No money was returned, nor any legal action taken. It was hoped that miners would leave for another pool as they approached majority share, but that didn’t happen.”

IIRC GHash.io bled significant mining pool share following the double spend incident as they closed in at 50% share of the network. I don’t think they ever actually hit 50% and are now at only 2.1% of the network (http://mempool.info/pools). Doesn’t that show that miners are willing to switch pools if a mining pool acts against the best interests of Bitcoin?

Author comment by rusty | August 15, 2015 at 11:21 am

Unfortunately not. The DS was in September 2013 and revealed in October (see https://bitcointalk.org/index.php?topic=321630.0 )

They were around 50% in January 2014 and June 2014 according to https://en.bitcoin.it/wiki/GHash.IO

I’m told the backbone of GHash was Bitfury and CEX.io’s cloud mining. Bitfury started moving to their own pool starting June 2014 in response to the 50% concerns (http://www.coindesk.com/bitfury-pulls-power-ghash-community-uproar/ ) and CEX.io shut off their miners in Jan 2015 due to declining profitability ( http://www.coindesk.com/cex-io-halts-cloud-mining-service-due-low-bitcoin-price/ )

In summary: miners didn’t leave a pool which facilitates double spends. One did respond to the 50% problem after months of calls to do so and 5 months after it first occurred. It’s disappointing, but the conclusion stands: we shouldn’t count on miners to sacrifice their own profitability for more abstract “network health”.

Edmund Edgar | August 13, 2015 at 10:15 am

> How about decreasing the average block generation time from 10 minutes to, say, 5 minutes?

> It would allow the network to generate more blocks without requiring larger sizes.

This may be a good idea for other reasons but what people are worried about here is the proportion of the time badly-connected miners aren’t mining on the latest block. If it takes you 10 seconds out of 10 minutes at 1MB, making it 10 seconds out of 5 minutes doubles it just as much as making it 20 seconds out of 10 minutes at 2MB.

Author comment by rusty | August 15, 2015 at 11:10 am

Reducing time also doesn’t give much improvement. Even the most optimistic estimate I’ve seen (Joseph Poon assuming everyone uses lightning for all transactions and only funds twice a year) requires 133MB blocks.

«

»

Find it!

Theme Design by devolux.org

Tag Cloud