Last week I had the honor of hosting the key Lightning developers in my hometown of Adelaide, Australia. I put this article on yalls.org for a week before publishing it on here Medium (the resulting US$6 has gone into my c-lightning testing node 024b9a1fa8e006f1e3937f65f66c408e6da8e1ca728ea43222a7381df1cc449605 which I use for random test payments: thanks!).

The Path To The Summit

Once we had announced a spec freeze for 1.0, there was always an intention to have a face-to-face meeting to decide on targets for 1.1. Unlike Milan, I worried that we’d be overwhelmed with explaining details to newcomers and not be productive, so I modelled the Summit on the Kernel Summit, which is invitation only. Elizabeth Stark, Pierre-Marie Padiou and I formed the committee; we set a hard-cap at 20 people, and we gave three places to each of Lightning Labs, ACINQ and Blockstream, then published a “request for invitation” Google Form for everyone else. We then selected the remaining attendees based on their existing contributions to the specification, and ACINQ and Blockstream also contributed financial support in case that was a barrier to any invited attendee.

There was a Google Doc for what we should discuss, which I organized roughly into an Agenda the week before the Summit.

The Summit Day #1

The first day of the summit began at 9am, with all gathered on a U-shaped table at the Stamford Grand on a cool, cloudy day with a welcome and quick introduction to Chatham House rules (Blockstream footed the bill, as hosts, for both the conference facilities and the catering). We then did a quick pass through the points on the Doc, assigning a “champion” to lead discussion on each point. Stepping through everything in front of us focused our minds, and also daunted us enough that we agreed that anything requiring a bitcoin softfork (Schnorr, Eltoo, SIGHASH_NOINPUT) would be deferred.

We chose a single (trivial) issue to test our process: the “correct” formatting of a short-channel-id. Currently Eclair uses hex (eg 08645B00061E00) lnd uses decimal (eg. 2362141818887680), and c-lightning uses colon-separated decimal (eg. 549979:1566:0) This gave us an opportunity to experience bike-shedding and humorous debate at its finest, with the final decision being 549979x1566x0 (which is readable, and easy to select on all platforms we tried).

After morning tea, we started making reasonable progress on the issues from top to bottom. I failed to gain consensus on many of my more ambitious attempts to remove some knobs from the protocol, but we our progress was quite solid until we hit the exact questions around removal of commitment fees.

The First Really Hard Question

For background, we try to create commitment transactions with sufficient fees that they will be usable when we need them, but this is both a practical problem (peers disagree on fees) and theoretical (predicting the future is hard). So there was universal desire to switch to a better mechanism: as we cannot alter commitment transaction txids, Replace-by-Fee is not possible, so the idea was to have a “pushme” output which exists solely to allow Child-Pays-For-Parent.

Unfortunately, designing this in detail is difficult: there is a DoS where an attacker can create a huge transaction which spends this output but is too low-fee to enter a block. The current RBF rules in bitcoind require any replacement to increase the entire fee, which means you could require a disproportionately large fee (500x normal) to push the transaction, and lightning nodes would have to know this was happening.

Yet the idea of allowing anyone to spend the “pushme” output is appealing, as it avoids spamming the UTXO set; if the transaction goes in without pushing, at least someone could collect the output later. We were at an impasse.

We took a break, stretched our legs and many wandered out onto the balcony overlooking the beach for some fresh air. It was fascinating to see so many great minds mulling different approaches, but it was during the break that we came up with a scheme which didn’t immediately fall apart. The exact details will be debated on the list, but the idea is to have one 1000-satoshi “pushme” output for each side, which can be spent by that side’s key for CPFP, OR anyone after 10 confirms. And a corner-case DoS possibility change will be proposed for bitcoind.

The Dinner

ACINQ paid for a 3-course meal for everyone at The Wharf, a few minutes walk away. The meal was excellent and the company even better: I was a bit regretful to leave around 10pm, knowing we had a full day still to go.

Day 2: Everything Else

As we’d only decided on 20 of the 60 points in front of us, we got straight to work on the second and last day. We began with a quick presentation on Nayuta’s Lightning protocol decoding plugin for Wireshark. I expect every implementation to add support for it very soon, as it’s extremely helpful for debugging.

Fees in general continued to be a headache: currently for simplicity the initiator of the channel pays the onchain fees except for mutual close. After going around a few times, we have kept that scheme, even if both peers end up contributing to the channel. The perfect is the enemy of the good sometimes.

The question of errors came up: we’re seeing far more stuck payments than we would expect. It turns out that c-lightning is the only implementation which actively checks for peer liveness before committing to an HTLC; I expect the others to start doing so. In the meantime, the summit recommendation was for nodes to randomly probe the network to encourage reliability; if a channel is going to fall over as soon as it’s used, and lock up a payment for a few hours, better that it be a small-value probe than a real payment.

Two proposals surprised me, though. One was the decision to use Schnorr signature scheme to dramatically shrink gossip; someone brighter than me noted that we can use these signatures for our own channel_announcement messages, combining 4 signatures into 1. We can save another 32 bytes by making bitcoin mainnet the default (avoiding the chain_hash field). The jury is still out on the idea of using a more primitive Schnorr method on the channel_update signatures, which would allow us to provide lite nodes with a batch of channel_update but a single combined signature which proved them all. [Note: I was wrong, we need pairing crypto for this!]

The second was a method of merchant anonymity: we currently only provide payer anonymity, and the merchant tells they payer where to send the payment. Christian Decker has a (we think!) workable scheme which only requires nodes to do a simple tweak, then merchants could provide a “pre-made” payment onion from any node to themselves. The payer would create a route to that node, and complete the onion generation. While the chosen node would have to tweak the onion, an implementation should also require such tweaks randomly so that there’s no distinction between such a rendevous setup and a normal payment.

Conclusion

Even without considering (very desirable) bitcoin upgrades, we have more than enough improvements to lightning in the pipeline. As expected, we get dual-funding (option_dual_fund), advertisement for dual funding (option_will_fund_for_food), and splicing funds in and out of channels (option_splice). The teams are now nicely familiar with protocol upgrades and the pain points our early users are experiencing. The side-conversations on future work were extremely distracting, in the most excellent way.

And there’s a camaraderie among and between the teams which makes Lightning such a welcoming and fun community to develop with, and the reason I get up in the morning for 5:30am calls. Hosting these discussions challenged me intellectually, and the insights I gained from others delighted me.

I can’t wait to see what the next three years of Lightning development looks like!

My Experience of the Second Lightning Developer Summit was originally published in Blockstream Engineering Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.