Extended plugin features and dynamic add & remove

Written by Rusty Russell

The c-lightning team is proud to announce the v0.7.2 release of c-lightning. This is the first of our new “every two months” timed releases, and it has improvements across the board. This is a recommended upgrade!

  • Plugin improvements: External contributor Antoine Poinsot contributed dynamic plugins: this means you can now add and remove plugins to a running lightning daemon without restarting.
  • New plugin notifications: invoice_payment tells you when an invoice has been paid, forward_event tells you when an HTLC is forwarded and channel_opened tells you when a new channel has been opened.
  • Better handling of peer errors: together with Connor Fromknecht of Lightning Labs, we diagnosed and fixed a problem where we would receive a mysterious “sync error”. We now ignore this, as it was responsible for the majority of unilateral channel closes we were seeing.
  • Better startup documentation: the README.md file has been overhauled, lightningd now has its own manual page, and there’s a helper script to connect you to a few long-lived nodes when your node first starts up.

A New World of User-Friendly Plugins

Restarting your node to test plugins is somewhat painful for developers: Christian wrote an autoreload plugin which helped, but users also want to play with the growing library of plugins.

Enter the new plugin command:

  • plugin stop and plugin start stops and starts plugins, unless they’ve specifically marked themselves dynamic: False.
  • plugin rescan rescans directories for any new plugins and starts them up.
  • plugin startdir adds a new directory to the list plugin directories to search, then starts any plugins it finds there.
  • If you create a plugins directory inside your .lightning directory, lightning automatically includes that and any immediate subdirectories: a useful place to put all your personal plugins!

Another minor, but important feature for usability: plugin commands can return "format-hint": "simple" in their responses to make lightning-cli default output be human readable rather than JSON. This allows your plugin to do its own formatting, allowing implementation of a cowsay plugin to be more natural (I can’t believe this doesn’t exist yet!).

For more details on the specifics of plugin hooks and notifications, please see the plugin developer docs. And don’t forget to submit your plugin to the c-lightning plugin repository on Github.

A Plague of Channel Closings, Cured

We had some disturbing reports of mass channel closings, caused by peers sending an error packet with the simple word “sync error”. We suspected some subtle integration issue, but were unable to reproduce it, and we even went so far as to test all the corner cases of reconnection in the specification. Worse, it seemed to cluster: one of our test nodes lost eight channels at once on one restart, and there were similar bug reports of mass closings.

We patched lightningd to back off and try again when this error occurred (instead of simply closing the channel), and funded channels to all the nodes mentioned in bug reports to maximize our chance of seeing it again. Indeed, we did, and ignoring it and retrying seemed to eventually resolve it. But we still needed to know why it happened, and whether it had some sinister cause.

With the help of Connor Fromknecht we managed to coax our node into reproducing it against his node. Because that node runs permanently under the valgrind debugger to catch any errors, it can be so slow that it triggers lnd’s internal timeout which causes this mysterious error!

With that understanding, we knew our fix was correct. And to prevent such overload, we also now limit the rate we reconnect to everyone at restart; after the first five channels, we wait about a second between each one. This means even Rusty’s Raspberry Pi should be able to handle a few more channels.

More Protocol Experiments

There’s long been an option when you build c-lightning, called --enable-experimental-features; it just hasn’t done anything until now. With 0.7.2 it adds a proposed protocol extension for gossip, so you can ask for a dense summary of channel updates and selectively query them; Eclair’s mobile wallet has seen impressive speed gains from its use, but the specification is not yet finalized.

This option allows us to test interoperability with them (such 2-implementation testing being a requirement for final spec inclusion) as well as selectively deploy it if all goes well in the final phases.

The increased attention on the specification lead us to work on: generalized testing infrastructure for the protocol, which revealed a longstanding bug fixed in this release. For future compatibility, nodes should ignore odd-numbered messages they don’t recognize, but simple testing revealed that we weren’t doing that. It’s good to solve that now, before we need it!

If you want to get started using c-lightning or c-lightning development, we can be found on IRC (#c-lightning on Freenode) or the Blockstream Slack, as well as on GitHub reviewing bugs, feature requests, and new code!

New Release: c-lightning 0.7.2 was originally published in Blockstream Engineering Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.