LND v0.19.0-beta Release Candidate

[saubyk]

Hey Everyone,

The first release candidate for the upcoming LND v0.19.0 has been published. This release covers a lot in scope include DB migration for Invoices to a native SQL schema, RBF Coop close and optimal block synchronization within LND subsystems to name the highlights. We do expect a few rounds of RCs to iron out potential issues found during testing. Please test this release on your testing nodes.

The release can be found here:
https://github.com/lightningnetwork/lnd/releases/tag/v0.19.0-beta.rc1

The detailed release notes are available at:
https://github.com/lightningnetwork/lnd/blob/master/docs/release-notes/release-notes-0.19.0.md

Below are some of the release highlights:

RBF Coop Close
This release includes significant internal changes to how lnd handles protocol state machines and peer communication, paving the way for more flexible and robust protocol implementations. The key changes are:

• Introduction of the protofsm Package: A new package, protofsm, has been added to provide a generic framework for driving event-driven protocol Finite State Machines (FSMs). This package abstracts away common themes in state machine development, such as defining states, transitions, handling I/O via daemon adapters, cleanup, and notifying callers of state updates. This will allow developers to focus primarily on the logic of state transitions.
• Implementation of a new RBF Cooperative Channel Close: A new Replace-By-Fee (RBF) based cooperative channel close state machine has been added using the protofsm package. This new state machine also functions as a peer.MsgEndpoint, allowing it to integrate with the new message router. The new co-op close process allows either party to initiate a close and to bump fees by sending the shutdown message again, enabling retries with higher fee rates.
• Integration of the new RBF Co-op Close into the Peer Flow: The new RBF co-op close FSM has been integrated into the existing peer connection flow. This involves registering the new state machine with the message router for eligible channels upon peer connection. The integration also includes handling restart scenarios and adds end-to-end tests to verify the RBF cooperative close process.

This new feature can be activated with a new config flag: --protocol.rbf-coop-close. With this new co-op close type, users can issue multiple lncli closechannnel commands with increasing fee rates to use RBF to bump an existing signed co-op close transaction.

Migration of invoices to native SQL
This release introduces the functionality to migrate existing key-value (KV) invoices to a new native SQL schema for users who have configured a KV SQL backend (like Postgres or SQLite). This migration will be triggered automatically on startup when the --db.use-native-sql flag is set.
How to Perform or Skip the Migration:

• Performing the Migration: The migration will occur automatically if you are using a KV SQL backend (e.g., Postgres or SQLite) and have set the --db.use-native-sql flag when starting lnd. No additional steps are required to initiate the migration under these conditions.
• Skipping the Migration (if required): You can skip the invoice migration by also specifying the --db.skip-native-sql-migration flag when starting lnd.
• Important Considerations:
  • This migration is specifically for users already utilizing a KV SQLite/Postgres backend. Users of bbolt or etcd will require an initial step to migrate their entire KV database to SQLite/Postgres before the invoice migration can occur. Please refer to multi: add migrate-db command lightninglabs/lndinit#21 for more context on this initial KV to SQL migration. Please note that this script is still undergoing review and will be published before the final release of lnd v0.19.0
  • Once the invoice migration to the native SQL database is complete, a tombstone key will be set in the invoice bucket. This prevents reverting back to the KV invoice database, ensuring data consistency and avoiding potential issues like lingering invoices or partial state in KV tables. This tombstone will be checked on startup to block any manual attempts to revert the migration.

Blockbeat
This release introduces a significant internal refactor focused on improving the handling of new blocks within lnd through a new service called Blockbeat. This change aims to enhance the reliability and consistency of various subsystems by providing a more synchronized view of the blockchain.
Key changes:

• A new chainio package has been introduced to handle blockchain data access.
• The Blockbeat service has been implemented to manage the distribution of new blocks to registered consumer subsystems of block data.
• Subsystems like ChainArb, UtxoSweeper, and TxPublisher are now registered as Blockbeat consumers and receive new block information sequentially for processing.
• Several internal improvements were made to the itest miner, and itests were fixed and refactored to accommodate the new Blockbeat behavior. This includes removing hacks related to the sweeper, reflecting the new sequential processing of blocks, flattening multi-hop tests for better maintainability, and removing redundant block mining.
• The sweeper's handling of broadcast errors has been improved, and new sweeping states and checks for input maturity have been added as preparatory steps for Blockbeat.
• User benefits
  • Improved Reliability: The sequential and synchronized processing of blocks by critical subsystems should lead to more consistent and reliable behavior, reducing the likelihood of race conditions or inconsistencies arising from different subsystems having different views of the blockchain height.
  • Enhanced Stability: By addressing issues related to block synchronization, this refactor contributes to a more stable lnd implementation. The fixes in the integration tests further validate this improved stability.
  • Simplified Development and Maintenance: The refactoring of testing utilities and multi-hop tests makes the codebase easier to understand, debug, and maintain in the future. The clearer separation of responsibilities in block handling should also simplify future development efforts.

This Blockbeat refactor lays the groundwork for further improvements in lnd's core functionality by establishing a more robust and synchronized approach to handling blockchain updates.

Rpc deprecation
The SendToRoute, SendToRouteSync, SendPayment, and SendPaymentSync endpoints will be removed in release 0.21. In preparation for this removal, deprecation warnings have been added to these endpoints in the current release. Users should migrate to the new v2 methods: SendToRouteV2, SendPaymentV2, and TrackPaymentV2.

Other improvements and bug fixes

• Support added for testnet4
• The lncli constrainmacaroon subcommand now includes an ip_range flag, allowing users to restrict access to a resource to a specific IP range when using a macaroon (authentication token). Previously, macaroons could only allow or deny access based on specific IP addresses, not ranges.
• This update introduces a configurable limit on initial access permissions for certain peers, managed by the --num-restricted-slots flag. Access levels are assigned based on channel history: peers with a confirmed channel receive protected access, those with an unconfirmed channel receive temporary access, and all others are given restricted access.
• The lncli closechannel command now supports cooperative channel closures even when there are active HTLCs. LND will halt the channel to prevent new HTLCs and wait for existing HTLCs to be resolved before starting negotiation. To enable this, users must set the no_wait parameter. This update also ensures that the max_fee_rate setting is enforced for both parties during cooperative channel closure.
• Fixed an edge case where the payment may get stuck if the invoice times out while the node restarted.
• Support for experimental endorsement signal relay was added, to assist in ongoing channel jamming research
• The introduction of namespaced storage for mission control data facilitates path finding A/B testing without impacting standard observations.

There are a lot of other bug fixes and minor changes which can be tracked via the release notes linked above. Reports of issues found during testing will be highly appreciated 🫂

Thanks

[twofaktor]

manifest-roasbeef-v0.19.0-beta.rc2.sig file was forgotten on the rc2 release?

https://github.com/lightningnetwork/lnd/releases/tag/v0.19.0-beta.rc2