Skip to content

Trampoline Payload Construction Method #3386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

arik-so
Copy link
Contributor

@arik-so arik-so commented Oct 28, 2024

Introduce a method for constructing Trampoline payloads.

This will allow the non-Trampoline method to be modified to generate a Trampoline onion and finish its own regular payload construction.

Builds on top of #3333.

@@ -454,6 +454,9 @@ impl_writeable_tlv_based!(BlindedTail, {
pub struct Path {
/// The list of unblinded hops in this [`Path`]. Must be at least length one.
pub hops: Vec<RouteHop>,
/// The list of unblinded Trampoline hops. If present, must be at least one.
/// The public key of the first Trampoline hop must match the public key of the last regular hop.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why this needs to be true. Can we just have each trampoline hop be next hops starting after the last unblinded node.

@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch 2 times, most recently from 7cad33e to 856ce97 Compare November 19, 2024 23:04
@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch from 856ce97 to 5ea3ada Compare November 21, 2024 17:15
Copy link

codecov bot commented Nov 21, 2024

Codecov Report

Attention: Patch coverage is 91.59236% with 66 lines in your changes missing coverage. Please review.

Project coverage is 89.31%. Comparing base (31b32c5) to head (e5bda32).
Report is 41 commits behind head on main.

Files with missing lines Patch % Lines
lightning/src/ln/onion_utils.rs 90.50% 20 Missing and 12 partials ⚠️
lightning/src/routing/router.rs 53.33% 17 Missing and 4 partials ⚠️
lightning/src/events/mod.rs 33.33% 8 Missing ⚠️
lightning/src/ln/onion_route_tests.rs 98.14% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3386      +/-   ##
==========================================
+ Coverage   88.53%   89.31%   +0.78%     
==========================================
  Files         149      149              
  Lines      114567   122538    +7971     
  Branches   114567   122538    +7971     
==========================================
+ Hits       101428   109443    +8015     
- Misses      10645    10694      +49     
+ Partials     2494     2401      -93     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I guess my previous comments still need to be addressed.

@@ -404,13 +404,16 @@ pub struct BlindedTail {
pub excess_final_cltv_expiry_delta: u32,
/// The total amount paid on this [`Path`], excluding the fees.
pub final_value_msat: u64,
/// Used for determining the type of Trampoline path to construct
pub final_hop_supports_trampoline: bool
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we instead consider storing a *Features of some form?

@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch from 5ea3ada to af1818b Compare December 5, 2024 01:19
@arik-so arik-so mentioned this pull request Dec 5, 2024
31 tasks
@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch 4 times, most recently from ee68783 to 4dd0fea Compare December 9, 2024 05:31
@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch 4 times, most recently from dc679c0 to 7477b1a Compare December 12, 2024 17:14
cur_value_msat += final_value_msat;
callback(
PayloadCallbackAction::PushBack,
msgs::OutboundTrampolinePayload::BlindedReceive {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like in https://github.com/lightning/bolts/blob/fa0594ac2af3531d734f1d707a146d6e13679451/bolt04/trampoline-payment-onion-test.json there can be final trampoline hops that are unblinded, but that doesn't look supported in the code atm. Are we explicitly only supporting trampoline to blinded paths?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, we deliberately are. We originally supported non-blinded destinations but decided not to.

@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch from 7477b1a to ba71d6e Compare December 13, 2024 10:34
@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch 4 times, most recently from 09c7f70 to adb0b88 Compare December 15, 2024 22:58
@valentinewallace
Copy link
Contributor

Haven't looked closely yet but it looks like the non-test changes in the latest commit belong in earlier commit(s). Feel free to squash IMO!

@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch from adb0b88 to 9535d78 Compare December 16, 2024 19:38
@arik-so
Copy link
Contributor Author

arik-so commented Dec 16, 2024

I think the squashing should have taken care of that, though let me know if you'd like any other changes move around.

/// The node_announcement features of the node at this hop. For the last hop, these may be
/// amended to match the features present in the invoice this node generated.
pub node_features: NodeFeatures,
/// The fee taken on this hop (for paying for the use of the *next* channel in the path).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it the fee taken for the next several hops until the next trampoline node, or the intro node? I think the docs for this field might need a glance to make sure they're accurate for trampoline, same for cltv_expiry_delta

@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch 2 times, most recently from ab320ba to e191a3d Compare December 18, 2024 07:18
Copy link
Contributor

@valentinewallace valentinewallace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -110,24 +110,167 @@ pub(crate) fn next_hop_pubkey<T: secp256k1::Verification>(
curr_pubkey.mul_tweak(secp_ctx, &Scalar::from_be_bytes(blinding_factor).unwrap())
}

pub(super) trait HopInfo {
trait HopInfo {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this fixup 4911a3a belongs to the commit Trampoline payload construction method instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this produced significant merge conflicts, I'm testing that reordering

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like I got it to work

@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch 3 times, most recently from 97b2adb to 68f4bcf Compare January 29, 2025 01:55
Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, looking again not sure I actually understand the new Route changes?

@@ -459,6 +484,8 @@ impl_writeable_tlv_based!(BlindedTail, {
pub struct Path {
/// The list of unblinded hops in this [`Path`]. Must be at least length one.
pub hops: Vec<RouteHop>,
/// The list of unblinded Trampoline hops. When using Trampoline, must contain at least one hop.
pub trampoline_hops: Vec<TrampolineHop>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need to update the fee_msat and final_value_msat methods? And probably final_cltv_expiry_delta too?

Also, how exactly are we handling trampoline in terms of hops? For blinded tail we implicitly "included" the blinded path in the last RouteHop ('s fees/CLTV) because it let existing code still work without looking at the blinded path itself. I assume we're not doing that here (which I think is okay?) but we should probably make that clearer in the docs.


let trampoline_paths: Vec<Vec<TrampolineHop>> = trampoline_paths.unwrap_or(Vec::new());
if !trampoline_paths.is_empty() {
if trampoline_paths.len() != paths.len() { return Err(DecodeError::InvalidValue) }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this?

@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch from 68f4bcf to d51bcc7 Compare February 3, 2025 06:06
@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch from d51bcc7 to c52eabc Compare February 4, 2025 19:26
/// The list of unblinded Trampoline hops. When using Trampoline, must contain at least one hop.
///
/// Note that the first TrampolineHop node must also be present as the last RouteHop node, where
/// the RouteHop's fee_msat is the total outgoing amount set to the Trampoline hop, whereas
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not sure I understand this, is this right?

Suggested change
/// the RouteHop's fee_msat is the total outgoing amount set to the Trampoline hop, whereas
/// the RouteHop's fee_msat is the total outgoing amount sent to all [`TrampolineHop`]s, combined, including any blinded path, whereas

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, your phrasing is clearer, I think. Really the underlying reason is us not having a final_value_msat equivalent like we do with blinded paths, so that's what helps us know the total amount that is being funneled onwards into the Trampoline route.

@@ -12597,7 +12597,7 @@ impl Readable for HTLCSource {
// instead.
payment_id = Some(PaymentId(*session_priv.0.unwrap().as_ref()));
}
let path = Path { hops: path_hops, blinded_tail };
let path = Path { hops: path_hops, trampoline_hops: vec![], blinded_tail };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing trampoline hops serialization in this struct

@@ -35,6 +35,8 @@ use bitcoin::secp256k1::{PublicKey, Scalar, Secp256k1, SecretKey};
use crate::io::{Cursor, Read};
use core::ops::Deref;

use crate::ln::msgs::{FinalOnionHopData, OutboundOnionPayload, TrampolineOnionPacket};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: put this up with the other msgs:: import

@@ -400,6 +400,8 @@ pub struct RouteHop {
/// If this is the last hop in [`Path::hops`]:
/// * if we're sending to a [`BlindedPaymentPath`], this is the fee paid for use of the entire
/// blinded path
/// * if we're sending to a Trampoline entrypoint, this is the total incoming amount to the
/// Trampoline hop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this change is in the wrong commit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved

secp_ctx,
&path,
&path.hops,
path.blinded_tail.as_ref(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bump

_total_msat: u64, _amt_to_forward: u64, _outgoing_cltv_value: u32,
_recipient_onion: &'a RecipientOnionFields, _packet: TrampolineOnionPacket,
) -> Self {
unreachable!("Trampoline onions cannot contain Trampoline entrypoints");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still would prefer to error over having a buried panic

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@@ -1014,6 +1014,138 @@ fn test_trampoline_onion_payload_serialization() {
assert_eq!(carol_payload_hex, "2e020405f5e10004030c35000e2102edabbd16b41c8371b92ef2f04c1185b4f03b6dcd52ba9b78d9d7c89c8f221145");
}

#[test]
fn test_trampoline_onion_payload_assembly_values() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a comment with a high level summary of what this test is for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

onion_keys,
prng_seed,
payment_hash,
None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably have a TODO that this should be fixed size at some point

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -493,12 +493,22 @@ pub struct Path {
impl Path {
/// Gets the fees for a given path, excluding any excess paid to the recipient.
pub fn fee_msat(&self) -> u64 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this method a little hard to read atm. Looks like it can be simplified:

	pub fn fee_msat(&self) -> u64 {
		let num_unblinded_hops = self.hops.len() + self.trampoline_hops.len();
		let unblinded_hop_fees_iter = self.hops.iter().map(|hop| hop.fee_msat)
			.chain(self.trampoline_hops.iter().map(|hop| hop.fee_msat));
		match &self.blinded_tail {
			Some(_) => unblinded_hop_fees_iter.sum::<u64>(),
			None => {
				// Do not count last hop of each path since that's the full value of the payment
				unblinded_hop_fees_iter.take(num_unblinded_hops - 1).sum()
			}
		}
	}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sadly this is not quite correct, but I tried a different approach to simplify the arithmetic

@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch 3 times, most recently from f7570d6 to 32d576a Compare February 5, 2025 01:02
@@ -400,6 +400,8 @@ pub struct RouteHop {
/// If this is the last hop in [`Path::hops`]:
/// * if we're sending to a [`BlindedPaymentPath`], this is the fee paid for use of the entire
/// blinded path
/// * if we're sending to a Trampoline entrypoint, this is the total incoming amount to the
/// Trampoline hop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Trampoline hop
/// Trampoline entrypoint

@@ -459,36 +486,58 @@ impl_writeable_tlv_based!(BlindedTail, {
pub struct Path {
/// The list of unblinded hops in this [`Path`]. Must be at least length one.
pub hops: Vec<RouteHop>,
/// The list of unblinded Trampoline hops. When using Trampoline, must contain at least one hop.
///
/// Note that the first TrampolineHop node must also be present as the last RouteHop node, where
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in general, please tick and link links (also elsewhere):

Suggested change
/// Note that the first TrampolineHop node must also be present as the last RouteHop node, where
/// Note that the first [`TrampolineHop`] node must also be present as the last [`RouteHop`] node, where

/// The list of unblinded Trampoline hops. When using Trampoline, must contain at least one hop.
///
/// Note that the first TrampolineHop node must also be present as the last RouteHop node, where
/// the RouteHop's fee_msat is the total outgoing amount sent to all [`TrampolineHop`]s combined,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes no sense, there is no "total outgoing amount sent to all trampoline hops combined", and it seems to imply its referring to the fees received?

Suggested change
/// the RouteHop's fee_msat is the total outgoing amount sent to all [`TrampolineHop`]s combined,
/// that RouteHop's fee_msat is the total amount received by the first [`TrampolineHop`],

// If we don't have a blinded tail or if we have Trampoline hops, we skip the last RouteHop's fee
unblinded_hop_fee_iterator.skip(1).sum()
} else {
unblinded_hop_fee_iterator.sum()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to skip the last hop here, too, as the last hop in the normal case is the amount we're paying, not a fee.

.chain(self.trampoline_hops.iter().map(|hop| hop.fee_msat));

if self.blinded_tail.is_none() || !self.trampoline_hops.is_empty() {
// If we don't have a blinded tail or if we have Trampoline hops, we skip the last RouteHop's fee
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused, the old code didn't do this? The old code included the last hop's fee if it was to a blinded path.

@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch 2 times, most recently from 733c53c to e5bda32 Compare February 5, 2025 23:53
@valentinewallace
Copy link
Contributor

@TheBlueMatt how committed are we to only supporting trampoline to blinded paths? I know it's late in the game, but it seems like if we're committed then it might simplify the code to put the trampoline hops in the BlindedTail (maybe with a rename), unless we want the router specifically to be more generic?

@TheBlueMatt
Copy link
Collaborator

@TheBlueMatt how committed are we to only supporting trampoline to blinded paths? I know it's late in the game, but it seems like if we're committed then it might simplify the code to put the trampoline hops in the BlindedTail (maybe with a rename), unless we want the router specifically to be more generic?

I think fairly committed? I mean I'm definitely open to rethinking that if we want to support clients without a network graph sending payments to a non-blinded destination (ie supporting phoenix/electrum clients sending to bolt 11 destinations), but I dunno if we have a big market for that tbh (phoenix will just use eclair as the trampoline anyway and electrum clients maybe, but we're probably better off convincing them to use RGS) and RGS has worked really well in practice.

All that to say I don't have any specific views on putting the trampoline hops in the blinded tail or not. Fine with doing that, fine with not doing that.

To define the hops in a Path that are meant as Trampoline hops, we
need to introduce a new hop type much alike `RouteHop`, except these
hops would not be defined by channel ID used to route payments.

Instead, the preceding Trampoline hop would determine its own optimal
path to route the payment onwards.
@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch from e5bda32 to 31d5507 Compare February 7, 2025 08:23
Copy link
Contributor

@valentinewallace valentinewallace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Comment on lines 465 to 466
/// Note that the first [`TrampolineHop`] node must also be present as the last [`RouteHop`] node,
/// where the RouteHop's fee_msat is the total amount received by the first [`TrampolineHop`],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, this says the last RouteHop::fee_msat is the total amount received by the first trampoline hop, but the RouteHop::fee_msat docs say that it's the fee paid for all trampoline + blinded hops?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, nope, gotta be the fee, rephrasing

@valentinewallace
Copy link
Contributor

Ah looks like rustfmt is failing as well.

Given that we do not intend to allow sending to unblinded recipients
via Trampoline hops, we're adding a vector of unblinded `TrampolineHop`s
directly to the `BlindedTail` struct.
Trampoline routing relies on serializing an inner onion within the
hop data. In this commit, we introduce the method to serialize it based
on the augmented path data introduced in the previous commit.
Now that we can generate the inner Trampoline onions, we need to update
the `create_payment_onion` utility method to consider the newly
available path data and incorporate the encoded Trampoline onion in the
outer hop data.

This also comes with a behavioral change for blinded tails: when
Trampoline are present, we assume that they precede the blinded tail,
which must therefore be serialized as part of the Trampoline onion.
@arik-so arik-so force-pushed the arik/trampoline/payload-construction branch from 31d5507 to 7b56d51 Compare February 7, 2025 20:19
@valentinewallace
Copy link
Contributor

LGTM, can you take a look @TheBlueMatt?

Comment on lines +167 to +208
trait OnionPayload<'a, 'b> {
type PathHopForId: PathHop + 'b;
type ReceiveType: OnionPayload<'a, 'b>;
fn new_forward(
hop_id: <<Self as OnionPayload<'a, 'b>>::PathHopForId as PathHop>::HopId,
amt_to_forward: u64, outgoing_cltv_value: u32,
) -> Self;
fn new_receive(
recipient_onion: &'a RecipientOnionFields, keysend_preimage: Option<PaymentPreimage>,
sender_intended_htlc_amt_msat: u64, total_msat: u64, cltv_expiry_height: u32,
) -> Result<Self::ReceiveType, APIError>;
fn new_blinded_forward(
encrypted_tlvs: &'a Vec<u8>, intro_node_blinding_point: Option<PublicKey>,
) -> Self;
fn new_blinded_receive(
sender_intended_htlc_amt_msat: u64, total_msat: u64, cltv_expiry_height: u32,
encrypted_tlvs: &'a Vec<u8>, intro_node_blinding_point: Option<PublicKey>,
keysend_preimage: Option<PaymentPreimage>, invoice_request: Option<&'a InvoiceRequest>,
custom_tlvs: &'a Vec<(u64, Vec<u8>)>,
) -> Self;
fn new_trampoline_entry(
total_msat: u64, amt_to_forward: u64, outgoing_cltv_value: u32,
recipient_onion: &'a RecipientOnionFields, packet: msgs::TrampolineOnionPacket,
) -> Result<Self::ReceiveType, APIError>;
}
impl<'a, 'b> OnionPayload<'a, 'b> for msgs::OutboundOnionPayload<'a> {
type PathHopForId = &'b RouteHop;
type ReceiveType = msgs::OutboundOnionPayload<'a>;
fn new_forward(short_channel_id: u64, amt_to_forward: u64, outgoing_cltv_value: u32) -> Self {
Self::Forward { short_channel_id, amt_to_forward, outgoing_cltv_value }
}
fn new_receive(
recipient_onion: &'a RecipientOnionFields, keysend_preimage: Option<PaymentPreimage>,
sender_intended_htlc_amt_msat: u64, total_msat: u64, cltv_expiry_height: u32,
) -> Result<Self::ReceiveType, APIError> {
Ok(Self::Receive {
payment_data: recipient_onion
.payment_secret
.map(|payment_secret| msgs::FinalOnionHopData { payment_secret, total_msat }),
payment_metadata: recipient_onion.payment_metadata.as_ref(),
keysend_preimage,
custom_tlvs: &recipient_onion.custom_tlvs,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few linebreaks in here would be really nice :p


trait OnionPayload<'a, 'b> {
type PathHopForId: PathHop + 'b;
type ReceiveType: OnionPayload<'a, 'b>;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We definitely don't need a type for something that's always Self.


/// Build a payment onion, returning the first hop msat and cltv values as well.
/// `cur_block_height` should be set to the best known block height + 1.
pub(crate) fn create_payment_onion_internal<T: secp256k1::Signing>(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be really nice if we didn't expose test stuff outside of this module, it makes it really hard to figure out what is or isn't a "public" API. Here and build_onion_payloads can we expose them as test-only wrappers?

@TheBlueMatt TheBlueMatt merged commit 7869953 into lightningdevkit:main Feb 7, 2025
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
weekly goal Someone wants to land this this week
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants