-
Notifications
You must be signed in to change notification settings - Fork 402
Trampoline Payload Construction Method #3386
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trampoline Payload Construction Method #3386
Conversation
lightning/src/routing/router.rs
Outdated
@@ -454,6 +454,9 @@ impl_writeable_tlv_based!(BlindedTail, { | |||
pub struct Path { | |||
/// The list of unblinded hops in this [`Path`]. Must be at least length one. | |||
pub hops: Vec<RouteHop>, | |||
/// The list of unblinded Trampoline hops. If present, must be at least one. | |||
/// The public key of the first Trampoline hop must match the public key of the last regular hop. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see why this needs to be true. Can we just have each trampoline hop be next hops starting after the last unblinded node.
7cad33e
to
856ce97
Compare
856ce97
to
5ea3ada
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3386 +/- ##
==========================================
+ Coverage 88.53% 89.31% +0.78%
==========================================
Files 149 149
Lines 114567 122538 +7971
Branches 114567 122538 +7971
==========================================
+ Hits 101428 109443 +8015
- Misses 10645 10694 +49
+ Partials 2494 2401 -93 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I guess my previous comments still need to be addressed.
lightning/src/routing/router.rs
Outdated
@@ -404,13 +404,16 @@ pub struct BlindedTail { | |||
pub excess_final_cltv_expiry_delta: u32, | |||
/// The total amount paid on this [`Path`], excluding the fees. | |||
pub final_value_msat: u64, | |||
/// Used for determining the type of Trampoline path to construct | |||
pub final_hop_supports_trampoline: bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we instead consider storing a *Features
of some form?
5ea3ada
to
af1818b
Compare
ee68783
to
4dd0fea
Compare
dc679c0
to
7477b1a
Compare
lightning/src/ln/onion_utils.rs
Outdated
cur_value_msat += final_value_msat; | ||
callback( | ||
PayloadCallbackAction::PushBack, | ||
msgs::OutboundTrampolinePayload::BlindedReceive { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like in https://github.com/lightning/bolts/blob/fa0594ac2af3531d734f1d707a146d6e13679451/bolt04/trampoline-payment-onion-test.json there can be final trampoline hops that are unblinded, but that doesn't look supported in the code atm. Are we explicitly only supporting trampoline to blinded paths?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yup, we deliberately are. We originally supported non-blinded destinations but decided not to.
7477b1a
to
ba71d6e
Compare
09c7f70
to
adb0b88
Compare
Haven't looked closely yet but it looks like the non-test changes in the latest commit belong in earlier commit(s). Feel free to squash IMO! |
adb0b88
to
9535d78
Compare
I think the squashing should have taken care of that, though let me know if you'd like any other changes move around. |
lightning/src/routing/router.rs
Outdated
/// The node_announcement features of the node at this hop. For the last hop, these may be | ||
/// amended to match the features present in the invoice this node generated. | ||
pub node_features: NodeFeatures, | ||
/// The fee taken on this hop (for paying for the use of the *next* channel in the path). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't it the fee taken for the next several hops until the next trampoline node, or the intro node? I think the docs for this field might need a glance to make sure they're accurate for trampoline, same for cltv_expiry_delta
ab320ba
to
e191a3d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@@ -110,24 +110,167 @@ pub(crate) fn next_hop_pubkey<T: secp256k1::Verification>( | |||
curr_pubkey.mul_tweak(secp_ctx, &Scalar::from_be_bytes(blinding_factor).unwrap()) | |||
} | |||
|
|||
pub(super) trait HopInfo { | |||
trait HopInfo { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this fixup 4911a3a belongs to the commit Trampoline payload construction method
instead
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this produced significant merge conflicts, I'm testing that reordering
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks like I got it to work
97b2adb
to
68f4bcf
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, looking again not sure I actually understand the new Route
changes?
lightning/src/routing/router.rs
Outdated
@@ -459,6 +484,8 @@ impl_writeable_tlv_based!(BlindedTail, { | |||
pub struct Path { | |||
/// The list of unblinded hops in this [`Path`]. Must be at least length one. | |||
pub hops: Vec<RouteHop>, | |||
/// The list of unblinded Trampoline hops. When using Trampoline, must contain at least one hop. | |||
pub trampoline_hops: Vec<TrampolineHop>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't we need to update the fee_msat
and final_value_msat
methods? And probably final_cltv_expiry_delta
too?
Also, how exactly are we handling trampoline in terms of hops
? For blinded tail we implicitly "included" the blinded path in the last RouteHop
('s fees/CLTV) because it let existing code still work without looking at the blinded path itself. I assume we're not doing that here (which I think is okay?) but we should probably make that clearer in the docs.
lightning/src/routing/router.rs
Outdated
|
||
let trampoline_paths: Vec<Vec<TrampolineHop>> = trampoline_paths.unwrap_or(Vec::new()); | ||
if !trampoline_paths.is_empty() { | ||
if trampoline_paths.len() != paths.len() { return Err(DecodeError::InvalidValue) } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure I understand this?
68f4bcf
to
d51bcc7
Compare
d51bcc7
to
c52eabc
Compare
lightning/src/routing/router.rs
Outdated
/// The list of unblinded Trampoline hops. When using Trampoline, must contain at least one hop. | ||
/// | ||
/// Note that the first TrampolineHop node must also be present as the last RouteHop node, where | ||
/// the RouteHop's fee_msat is the total outgoing amount set to the Trampoline hop, whereas |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still not sure I understand this, is this right?
/// the RouteHop's fee_msat is the total outgoing amount set to the Trampoline hop, whereas | |
/// the RouteHop's fee_msat is the total outgoing amount sent to all [`TrampolineHop`]s, combined, including any blinded path, whereas |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, your phrasing is clearer, I think. Really the underlying reason is us not having a final_value_msat
equivalent like we do with blinded paths, so that's what helps us know the total amount that is being funneled onwards into the Trampoline route.
lightning/src/ln/channelmanager.rs
Outdated
@@ -12597,7 +12597,7 @@ impl Readable for HTLCSource { | |||
// instead. | |||
payment_id = Some(PaymentId(*session_priv.0.unwrap().as_ref())); | |||
} | |||
let path = Path { hops: path_hops, blinded_tail }; | |||
let path = Path { hops: path_hops, trampoline_hops: vec![], blinded_tail }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing trampoline hops serialization in this struct
lightning/src/ln/onion_utils.rs
Outdated
@@ -35,6 +35,8 @@ use bitcoin::secp256k1::{PublicKey, Scalar, Secp256k1, SecretKey}; | |||
use crate::io::{Cursor, Read}; | |||
use core::ops::Deref; | |||
|
|||
use crate::ln::msgs::{FinalOnionHopData, OutboundOnionPayload, TrampolineOnionPacket}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: put this up with the other msgs::
import
lightning/src/routing/router.rs
Outdated
@@ -400,6 +400,8 @@ pub struct RouteHop { | |||
/// If this is the last hop in [`Path::hops`]: | |||
/// * if we're sending to a [`BlindedPaymentPath`], this is the fee paid for use of the entire | |||
/// blinded path | |||
/// * if we're sending to a Trampoline entrypoint, this is the total incoming amount to the | |||
/// Trampoline hop |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like this change is in the wrong commit
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
moved
lightning/src/ln/onion_utils.rs
Outdated
secp_ctx, | ||
&path, | ||
&path.hops, | ||
path.blinded_tail.as_ref(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bump
lightning/src/ln/onion_utils.rs
Outdated
_total_msat: u64, _amt_to_forward: u64, _outgoing_cltv_value: u32, | ||
_recipient_onion: &'a RecipientOnionFields, _packet: TrampolineOnionPacket, | ||
) -> Self { | ||
unreachable!("Trampoline onions cannot contain Trampoline entrypoints"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still would prefer to error over having a buried panic
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated
@@ -1014,6 +1014,138 @@ fn test_trampoline_onion_payload_serialization() { | |||
assert_eq!(carol_payload_hex, "2e020405f5e10004030c35000e2102edabbd16b41c8371b92ef2f04c1185b4f03b6dcd52ba9b78d9d7c89c8f221145"); | |||
} | |||
|
|||
#[test] | |||
fn test_trampoline_onion_payload_assembly_values() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add a comment with a high level summary of what this test is for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added
lightning/src/ln/onion_utils.rs
Outdated
onion_keys, | ||
prng_seed, | ||
payment_hash, | ||
None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should probably have a TODO that this should be fixed size at some point
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
lightning/src/routing/router.rs
Outdated
@@ -493,12 +493,22 @@ pub struct Path { | |||
impl Path { | |||
/// Gets the fees for a given path, excluding any excess paid to the recipient. | |||
pub fn fee_msat(&self) -> u64 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this method a little hard to read atm. Looks like it can be simplified:
pub fn fee_msat(&self) -> u64 {
let num_unblinded_hops = self.hops.len() + self.trampoline_hops.len();
let unblinded_hop_fees_iter = self.hops.iter().map(|hop| hop.fee_msat)
.chain(self.trampoline_hops.iter().map(|hop| hop.fee_msat));
match &self.blinded_tail {
Some(_) => unblinded_hop_fees_iter.sum::<u64>(),
None => {
// Do not count last hop of each path since that's the full value of the payment
unblinded_hop_fees_iter.take(num_unblinded_hops - 1).sum()
}
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sadly this is not quite correct, but I tried a different approach to simplify the arithmetic
f7570d6
to
32d576a
Compare
lightning/src/routing/router.rs
Outdated
@@ -400,6 +400,8 @@ pub struct RouteHop { | |||
/// If this is the last hop in [`Path::hops`]: | |||
/// * if we're sending to a [`BlindedPaymentPath`], this is the fee paid for use of the entire | |||
/// blinded path | |||
/// * if we're sending to a Trampoline entrypoint, this is the total incoming amount to the | |||
/// Trampoline hop |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/// Trampoline hop | |
/// Trampoline entrypoint |
lightning/src/routing/router.rs
Outdated
@@ -459,36 +486,58 @@ impl_writeable_tlv_based!(BlindedTail, { | |||
pub struct Path { | |||
/// The list of unblinded hops in this [`Path`]. Must be at least length one. | |||
pub hops: Vec<RouteHop>, | |||
/// The list of unblinded Trampoline hops. When using Trampoline, must contain at least one hop. | |||
/// | |||
/// Note that the first TrampolineHop node must also be present as the last RouteHop node, where |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in general, please tick and link links (also elsewhere):
/// Note that the first TrampolineHop node must also be present as the last RouteHop node, where | |
/// Note that the first [`TrampolineHop`] node must also be present as the last [`RouteHop`] node, where |
lightning/src/routing/router.rs
Outdated
/// The list of unblinded Trampoline hops. When using Trampoline, must contain at least one hop. | ||
/// | ||
/// Note that the first TrampolineHop node must also be present as the last RouteHop node, where | ||
/// the RouteHop's fee_msat is the total outgoing amount sent to all [`TrampolineHop`]s combined, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes no sense, there is no "total outgoing amount sent to all trampoline hops combined", and it seems to imply its referring to the fees received?
/// the RouteHop's fee_msat is the total outgoing amount sent to all [`TrampolineHop`]s combined, | |
/// that RouteHop's fee_msat is the total amount received by the first [`TrampolineHop`], |
lightning/src/routing/router.rs
Outdated
// If we don't have a blinded tail or if we have Trampoline hops, we skip the last RouteHop's fee | ||
unblinded_hop_fee_iterator.skip(1).sum() | ||
} else { | ||
unblinded_hop_fee_iterator.sum() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to skip the last hop here, too, as the last hop in the normal case is the amount we're paying, not a fee.
lightning/src/routing/router.rs
Outdated
.chain(self.trampoline_hops.iter().map(|hop| hop.fee_msat)); | ||
|
||
if self.blinded_tail.is_none() || !self.trampoline_hops.is_empty() { | ||
// If we don't have a blinded tail or if we have Trampoline hops, we skip the last RouteHop's fee |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm confused, the old code didn't do this? The old code included the last hop's fee if it was to a blinded path.
733c53c
to
e5bda32
Compare
@TheBlueMatt how committed are we to only supporting trampoline to blinded paths? I know it's late in the game, but it seems like if we're committed then it might simplify the code to put the trampoline hops in the |
I think fairly committed? I mean I'm definitely open to rethinking that if we want to support clients without a network graph sending payments to a non-blinded destination (ie supporting phoenix/electrum clients sending to bolt 11 destinations), but I dunno if we have a big market for that tbh (phoenix will just use eclair as the trampoline anyway and electrum clients maybe, but we're probably better off convincing them to use RGS) and RGS has worked really well in practice. All that to say I don't have any specific views on putting the trampoline hops in the blinded tail or not. Fine with doing that, fine with not doing that. |
To define the hops in a Path that are meant as Trampoline hops, we need to introduce a new hop type much alike `RouteHop`, except these hops would not be defined by channel ID used to route payments. Instead, the preceding Trampoline hop would determine its own optimal path to route the payment onwards.
e5bda32
to
31d5507
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
lightning/src/routing/router.rs
Outdated
/// Note that the first [`TrampolineHop`] node must also be present as the last [`RouteHop`] node, | ||
/// where the RouteHop's fee_msat is the total amount received by the first [`TrampolineHop`], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait, this says the last RouteHop::fee_msat
is the total amount received by the first trampoline hop, but the RouteHop::fee_msat
docs say that it's the fee paid for all trampoline + blinded hops?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah, nope, gotta be the fee, rephrasing
Ah looks like rustfmt is failing as well. |
Given that we do not intend to allow sending to unblinded recipients via Trampoline hops, we're adding a vector of unblinded `TrampolineHop`s directly to the `BlindedTail` struct.
Trampoline routing relies on serializing an inner onion within the hop data. In this commit, we introduce the method to serialize it based on the augmented path data introduced in the previous commit.
Now that we can generate the inner Trampoline onions, we need to update the `create_payment_onion` utility method to consider the newly available path data and incorporate the encoded Trampoline onion in the outer hop data. This also comes with a behavioral change for blinded tails: when Trampoline are present, we assume that they precede the blinded tail, which must therefore be serialized as part of the Trampoline onion.
31d5507
to
7b56d51
Compare
LGTM, can you take a look @TheBlueMatt? |
trait OnionPayload<'a, 'b> { | ||
type PathHopForId: PathHop + 'b; | ||
type ReceiveType: OnionPayload<'a, 'b>; | ||
fn new_forward( | ||
hop_id: <<Self as OnionPayload<'a, 'b>>::PathHopForId as PathHop>::HopId, | ||
amt_to_forward: u64, outgoing_cltv_value: u32, | ||
) -> Self; | ||
fn new_receive( | ||
recipient_onion: &'a RecipientOnionFields, keysend_preimage: Option<PaymentPreimage>, | ||
sender_intended_htlc_amt_msat: u64, total_msat: u64, cltv_expiry_height: u32, | ||
) -> Result<Self::ReceiveType, APIError>; | ||
fn new_blinded_forward( | ||
encrypted_tlvs: &'a Vec<u8>, intro_node_blinding_point: Option<PublicKey>, | ||
) -> Self; | ||
fn new_blinded_receive( | ||
sender_intended_htlc_amt_msat: u64, total_msat: u64, cltv_expiry_height: u32, | ||
encrypted_tlvs: &'a Vec<u8>, intro_node_blinding_point: Option<PublicKey>, | ||
keysend_preimage: Option<PaymentPreimage>, invoice_request: Option<&'a InvoiceRequest>, | ||
custom_tlvs: &'a Vec<(u64, Vec<u8>)>, | ||
) -> Self; | ||
fn new_trampoline_entry( | ||
total_msat: u64, amt_to_forward: u64, outgoing_cltv_value: u32, | ||
recipient_onion: &'a RecipientOnionFields, packet: msgs::TrampolineOnionPacket, | ||
) -> Result<Self::ReceiveType, APIError>; | ||
} | ||
impl<'a, 'b> OnionPayload<'a, 'b> for msgs::OutboundOnionPayload<'a> { | ||
type PathHopForId = &'b RouteHop; | ||
type ReceiveType = msgs::OutboundOnionPayload<'a>; | ||
fn new_forward(short_channel_id: u64, amt_to_forward: u64, outgoing_cltv_value: u32) -> Self { | ||
Self::Forward { short_channel_id, amt_to_forward, outgoing_cltv_value } | ||
} | ||
fn new_receive( | ||
recipient_onion: &'a RecipientOnionFields, keysend_preimage: Option<PaymentPreimage>, | ||
sender_intended_htlc_amt_msat: u64, total_msat: u64, cltv_expiry_height: u32, | ||
) -> Result<Self::ReceiveType, APIError> { | ||
Ok(Self::Receive { | ||
payment_data: recipient_onion | ||
.payment_secret | ||
.map(|payment_secret| msgs::FinalOnionHopData { payment_secret, total_msat }), | ||
payment_metadata: recipient_onion.payment_metadata.as_ref(), | ||
keysend_preimage, | ||
custom_tlvs: &recipient_onion.custom_tlvs, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few linebreaks in here would be really nice :p
|
||
trait OnionPayload<'a, 'b> { | ||
type PathHopForId: PathHop + 'b; | ||
type ReceiveType: OnionPayload<'a, 'b>; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We definitely don't need a type
for something that's always Self
.
|
||
/// Build a payment onion, returning the first hop msat and cltv values as well. | ||
/// `cur_block_height` should be set to the best known block height + 1. | ||
pub(crate) fn create_payment_onion_internal<T: secp256k1::Signing>( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It'd be really nice if we didn't expose test stuff outside of this module, it makes it really hard to figure out what is or isn't a "public" API. Here and build_onion_payloads
can we expose them as test-only wrappers?
Introduce a method for constructing Trampoline payloads.
This will allow the non-Trampoline method to be modified to generate a Trampoline onion and finish its own regular payload construction.
Builds on top of #3333.