[Peers] Feature: Implement optional message dropping in Message send path

[julianknutsen]

As a Transport layer, I want to know which messages are optional and which messages are mandatory so I can make the right policy decision when my queue is full or the SocketDescriptor fails a full write.

Definition of Done:

• Optimal messages identified
• Policy decision made on if/when items should be dropped
• Unit tests updated to drop optional messages during flow control

Tech Discussion:
TODO from the codebase:

// TODO: There are some DoS attacks here where you can flood someone's outbound send
// buffer by doing things like announcing channels on another node. We should be willing to
// drop optional-ish messages when send buffers get full!

The current OutboundQueue allows the message path to push messages over the soft limit (the sync messages are the only things that care about the soft limit).

Seems like an update to the Transport API makes the most sense. Potentially adding a priority number or classification to each enqueue_message() where the Transport layer can make the policy decision to drop it if necessary.

The Transport unit tests should be easily updated to test this behavior.

[TheBlueMatt]

the sync messages are the only things that care about the soft limit

I think those may be the only messages that are optional. Without looking at the code, I'm not sure what the concrete issue here is.