BIP 434

Abstract

This BIP defines a peer-to-peer (P2P) message that can be used for announcements and negotiation related to support of new peer-to-peer features.

Motivation

Historically, new peer-to-peer protocol changes have been tied to bumping the protocol version, so that nodes know to only attempt feature negotiation with peers that support the feature. Coordinating the protocol version across implementations, when different clients may have different priorities for features to implement, is an unnecessary burden in the upgrade process for P2P features that do not require universal support. And at a more philosophical level, having the P2P protocol be permissionlessly extensible, with no coordination required between implementations or developers, seems ideal for a decentralized system.

Many earlier P2P protocol upgrades were implemented as new messages sent after a peer connection is set up (ie, after receipt of a verack message by both sides). See BIP 130 (sendheaders), BIP 133 (feefilter), and BIP 152 (compact blocks) for some examples. However, for some P2P upgrades, it is helpful to perform feature negotiation prior to a connection being fully established (ie, prior to the verack being received by both sides). BIP 155 (addrv2) and BIP 339 (wtxid-relay) are examples of this approach, which involves sending and receiving a single new message (sendaddrv2 and wtxidrelay respectively), in between version and verack to indicate support of the new feature.

In all these cases, sending new messages on the network raises the question of what non-implementing software will do with such messages. The common behavior observed on the network was for software to ignore unknown messages received from a peer, so these proposals posed minimal risk of potential network partitioning. In fact, supporting protocol extensibility in this manner was given as an explicit reason to ignore unknown messages in Bitcoin’s first release.

However, if nodes respond to unknown messages by disconnecting, then the network might partition in the future as incompatible software is deployed. And in fact, some clients on the network have historically discouraged or disallowed unknown messages, both between version and verack (eg, Bitcoin Core discouraged such messages between PR#9720 and PR#19723, and btcd disallowed such messages until PR#1812, but see also discussion in #1661), as well as after verack.

To maximise compatibility with such clients, most of these BIPs require that peers bump the protocol version:

• BIP 130 requires version 70012 or higher,
• BIP 133 requires version 70013 or higher,
• BIP 152 recommends version 70014/70015 or higher, and
• BIP 339 requires version 70016 or higher.

And while BIP 155 does not specify a minimum protocol version, implementations have added a de facto requirement of version 70016 or higher.

In this BIP, we propose codifying and generalising the mechanism used by BIP 339 for future P2P upgrades, by adding a single new feature negotiation message that can be reused for advertising arbitrary new features, and requiring that implementing software ignore unknown features that might be advertised. This allows future upgrades to negotiate new features by exchanging messages prior to exchanging verack messages, without concerns of being unnecessarily disconnected by a peer which doesn’t understand the messages, and without needing to coordinate updating the protocol version.

Specification

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

For the purposes of this section, CompactSize refers to the variable-length integer encoding used across the existing P2P protocol to encode array lengths, among other things, in 1, 3, 5 or 9 bytes. Only CompactSize encodings which are minimally-encoded (ie the shortest length possible) are used by this specification.

Nodes implementing this BIP:

• MUST advertise a protocol version number >= 70017,
• MUST NOT send feature messages to peers that advertise a protocol version number < 70017,
• MUST accept feature messages received after the version message and before the verack message, and
• MUST NOT send feature messages after sending the verack message.

In addition, nodes implementing this BIP:

• SHOULD ignore unknown messages received after the version message and before the verack message,
• MAY ignore feature messages sent after verack, and
• MAY disconnect peers who send feature messages after verack.

Feature specifications based on this BIP:

• MUST forbid sending messages it introduces after verack to a peer that has not indicated support for the feature via a feature message.

feature message

The payload of the feature message contains exactly the following data:

Type	Name	Description
string	featureid	Unique identifier for the feature
byte-vector	featuredata	Feature-specific configuration data

The featureid is encoded in the usual way, that is, as a CompactSize specifying the string length, followed by that many bytes. The string length MUST be between 4¹ and 80², inclusive. The string SHOULD include only printable ASCII characters (ie, each byte should have a value between 32 and 126, inclusive).

Likewise, featuredata is encoded as a CompactSize specifying the byte-vector size, followed by that many bytes. How these bytes are interpreted is part of the feature’s specification. The byte-vector size MUST NOT be more than 512 bytes³. Note that the featuredata field is not optional, so if no data is required, an empty vector should be provided, ie serialized as CompactSize of 0.

The featureid MUST be a globally unique identifier for the feature.

For features published as a BIP, the featureid SHOULD be the assigned BIP number, eg “BIP434”, or be based on the BIP number (eg, “BIP434v2” where the “v2” suffix covers versioning, or “BIP434.3” where the “.3” suffix covers part 3 of the BIP). For experimental features that do not (yet) have a BIP number assigned, some other unique identifier MUST be chosen, such as a URL to the repository where development is taking place, or the sha256 digest of some longer reference.

Nodes implementing this BIP MUST ignore feature messages specifying a featureid they do not support, so long as the payload conforms to the requirements above.

Nodes implementing this BIP MAY disconnect peers that send feature messages where the feature message’s payload cannot be correctly parsed (including having missing or additional data), even if they do not recognise the featureid.

feature message 1-byte identifier

Nodes implementing both this BIP and BIP 324 (v2 P2P encrypted transport) MUST treat a message with a 1-byte message_type equal to 37 that is received prior to verack as the feature message.

Feature negotiation

It is RECOMMENDED that feature negotiation be designed and implemented as follows:

• all feature messages and the verack message should be sent immediately on receipt of the peer’s version message
• any negotiation calculations should be performed immediately on receipt of the peer’s verack message

This structure is fairly easy to implement, and avoids introducing any significant latency that might result from more interactive negotiation methods.

Feature specifications defining a featureid MAY make use of the following approaches:

Feature advertisement:

1. Send a feature message advertising the featureid unconditionally
2. Accept messages related to the feature unconditionally
3. Only send messages defined by the feature if the peer sent a valid feature message for the featureid.

This approach is appropriate for many simple features that define new messages, particularly where an implementation might only implement sending or receiving a message, but not both, eg BIP 35 (mempool).

Feature coordination:

1. Send a feature message advertising the featureid unconditionally
2. Check if the peer sends the same feature message (or a compatible one), and enable the feature for this peer if so.
3. Only send/accept messages or encode data items according to the feature’s specification if the feature is enabled for this peer.

This approach is appropriate for upgrades to data encoding in P2P messages, eg BIP 339 (wtxidrelay) or BIP 155 (addrv2).

Feature versioning:

1. Send feature messages for multiple incompatible features, eg BIP434v3, BIP434v2, BIP434v1, ordered from most preferred to least.
2. Track the corresponding feature messages from your peer.
3. If you were the listening peer, enable your highest preference feature that your peer also supports.
4. If you were the initiating peer, enable the first feature that your peer announced, that you also support.
5. For example if the listening peer sends BIP434v3, BIP434v2, BIP434v1, and the initiating peer sends BIP434v1, BIP434v2, then the listening peer should select BIP434v2 when verack is received, and the initiating peer should select BIP434v2 as soon as feature BIP434v2 is received.
6. Conversely, if the initiating peer sends BIP434v3, BIP434v2, BIP434v1, and the listening peer sends BIP434v1, BIP434v2, then the listening peer should select BIP434v1 when verack is received, and the initiating peer should select BIP434v1 as soon as feature BIP434v1 is received.
7. In most cases, implementations should simply advertise incompatible features in order from most recent to oldest, on the basis that the only reason to make incompatible updates is because there are significant improvements. Exceptions to that may occur when two incompatible features are both receiving active development, or when an implementation has only partially implemented the latest spec, and the older spec is better supported (and thus should be listed first, as the preferred protocol to adopt).

This approach may be appropriate when making substantial changes to a deployed protocol and backwards compatibility is desirable on a short-term basis, or when there is disagreement amongst implementations or users as to which approach is most desirable.

Considerations

The advantage this approach has over bumping the protocol version number when introducing new P2P messages or data structures, is that no coordination is required (that is, there is no longer a question whether version “n+1” belongs to Alice’s new feature, or Bob’s new feature), and there is no implication that supporting each new feature means all prior features are also supported.

The advantage this approach has over defining new messages for each feature is that the featureid can be much longer (at up to 80 bytes) than a message type id (which are limited to 12 bytes). With a BIP 324 one-byte message_type, the overhead compared to that approach is also kept small.

This approach is largely equivalent to adding a payload to the verack message (eg, a vector of featureid, featuredata pairs). It was chosen because:

• it retains compatibility with any implementations that expect verack to have no payload;
• it allows peers to process each feature request individually, rather than having to first load the configuration information for all features into memory at once (in order to validate the message’s checksum), and then deal with each feature’s configuration;
• limiting the maximum message payload size you accept (eg to 4MB) does not limit the number of features you can accept; and
• we have experience with negotiating features with individual messages, but no experience with doing so via verack payload.

A mild disadvantage compared to using a verack payload is that this approach allows the possibility of interactive feature negotiation prior to verack, which would make negotiation more complex and increase latency. However interactive feature negotiation is always possible simply by having the initiating peer disconnect and reconnect after discovering the listening peer’s supported features.

This specification attempts to maximise compatibility with implementations that prefer to fully validate each message received:

• feature messages, even for unknown features, must always be fully parseable into a featureid and featuredata
• Ignoring unknown messages prior to verack is only a recommendation, not a requirement, so compliant implementations may disconnect on an unknown message that cannot be validated.
• Sending unknown messages after verack is explicitly forbidden, in so far as that is possible.

Reference implementation

This feature was implemented in Bitcoin Core PR#35221, which may be used as a reference implementation.

Backward compatibility

Clients specifying a version number prior to 70017 remain fully compatible with this change.

Clients specifying a version number of 70017 or higher that do not implement this BIP remain fully compatible provided they do not disconnect peers upon receiving unexpected messages received between version and verack.

Acknowledgements

Much of the logic here, and much of the text in the motivation section, is based on Suhas Daftuar’s 2020 post on Generalizing feature negotiation.

Copyright

This BIP is licensed under the 2-clause BSD license.

---

1. The minimum featureid length of 4 characters corresponds with a “BIPx” string for BIPs in the range 1-9. Longer identifiers than this will almost always be needed in order to meet the “globally unique” requirement.↩︎

2. The maximum values for the featureid and featuredata lengths serve only to bound the volume of the feature message payload, particularly as this data will be automatically advertised even to peers that do not support the features in question.↩︎

3. The maximum values for the featureid and featuredata lengths serve only to bound the volume of the feature message payload, particularly as this data will be automatically advertised even to peers that do not support the features in question.↩︎