The CONNECT-IP HTTP method for proxying IP traffic

1.1. Tunnel mode

In tunnel mode the client requests tunnel IP packets to and from the proxy. The Connect-IP request establishes such a tunnel and optionally indicates the IP address or IP range that will be allowed to be forwarded.¶

The tunnel mode is indicated by the ":authority" pseudo-header field of the CONNECT-IP request contain the host and listing port of the proxy itself. In this mode the proxy just blindly forwards all payload on its external interface without any modification and also forwards all incoming traffic to registered clients as payload within the respective tunneling association. However, a proxy MUST offer this service only for known clients and clients MUST be authentificated during connection establishment. The proxy SHOULLD inspect the source IP address of the IP packet in the tunnel payload and only forward if the IP address matches a set of registered client IP address. Optionally, a proxy also MAY offer this service only for a limited set of target addresses. In such a case the proxy SHOULD also inspect the destination IP address and reject packets with unknown destination address with an error message.¶

1.2. Flow Forwarding mode

In flow forwarding mode the CONNECT-IP method establishes an outgoing IP flow, from the MASQUE server's external address to the target server's address specified by the client for a particular upper layer protocol. This mode also enables reception and relaying of the reverse IP flow from the target address to the MASQUE server to ensure that return traffic can be received by the client. However, it does not support flow establishment by an external peer. This specification supports forwarding of incoming traffic to one of the clients only if an active mapping has previously been created based on an IP-CONNECT request. Clients that need to support reception of flows established by external peer need to use tunnel mode.¶

This mode covers the point-to-point use case and allows for flow-based optimizations and a larger effective maximum packet size. The target IP address is provided by the client as part of the CONNECT-IP request. The sources address is eiter independently selected by the proxy or can be requested to be either the same as used in a previous and currently active CONNECT-IP request or different from currently requests by the same client. The client also indicates the upper layer protocol, thus defining the three tuple used as primary selector for the flow.¶

In this mode the payload between the client and proxy does not contain the IP header in order to reduce overhead. Any additional information (other than the source and destination IP addresses and ports as well as the upper layer protocol identifier) that is needed to construct the IP header or to inform the client about information from received IP packets can be signalled as part of the CONNECT-IP requst or using HTTP/3 Datagram [I-D.ietf-masque-h3-datagram] later.¶

As such, in flow forwarding mode usually one upper-layer end-to-end connection is associated to one CONNECT-IP forwarding association. While it would be possible for a client to use the same forwadring association for multiple end-to-end connections to the same target server, as long as they all require the same Protocol (IPv4) / Next Header (IPv6) value, this would lead to the use of the same flow ID for all connections. As such this is not recommended for connection-oriented transmissions. In order to enable multiple flow forwadring asssociation to the same server, the flow forwading mode supports a way to specify some additional upper layer protocol selectors, e.g. TCP source and destination port, to enable multiple CONNECT-IP request for the same three tuple, see CONN-ID header Section 3.5.¶

This proposal is based on the analysis provided in [I-D.westerlund-masque-transport-issues] indicating that most information in the IP header is either IP flow related or can or even should be provided by the proxy as the IP communication endpoint without the need for input from the client. The most crucial information identified that requires client interaction is ECN [RFC3168] and ICMP [RFC0792] [RFC4443] handling.¶

This document defines the following IP header field treatment.¶

Required to be determined in Connect-IP request and response:¶

• IP version¶
• IP Source Address¶
• IP Destination Address (target address)¶
• Upper Layer Protocol (IPv4 Protocol field / IPv6 Next Header field)¶

Can be chosen by Proxy on transmission:¶

• IPv6 Flow label (per Connect-IP flow mode request)¶
• IPv4 Time to live / IPv6 Hop Limit (proxy configured)¶
• Diffserv Codepoint per default set to 0 (Best Effort)¶

May optionally be provided on a per packet basis¶

• Explicit Congestion Notification in both directions.¶

The consequence of this is certain limitations that future extension can address. For packets that are sent from the target server to the client, the clieent will not get any information on the actual value of TTL/Hop Count, DSCP, or flow label when received by the proxy. Instead these field are set and consumed by the proxy only.¶

Signalling of other dedicated values may be desired in certain deployments, e.g for DCSP [RFC2474]. However, DSCP is in any case a challenge due to local domain dependency of the used DSCP values and the forwarding behavior and traffic treatment they represent. Future use cases for DSCP, as well as new IPv6 extension headers or destination header options [RFC8200] may require additional signaling. Therefore, it is important that the signaling is extensible.¶

1.3. Definitions

• Proxy: This document uses proxy as synonym for the MASQUE Server or an HTTP proxy, depending on context.¶
• Client: The endpoint initiating a MASQUE tunnel and IP relaying with the proxy.¶
• Target host: A remote endpoint the client wishes to establish bi-directional communication with via tunnelling over the proxy.¶
• IP proxying: A proxy forwarding IP payloads to a target for an IP flow. Data is decapsulate at the proxy and amended by a IP header before forwarding to the target. Packet boundaries need to be preserved or signalled between the client and proxy.¶
• IP flow: A flow of IP packets between two hosts as identified by their IP addresses, and where all the packets share some properties. These properties include source/destination address, protocol / next header field, flow label (IPv6 only), and DSCP per direction.¶

Address = IP address

                     Target Address --+
                                       \
+--------+           +--------+         \ +--------+
|        |  Path #1  |        | Path #2  V|        |
| Client |<--------->|  Proxy |<--------->| Target |
|        |          ^|        |^          |        |
+--------+         / +--------+ \         +--------+
                  /              \
                 /                +-- Proxy's external address
                /
               +-- Proxy's service address

Figure 1 provides an overview figure of the involved nodes, i.e. client, proxy, and target host. There are also two network paths. Path #1 is the client to proxy path, where IP proxying is provided over an HTTP/3 session, usually over QUIC, to tunnel IP flow(s). Path #2 is the path between the proxy and the target.¶

The client will use the proxy's service address to establish a transport connection on which to request IP proxying using HTTP/3 CONNECT-IP. The proxy will then relay the client's IP flows to the target host. The IP header from the proxy to the target carries the proxy's external address as source address and the target's address as destination address.¶

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

2.1. Data encapsualtion

Once the CONNECT-IP method has completed, only CAPSULATE [I-D.ietf-masque-h3-datagram] frames are permitted to be sent on that stream. Extension frames MAY be used if specifically permitted by the definition of the extension. Receipt of any other known frame type MUST be treated as a connection error of type H3_FRAME_UNEXPECTED.¶

Each HTTP Datagram frame contains one of the below specified data formats (Section 2.2) depending on request forwarding mode and given headers and paramters.¶

Stream based forwadring provides in-order and reliable delivery but may introduce Head of Line (HoL) Blocking if independent messages are send over the same CONNECT-IP association. On streams payload data is encapsulated in the CAPSULATE Frame using the DATAGRAM capsule (type=0x02) [I-D.ietf-masque-h3-datagram].¶

The client can, in addition to stream-based forwarding, request use of HTTP/3 datagrams [I-D.ietf-masque-h3-datagram].¶

To request datagram support the client sends H3_DATAGRAM SETTINGS parameter with a value of 1 [I-D.ietf-masque-h3-datagram]. Datagram suppot MUST only be requested when the QUIC datagram extension [I-D.ietf-quic-datagram] was successfully negotiated during the QUIC handshake.¶

Datagrams provide un-ordered and unreliable delivery. In theory both, stream- as well as datagram-based forwarding, can be used in parallel, however, for most transmissions it is expected to only use one.¶

While IP packets sent over streams only have to respect the end-to-end MTU between the client and the target server, packets sent in datagrams are further restricted by the QUIC packet size of the QUIC tunnel and any overhead within the QUIC tunnel packet. The proxy should provide MTU and overhead information to the client. The client MUST take this overhead into account when indicating the MTU to the application.¶

2.2. Datagram Formats

This section defined the different datagram formats used by Connect-IP. Even if only one format is currently used it is expected that for some usages future extension may require the flexibility to use multiple different formats for a given CONNECT-IP request.¶

3.1. IP-Protocol Header for CONNECT-IP

In order to construct the IP header the MASQUE server needs to fill the "Protocol" field in the IPv4 header or "Next header" field in the IPv6 header. As the IP payload is otherwise mostly opaque to the MASQUE forwarding server, this information has to be provided by the client for each CONNECT-IP request for flow forwarding.¶

IP-Protocol is a Item Structured Header [RFC8941]. Its value MUST be an Integer. Its ABNF is:¶

  IP-Protocol = sf-integer

3.2. IP-Version header for CONNECT-IP

IP-Version is a Item Structured Header [RFC8941]. Its value MUST be an Integer and either 4 or 6. This information is used by the proxy to check if the requested IP version is supported by the network that the proxy is connected to, as well as to check the destination or source IP address for compliance.¶

  IP-Version = sf-integer

3.3. IP-Address header for CONNECT-IP

IP-Address is a Item Structured Header [RFC8941]. Its value MUST be an String contain an IP address or IP range of the same IP version as indicated in the IP-Version header. The address must be specified in the format spectified by TBD.¶

This header is used to request the use of a certain IP address or IP address range. If the IP-Address header is not presented, the proxy is implicitly requested to assign an IP address or IP address range and provide this information to the client with the HTTP response.¶

If the the client does not provide an IP address or IP address range is has to wait for the proxy response before any payload data can be sent in tunnel mode. If the request is denied by the proxy as sent payload data will be discarded and a new CONNECT-IP request has to be sent.¶

The header is also used as a response header to indicate the actual IP address or range that will be used.¶

  IP-Address = sf-string

3.4. IP-Address-Handling Header for CONNECT-IP

This header can be used to request the use of a stable address for multiple active flow forwarding associations. The first association will be established with an IP selected by the proxy unless also the IP-Address header (Section 3.3) is provided and accepted by proxy. However, additional forwarding association can be requested by the client to use the same IP address as a previous request by specifying the stream ID as value in this header. This header can also be used to ensure that a "new", not yet for this client used address is selected by setting a value that is larger than the maximum stream ID.¶

IP-Address-Handling is a Item Structured Header [RFC8941]. Its value MUST be an Integer and indicates the stream ID of the corresponding active flow forwarding association. Its ABNF is:¶

  IP-Address-Handling = sf-integer

3.5. Conn-ID Header for CONNECT-IP

This document further defines a new header field to be used with CONNECT-IP "Conn-ID". The Conn-ID HTTP header field indicates the value, offset, and length of a field in the IP payload that can be used by the MASQUE as a connection identifier in addition to the IP address and protocol tuple when multiple connections are proxied to the same target server for incoming traffic on the external address.¶

Conn-ID is a Item Structured Header [RFC8941]. Its value MUST be a Byte Sequence. Its ABNF is:¶

  Conn-ID = sf-binary

The following parameters are defined:¶

• A parameter whose name is "offset", and whose value is an Integer indicating the offset of the identifier field starting from the beginning of a datagram or HTTP frame on the forwarding stream.¶
• A parameter whose name is "length", and whose value is an Integer indicating the length of the identifier field starting from the offset.¶

Both parameters MUST be present and the header MUST be ignored if these parameter are not present.¶

This function can be used to e.g. indicate the source port field in the IP payload when containing a TCP packet.¶

3.6. MTU Header for CONNECT-IP

This document defines a new HTTP request and response header to be used with the CONNECT-IP method. The MTU HTTP header indicates the largest IP packet size the requesting or responding node is capable of forwarding without fragmentation related to the target of the CONNECT-IP request.¶

An HTTP Intermediary that process this header should update the header to a lower value if its effective MTU is less than the value in the MTU header. However, an HTTP intermediary MUST NOT increase the value.¶

MTU is a Item Structured Header [RFC8941]. Its value MUST be an Integer and indicates the number of bytes of the effective IP MTU.¶

  MTU = sf-integer

4.1. Requesting flow forwarding

To request flow forwarding, the client sends a CONNECT-IP request to the forwarding proxy indicating the target host and port in the ":authority" pseudo-header field. The host portion is either an IP literal encapsulated within square brackets, an IPv4 address in dotted-decimal form, or a registered name. Further the CONNECT-IP request MUST contain the IP-Protocol header (Section 3.1) and MAY contain the IP-Address-Handling (Section 3.4) or the Conn-ID (Section 3.5) header.¶

4.2. Requesting tunnel mode

In tunnel mode, the CONNECT-IP request MUST contain the IP-Version header to indicate if IPv4 or IPv6 is used for the IP packet in the tunnel payload. Further, the request MAY contain an IP-Address header to request use of an IP addres or IP address range.¶

5.1. Error handling

TBD (e.g. out of IP address, conn-id collision)¶

5.2. IP address selection in flow forwarding mode

In flow forwarding mode the proxy constructs the IP header when sending the IP payload towards the target server and it selects an source IP address from its pool of IP addresses that are routed to the MASQUE server.¶

If no additional information about a payload field that can be used as an identifier based on Conn-ID header is provided by the client, the masque server uses the source/destination address and protocol ID 3-tuple in order to map an incoming IP packet to an active forwarding connection. The proxy MUST also consider if IP-Address-Handling header Section 3.4 is included and its value. If the IP-Address-Handling header is not included and the there has been prior request the proxy SHOULD give the client the same source Address as the first flow forwarding request. Given these constraints the MASQUE proxy MUST select a source IP address that leads to a unique tuple, and if that is not possible an error response is generated. The same IP address MAY be used for different clients when those client connect to different target servers. However, this also means that potentially multiple IP address are used for the same client when multiple connection to one target server are needed. This can be problematic if the source address is used by the target as an identifier. Therefore it is RECOMMENDED that clients are given unique addresses unless a large fraction of the pool has been exhausted.¶

If the Conn-ID header is provided, the MASQUE server should use that field as an connection identifier together with protocol ID, source and destination address, as a 4-tuple. In this case it is recommended to use a stable IP address for each client, while the same IP address might still be used for multiple clients, if not indicated differently by the client in the configuration file. Note that if the same IP address is used for multiple clients, this can still lead to an identifier collision and the IP-CONNECT request MUST be reject if such a collision is detect.¶

Note: Are we allowing multiple client's to share the same 3-tuple when using Conn-ID? It might be good for privacy reasons however, it significantly increases the collision risk.¶

5.3. Constructing the IP header in flow forwarding mode

To retrieve the source and destination address the proxy looks up the mapping for the datagram flow ID or stream identifier. The IP version, flow label, DiffServ codepoint (DSCP), and hop limit/TTL is selected by the proxy. The IPv4 Protocol or IPv6 Next Header field is set based on the information provided by the IP-Protocol header in the CONNECT-IP request.¶

MASQUE server MUST set the Don't Fragment (DF) flag in the IPv4 header. Payload that does not fit into one IP packet MUST be dropped. A dropping indication should be provided to the client. Further the MASQUE server should provide MTU information.¶

The ECN field is by default set to non-ECN capable transport (non-ECT). Further ECN handling is described in Section Section 6.1.¶

5.4. Decapsulating tunnel mode IP Packets

On receiving an HTTP Datagram containing any of the tunnel mode formats for IPv4 or IPv6 the proxy extracts the full IP packet.¶

The proxy MUST verify that the extracted IP packet's source IP address matches any address associated with this CONNECTION-IP request, i.e. the assgined address or IP range. This is to prevent source address spoofing in tunnel mode.¶

Further the proxy should verify that the IP header length field correspond to the extracted packets length.¶

5.5. Receiving an IP packet

When the MASQUE proxy receives an incoming IP packet on the external interface(s), it checks the packet selectors to find the mappings that match the given packet.¶

If a client has a tunnel as well as multiple flow forwarding associations, the proxy need to check the mappings for the flow forwarding associations first, and only send it over the the tunnel association if no active flow forwarding is found.¶

If one or more mappings exists, it further checks if this mapping contains additional identifier information as provided by the Conn-ID Header of the CONNECT-IP request. If this field maps as well, the IP payload is forwarded to the client. If no active mapping is found, the IP packet is discarded.¶

The above is achieve by using the selector with the most number of fields that match the packet.¶

If both datagram and stream based forwarding is supported, it is recommended for the proxy to use the same encapsulation as most recently used by the client or datagrams as default. Further considerations might be needed heree.¶

6.1. ECN

ECN requires coordination with the end-to-end communication points as it should only be used if the endpoints are also capable and willing to signal congestion notifications to the other end and react accordingly if a congestion notification is received.¶

Therefore, if ECN is used, the proxy needs to inform the client of a congestion notification (IP CE codepoint) was observed in any IP header of a received packet from the target server.¶

Possible realizations are: a) always have two bits before payload in flow forwarding model or use 4 different context IDs. This is work in process and will be further specified in a future version of this document.¶

6.2. ICMP handling

ICMP messages are directly forwarded in tunneling mode. In flow forwarding mode Context IDs can be used to forward if requested.¶

6.3. MTU considerations

For the MTU Discussion we need to update the figure for the involved nodes as given in {fig-node-model}. Instead we will use the below figure Figure 2.¶

+--------+        +--------+        +-------+        +--------+
|        | Path#1 | HTTP   | Path#2 |       | Path#3 |        |
| Client |<------>| Inter- |<------>| Proxy |<------>| Target |
|        |        | mediary|        |       |        |        |
+--------+        +--------+        +-------+        +--------+

In this figure the HTTP request and response are done between the Client and Proxy. However, the HTTP/3 connection from the client is terminated in an HTTP Intermediary, that forwards the request to the proxy over a second HTTP connection.¶

For CONNECT-IP there are several aspects that decides the effective end-to-end MTU for IP packets going from the client to the target. These aspects are:¶

• The supported QUIC datagram size between Client and the HTTP Intermediary on Path#1. This is limited by the actual IP MTU on the IP path between these nodes as well as the configuration of the QUIC endpoints. The IP MTU is first limited by the respective nodes Interfaces, which is known by each node. Next by the link layers and routers on Path#1 that may restrict the MTU, this value is normally now well known, and need to be determined. Finally, the QUIC endpoints will have their internal limits for how big IP/UDP/QUIC packets they will send, and these limits are impacted by implementation, as well as the QUIC transport parameter "max_udp_payload_size".¶
• Next is the question what datagram size will be supported between the HTTP Intermediary and the Proxy on Path#2. This has the same set of limiations as for Path#1 in case QUIC is used between them. However, it is not guaranteed that HTTP/3 is used on this path, thus other limiations may be in place. However, if HTTP 1.x or HTTP/2 are used then a reliable byte stream oriented transport will be used and thus these does not provide any stric limit on the supporte HTTP datagram sizes that can be passed.¶
• Next is the IP MTU for the Proxy's external interface that will be used to send any IP packet to the Target. This value is known by the proxy.¶
• Finally is the IP MTU of Path#3 and the receiving interface at Target. Once more a value that is not well know, and instead learned through ICMP Packet Too Big or other PMTUD methods.¶

The IP MTU limits of the two last are hard limits and will impact the transmitted end-to-end IP packets. Either being forced to be fragmented (IPv4 without Don't framgent bit set), or dropped (IPv6 or IPv4 with DF set).¶

One possible stand point here could be that client and targets end-to-end communication would be required to rely on Path MTU Discovery (PMTUD) without any support. However, I think it is worth considering the issues and impact of not provide support that can speed up convergence on end-to-end IP MTU.¶

The two first paths MTU are primarily performance impacting as one like to ensure that the IP packets or IP payloads (flow forwarding) fits within one HTTP Datagram that in its turn fits in a QUIC Datagram where available. If they don't fit the HTTP node will have to forward the HTTP Datagram in a CAPSULE over a relaible stream. This last is a necessary functionality to ensure that HTTP/3 over an path that only supports minimal IPv6 Size can tunnel or forward end-to-end IPv6 packets of the minimal 1280 size.¶

With the possibility of multipe HTTP intermediaries between client and proxy it is necessary to have an HTTP level mechanism to signal at least an intially known MTU for the HTTP Datagram path between client and proxy. This enables the proxy to also to take its external IP interfaces MTU, or any cached MTU restirction on the path to the target for a specific CONNECT-IP request. For this purpose a new HTTP header, named MTU is defined (Section 3.6.¶

As a complement to the HTTP MTU header, a client could support receiving ICMP messages, like Packet Too Big (PTB). This would enable the client to adjust its end-to-end IP MTU in cases where PTB messages are received by proxy for packets to the target on the proxy to target path (Path#3).¶

This leaves one set of remaining issue not resolved. If the IP MTU changes on Path#1 and Path#2 this could force an HTTP node to be forced to send the HTTP datagrams using CAPSULE over reliable stream to use the fragmentation on the byte stream. This is not a problem necessary to solve, however due to the use of reliable streams the endpoint will not have any clear signal that there is an MTU problem on these paths. Instead it will only manifest itself as a performance impact.¶

QUESTION: Maybe HTTP Datagrams should define a MTU signalling CAPSULE which would enable the HTTP Intermediaries to also initiate this message in relation to a stream.¶

QUESTION 2: There exist some possibility to avoid requiring the HTTP MTU header if an HTTP Datagram MTU Capsule would be defined that HTTP intermediaries may initiate and modify.¶

9.1. HTTP Method

This document (if published as RFC) registers "CONNECT-IP" in the HTTP Method Registry maintained at <https://www.iana.org/assignments/http-methods>.¶

  +--------------+------+------------+---------------+
  | Method Name  | Safe | Idempotent |   Reference   |
  +--------------+------+------------+---------------+
  | CONNECT-IP   |  no  |     no     | This document |
  +--------------+------+------------+---------------+

9.2. HTTP Header

This document (if published as RFC) registers the following header in the "Permanent Message Header Field Names" registry maintained at <https://www.iana.org/assignments/message-headers>.¶

  +---------------------+----------+--------+---------------+
  | Header Field Name   | Protocol | Status |   Reference   |
  +---------------------+----------+--------+---------------+
  | Conn-ID             |   http   |  exp   | This document |
  +---------------------+----------+--------+---------------+
  | IP-Protocol         |   http   |  exp   | This document |
  +---------------------+----------+--------+---------------+
  | IP-Address          |   http   |  exp   | This document |
  +---------------------+----------+--------+---------------+
  | IP-Address-Handling |   http   |  exp   | This document |
  +---------------------+----------+--------+---------------+
  | IP-Verison          |   http   |  exp   | This document |
  +---------------------+----------+--------+---------------+
  | MTU                 |   http   |  exp   | This document |
  +---------------------+----------+--------+---------------+