0

I stumbled upon a YT video- 5 mins which explains that there's technically no limit to the number of connections that can be opened between the clients and servers even if there's a reverse proxy between them. I see that at max 64k connections can be opened b/w the reverse proxy and one of the backend server(assuming that the backend server listens on a single port for now).

The video mentions that 64k is the limit for L4 proxy and that's why L4 proxy is not scalable. What does "not-scalable" mean here? It also mentions that we cannot reuse the same TCP connection for anything else, why so? After the inception of HTTP/2.0 we can use the same TCP connection to send all kinds of HTTP requests so why can't the L4 proxy just do the same? Why can't it forward all the traffic that it has to send to a given backend server through one single TCP connection, it just has to replace the older TCP and IP header with the new header(with new sourceIP as itself and new destinationIP as one of the backend, similarly replace the ports as well). It doesn't have to seek into the HTTP frames for anything at all. Let me know if I'm incorrect in my understanding. Also, why is L4 connection to the backend called as "stateful"?

AFAIK, a L7 proxy just gets to the HTTP frames and extracts the request body and finds which exact backend to forward request to. What extra intelligence does it have to use the same TCP connection to forward all kinds of requests that an L4 proxy doesn't have?

The only difference that I see b/w L4 and L7 is that L4 doesn't get into the detail of what is inside the HTTP request while L7 is aware of that.

asn
  • 121
  • 4
  • you are mistaken that TCP and HTTP are at all similar. L7 ALGs only know the application they are designed to work with, and leave everything else up to the OS. HTTP ALGs literally only speak HTTP, so they track "Sessions" made of requests/responses rather than "connections" the way L4 does (though the HTTP verb to start a session is `connect` it really is an entirely different kind of thing). by stacking HTTP sessions on top of TCP connections, the different parts of the system all have all the info they need to perform their particular function. – Frank Thomas May 31 '23 at 22:38

2 Answers2

2

What does "not-scalable" mean here?

Has a limited number of TCP connections to the backend which can not be exceeded.

... we cannot reuse the same TCP connection for anything else, why so?

There is a 1:1 relation between active connections from clients to the L4 proxy and connections from the proxy to the backend. Data are forwarded on these connections w/o further interpretation. A connection from proxy to backend gets closed at the same time the connection between client and proxy gets closed. One use the same source port after that (but only with some delay) but this is not a connection reuse - it is a new connection with the same port,ip tuple (but probably different sequence numbers).

After the inception of HTTP/2.0 we can use the same TCP connection to send all kinds of HTTP requests so why can't the L4 proxy just do the same?

Because L4 means transport layer only (TCP, UDP) and there is no concept of HTTP, requests etc at this level. The only way is to forward the data as their are. Note that L4 proxies can not only be used for HTTP but also for SSH etc, i.e. protocols which have not even the concept of a request like HTTP does.

it just has to replace the older TCP and IP header with the new header

An L4 proxy does not tunnel the original IP and port but replaces it with a new IP and port from the system where the proxy is running on. It cannot just replace source IP and port in a running TCP connection - that's not how TCP works.

why is L4 connection to the backend called as "stateful"?

Because it keeps an association between the external connection (client to proxy) and the internal connection (proxy to backend) - that's the state.

The only difference that I see b/w L4 and L7 is that L4 doesn't get into the detail of what is inside the HTTP request while L7 is aware of that.

An L7 proxy works at the request level and can use the same TCP connection for requests coming in from different TCP connections on the client side. An L4 proxy cannot do this.

Steffen Ullrich
  • 5,642
  • 17
  • 22
  • Regarding the NAT translation that happens on the proxy - All the clients hit the same port of the reverse proxy so how does the reverse proxy know which client it needs to forward the response that it got back from the backend? Is this differently handled in L4 v/s L7 proxy? – asn May 31 '23 at 20:01
  • *" All the clients hit the same port of the reverse proxy"* - but these are different connections. While they have the same destination IP and port they have different source IP and ports. – Steffen Ullrich May 31 '23 at 20:09
  • Yes, they are different connections. I'm asking if {proxyIP, proxyPort} is same for all the connections so how does reverse proxy map the incoming response from backend to a particular client(clientIP, clientPort) because the response from backend only has destinationIP as reverse proxy's IP? – asn May 31 '23 at 20:20
  • @ajaysinghnegi: that's what the state in "stateful" is about: having an association between the connection from client to proxy to the connection from proxy to server. Assuming that proxy ip and port and backend ip and port are the same, this is then (client_source_ip, client_source_port) of the connection from the client mapped to (proxy_source_ip, proxy_source_port) of the connection to the backend. – Steffen Ullrich May 31 '23 at 20:34
  • When backend-server returns response, response's TCP header contains (destination_ip: proxyIP, destination_port: proxyPort) and (sourceIP: backendIP, source_port: backendPort), so nowhere does the TCP header of the response has information regarding which source_client to forward the request to. Bcz. multiple clients with different sourceIPs will have the same destinationIP for the proxy so there a many to 1 relationship b/w the client and proxy so how does the proxy know out of many such client which one to forward the response? – asn May 31 '23 at 21:03
  • The proxy itself handles the mapping of external connection to backend connection. it remembers the TCP connection to it from the client, and which connection it created to the backend. as long as it remembers which connection goes with which, it has all the info it needs. – Frank Thomas May 31 '23 at 22:26
  • @FrankThomas I was just thinking about what sort of info is kept maintain this mapping? Is this all part of NAT? – asn May 31 '23 at 23:24
  • No NAT and proxies are very different technologies. NAT/PAT occurs at layers 3 and 4. transport proxies occur at layers 4 & 5 (socks) and application proxies (ALGs) occur at layer 7. The minimal info required for a proxy mapping would basically be internal identifies for the inbound (client -> proxy) and outbound connections (proxy -> backend), as well as some state info like whether the virtual circuit the proxy creates is "active" or "idle". stuff like that. you can look up socks proxies for the details on them. – Frank Thomas Jun 01 '23 at 00:04
  • @FrankThomas: You are right that NAT and proxy are different technologies - NAT keeping the original TCP connection and just rewriting the IP and port inside it, while proxy having distinct incoming and outgoing TCP connections. From the perspective of the backend it looks basically the same though, it has the same limitations regarding source port and the state information which are held (association between incoming and outgoing connection) are basically the same too, only implement in a different way. – Steffen Ullrich Jun 01 '23 at 04:26
  • So I'm not really sure where you;re trying to go with this.... @SteffenUllrich has done a reasonably nice job of responding to your statements/questions. what more do you want specifically to mark this as Answered? Please keep in mind that SE sites are not designed as fourms where you can seek understanding through conversation, but instead as question answer sites where every good question has exactly one good answer. while this is often perhaps an overly lofty goal, please consider it when scoping your question for this community. – Frank Thomas Jun 01 '23 at 07:26
  • @FrankThomas `The minimal info required for a proxy mapping would basically be internal identifies for the inbound (client -> proxy) and outbound connections (proxy -> backend)` - It is quite possible that 2 incoming TCP connections get to share the "same" outbound connection. During the response, how does the the "same" outbound connection know which incoming connection to give back the response to? Feel free to answer, I can ask this in a new question :) – asn Jun 03 '23 at 21:13
  • @SteffenUllrich `An L7 proxy works at the request level and can use the same TCP connection for requests coming in from different TCP connections on the client side. An L4 proxy cannot do this.` What makes the L4 proxy not able to use 1 single TCP connection(b/w itself and backend server) to pass different requests from multiple clients? – asn Jun 03 '23 at 21:37
  • @SteffenUllrich `Because it keeps an association between the external connection (client to proxy) and the internal connection (proxy to backend) - that's the state.`, if L7 uses multiple internal connections to backend server then does that make it stateful too? You'll have to maintain similar state right? – asn Jun 03 '23 at 21:39
  • *" It is quite possible that 2 incoming TCP connections get to share the "same" outbound connection."* - only for L7, which keeps states at the level of client connection + request and thus can map the response to a request back to the client connection. *" What makes the L4 proxy not able to use 1 single TCP connection(b/w itself and backend server) to pass different requests from multiple clients?"* - the proxy would have to understand the application traffic (requests, responses) for this, i.e. deal with OSI layer 7. This would thus be an L7 proxy, not an L4 proxy. – Steffen Ullrich Jun 04 '23 at 05:27
2

The root of your misunderstanding is perhaps understanding the nature of a connection, which is done using an ephemeral port.

The upstream server can handle only 64k connections from the same client because 64K is the limitation of the ephemeral port range at the client side.

Here is an example how you may assign more than 64K connections :

But you can assign several IP addresses to the same private interface of your load balancer and force server to use them in a round-robin fashion.

You can define several networks on the same interface of load balancer, for example:

  • 192.168.1.1,
  • 192.168.2.1,
  • 192.168.3.1

And define corresponding extra IP addresses at upstream server:

  • 192.168.1.2,
  • 192.168.2.2,
  • 192.168.3.2 .

With following upstream configuration load balancer will pass requests to the same upstream server while using different IP addresses:

upstream ipproxy {
  server 192.168.1.2:some-port;
  server 192.168.2.2:some-port;
  server 192.168.3.2:some-port;
}

Load balancer will be forced to use different IP addresses thus allowing you to bypass 64k connection limitation and achieve 192k connections.

harrymc
  • 455,459
  • 31
  • 526
  • 924
  • `The upstream server can handle only 64k connections from the same client because 64K is the limitation of the ephemeral port range at the client side.` - So, this 64k connections limitations should be shared by both the L4 and L7 balancers, right? – asn Jun 03 '23 at 21:15
  • Yes, this is a network limitation. – harrymc Jun 04 '23 at 09:07