14

Something I don't understand:

(Tens/hundreds of?) thousands of people simultaneously try to connect to a site like facebook.com or google.com.

From what I understand, they must all necessarily connect to the same initial server (because DNS will return the same IP to many of them, and so all the requests go to the same target destination).

So a single machine/router must handle all the initial requests, even if it plans to forward them to other machines.

How come that single device doesn't get overloaded when this happens?

Dennis
  • 48,917
  • 12
  • 130
  • 149
user541686
  • 23,663
  • 46
  • 140
  • 214

1 Answers1

20

Your understanding that they all connect to the same server is wrong, although the details of how you achieve those results are complex. http://highscalability.com/ has reference work on how some of the scalability solutions are put into play.

They have well more than just "one" server that clients connect to, even if the public IP address looks the same. Google, for example, make heavy use of anycast addressing to direct people, and usually they don't just have one IP address for each client - even if they return just one address when you ask.

Daniel Pittman
  • 3,688
  • 17
  • 15
  • +1 thanks for pointing out the error. I can't help but wonder, though: if subsequent requests go to a different server every time, then how does a server continue a different server's session? Or is the randomness on a per-machine/per-session basis? (I would imagine that they all synchronize at the backend, but it would seem very slow to synchronize thousands of servers holding information about millions of users simultaneously.) – user541686 Feb 03 '12 at 20:55
  • 1
    The answer is complicated, and depends on the implementation, but one approach is to have a pool of machines that do nothing but send the packets to the right destination without ever actually making a TCP connection. Look to F5, and other load balancer vendors, for the small end of how. Google, I think, use something they built themselves. – Daniel Pittman Feb 03 '12 at 21:00
  • You can also use a split-session methodology. There is a session between the user and the server they're directly connected to, and a "master logical session" between the user and the logical service. If the user moves to a different server, that server just resumes the same master logical session to the logical service. – David Schwartz Feb 03 '12 at 23:02