{"version":1,"lectureId":"01M1A1VXTSGXTXG2QW3QGQGPZJ","attempt":0,"publication":{"slug":"understanding-the-internet-from-first-principles-with-curl","title":"Understanding the Internet from First Principles with cURL","subject":"computer-science","summary":"A first look at how the internet actually works, built from the bottom up and made visible with one command line tool. We begin with the only two things involved in any web request, a client and a server, and then follow the four steps that happen every single time: a name is turned into an IP address by the domain name system, a TCP connection is opened by a three way handshake and then encrypted for https, an HTTP request is written out as a method, a path, headers and a body, and a reply comes back carrying a status code. Every one of those layers is then shown on screen with curl, whose verbose output labels each line by who said it, so the invisible parts of the network turn into plain text you can read for yourself. No programming or networking background is assumed.","metaDescription":"Clients, servers, IP addresses, DNS, TCP and HTTP explained from scratch, then made visible on screen with a few curl commands.","transcript":"You have done this thousands of times. You type an address, something happens for about a second, and a page appears. Almost none of that is visible, and almost all of it is simple. Underneath there are only two machines and two messages. One machine wants something. That is this one, and we call it the client. The other machine has the thing and is waiting to be asked for it. That is the server: a computer in a building somewhere, switched on all the time, doing nothing else. The client sends a request. One short piece of text saying what it wants, and a few facts about itself. The server sends back a reply. A short piece of text saying how it went, and then the thing itself. That is the whole shape of it, and everything else is detail about those two arrows. So let us do exactly that, using a program called curl. It plays the part of the client, once, and prints whatever comes back. Here is the whole command. The first line that came back is the status line. It says which version of the language the server spoke, and then a number. Two hundred means it worked. Then a few lines about the reply itself. This one says the thing coming back is a web page. Then one blank line, and then the page, which is what a browser would draw for you. Here it simply gets printed. Four separate things had to happen before any of that text existed, and they happen in the same order every single time. First the name in the address had to become a number. Then a connection had to be opened between the two machines. Then the request had to be written out in a very particular form. And then the reply came back, carrying a code. None of those four is mysterious, and by the end you will have watched each one happen on your own screen. Let us begin with the number. The name you typed is for you. It is a convenience for humans, and the machines in between do not use it at all. What they use is a number. Every machine that can be reached from the internet has one of these addresses. Here are four of them: four machines, somewhere in the world. One of them is the one we are after, the machine holding the page we asked for. It is this one, in green. Written down, an address is four small numbers with dots between them. Each one runs from zero up to two hundred and fifty five, which is as much as eight bits can hold. That number is the destination, and it is the only part of the address the network in between cares about. The name itself never travels anywhere. Which leaves us a problem. You typed a name. The wire wants a number. Something has to do the translation, and it has to happen before anything else. That something is called the domain name system. Picture it as one enormous table with two columns: names on the left, addresses on the right. No single machine holds that table. It is cut into pieces, spread over thousands of machines, and each piece is kept by whoever owns those particular names. So your computer asks a machine nearby, called a resolver. If the resolver does not know the answer, it asks the machines that do, and passes back what it gets. And the answer is remembered for a while, both on your machine and on the resolver. That is why the first visit to a site is a little slower than the second. You do not have to take any of this on faith. Add one flag to the command, dash v for verbose, and the program narrates its own work as it goes. The first thing it says is the number it looked up, and the door it is knocking on at that address. Four four three is called a port, and it is the one kept for encrypted web traffic. Then it says connected. From that moment the name is finished with, and everything that follows happens between two numbers. We know where to send things now. That still is not a conversation, because the network in between does not carry conversations. It carries small pieces, one at a time, and it promises nothing about them. Send four pieces in this order. Each one is passed from machine to machine along whatever path happens to be free, so they arrive in whatever order they manage. Here the third piece overtook the second. And sometimes a piece never arrives at all, and nothing along the way even notices. So something has to number the pieces, keep track of which ones landed, ask again for the missing ones, and hand the whole thing over in order. That something is TCP, and it is the reason you have never had to think about any of this. Setting one of those connections up takes three short messages, and nothing useful is said in any of them. Time runs downward here. The client sends the first one. It means: I would like to talk to you. It is called SYN, short for synchronise. The server sends the second, which means: I heard you, and I would like to talk to you as well. And the client sends the third: I heard that. Now both ends know the other one can hear them, and both know where to start counting. That is the three way handshake. It costs one full round trip before a single useful byte moves. That is why a server on the far side of the world feels slow even when it is not busy at all. One more thing happens when the address begins with https. Before any useful words are exchanged, the two ends agree on how to encrypt everything that follows. That agreement is a short exchange of its own, and it gets reported when it finishes. From here on, anybody watching the wire in between sees only noise. The server also hands over a certificate: a document signed by an organisation your machine already trusts, saying that this really is the machine for that name. It gets checked, and here it passed. If that check fails, the program stops and tells you, and your request is never sent. So now we have an address, and an open, private, orderly line. Time to say something. The connection is open, and both ends can now send plain text at each other. What the client sends first is called a request, and it always begins with a single line like this. There are three things on it, always in that order: what to do, which thing to do it to, and which version of the rules we are speaking. Get is the method, and it means send me this thing. The path is a single slash, which means the front page. The version is one point one, which almost everything still speaks. Under that first line come the headers. Each one is a name, a colon, and a value, and each is one extra fact about the request. Host says which site you meant, because one machine often answers for hundreds of them. User agent says who is asking. Accept says what kinds of thing you are willing to receive. Then one blank line. That is how the far end knows the headers are finished. After it comes the body, if the request has one, and a get request usually does not. The method is the interesting part, because it is a promise about what the request is going to do. Get asks for something and changes nothing. You can send it a hundred times over and the server is the same afterwards. Post carries something with it and expects the server to be different afterwards. Put replaces a thing with what you sent. Delete removes it. Those meanings are a convention rather than a law. Nothing stops a server doing something else entirely. Almost none of them do, and everything from your browser to a search engine is built on that. Now the other direction. The reply has exactly the same shape: one first line, then headers, then a blank line, then the body. But that first line carries a number where the request had a method, and the number is the whole point. Two hundred means it worked. The numbers come in families, and the family is the first digit. Two hundreds mean it worked. Three hundreds mean the thing is somewhere else. Four hundreds mean the request was wrong. Five hundreds mean the server itself broke. So four zero four is not a mysterious error message. It sits in the four hundreds family, and it means you asked for something that is not there. Then come the reply's own headers: what kind of thing this is, and how long it is. And after the blank line, the thing itself: the page, the picture, the data. That is the entire protocol. A first line, some headers, a blank line, a body, in both directions. All of this is invisible by default. One flag makes every bit of it visible: dash v, for verbose. Now every line that gets printed is marked, and the mark tells you who said it. A star means the program talking about its own work. A greater than sign means the line went out to the server. A less than sign means it came back. So read it from the top. A star: the connection is open, which is the handshake we just watched, all three messages of it. Two lines going out: the request line we took apart, and the host header underneath it. Two lines coming back: the status line with its two hundred, and one of the reply's own headers. That is the whole conversation, in order, on your own screen. The lookup, the connection, the request, the code. Each thing we have talked about is one line of this. Three more commands are worth knowing, and each one is the same request with one thing changed. Dash capital I asks for the headers and nothing else. It sends the HEAD method, so the server answers with the status line and the headers but no body. It is the quick way to ask whether something is there at all. Dash d sends a body. Adding it changes the method to POST all by itself, and whatever you write after the flag is what goes into the body. And dash capital L follows a redirect. When the reply comes back in the three hundreds, the new address is sitting in a header called location, and this makes the program go there and ask again. That last one is a status code doing real work, rather than being an error you read about. So, back to the four steps we started with, and every one of them has now been on the screen. The name became a number, and you watched the lookup. A connection was opened, and you watched the handshake. A request went out, and you read it line by line. A reply came back carrying a code, and you read that too. None of it is magic and none of it is hidden. It is four steps and a great deal of plain text, and one command will print that text for you. That is the internet, from the bottom up. Go and run the command on something you use every day, and read what comes back.","watch":{"version":1,"scenes":[{"title":"One Question and One Answer","start":0,"end":117.10795833333334,"objects":{"claim":"a Text [text] that says \"All four are ordinary text, and all four can be watched.\"","client":"a Polygon [blue] labelled \"upright(\"client\")\" drawn in stage (vertices=((0.4, 1.7), (3.0, 1.7), (3.0, 4.3), (0.4, 4.3)), fill_opacity=0.2)","command":"a Math [green] that says \"$upright(\"curl https://example.com\")$\"","head_plan":"a Heading that says \"Four Steps, Every Single Time\"","incoming":"an Arrow [red] labelled \"upright(\"reply\")\" drawn in stage (start=(6.7, 2.4), end=(3.3, 2.4))","kind":"a Math [text] that says \"$upright(\"content-type: text/html\")$\"","outgoing":"an Arrow [yellow] labelled \"upright(\"request\")\" drawn in stage (start=(3.3, 3.6), end=(6.7, 3.6))","page":"a Math [text] that says \"$upright(\"<html> ... </html>\")$\"","plan":"a Block [text] that says \"A name becomes a number. A connection is opened. A request is written out. A reply comes back with a code.\"","question":"a Panel that says \"You type an address, and a page appears. What actually happens in between?\"","server":"a Polygon [green] labelled \"upright(\"server\")\" drawn in stage (vertices=((7.0, 1.3), (9.6, 1.3), (9.6, 4.7), (7.0, 4.7)), fill_opacity=0.2)","stage":"a Figure (x_range=(0.0, 10.0), y_range=(0.0, 6.0), aspect=(5.0, 3.0))","status":"a Math [text] that says \"$upright(\"HTTP/1.1\") thin 200 thin upright(\"OK\")$\""},"beats":[{"start":0,"say":"You have done this thousands of times. You type an address, something happens for about a second, and a page appears. Almost none of that is visible, and almost all of it is simple.","live":[],"does":[[0,"question is shown on the screen, written out."],[10.7855,"question moves to a new place on the board."]]},{"start":11.9855,"say":"Underneath there are only two machines and two messages. One machine wants something. That is this one, and we call it the client.","live":["question"],"does":[[11.9855,"stage is shown on the screen, written out."],[19.892,"client is shown on the screen, written out."]]},{"start":21.3975,"say":"The other machine has the thing and is waiting to be asked for it. That is the server: a computer in a building somewhere, switched on all the time, doing nothing else.","live":["question","stage","client"],"does":[[25.821,"server is shown on the screen, written out."]]},{"start":31.8315,"say":"The client sends a request. One short piece of text saying what it wants, and a few facts about itself.","live":["question","stage","client","server"],"does":[[33.016,"outgoing is shown on the screen, written out."]]},{"start":39.3395,"say":"The server sends back a reply. A short piece of text saying how it went, and then the thing itself. That is the whole shape of it, and everything else is detail about those two arrows.","live":["question","stage","client","server","outgoing"],"does":[[41.046,"incoming is shown on the screen, written out."],[49.731,"outgoing is indicated — a transient flash."],[50.08337724412267,"incoming is indicated — a transient flash."]]},{"start":51.236000000000004,"say":"So let us do exactly that, using a program called curl. It plays the part of the client, once, and prints whatever comes back. Here is the whole command.","live":["question","stage","client","server","outgoing","incoming"],"does":[[61.151,"stage moves to a new place on the board."],[61.151,"command is shown on the screen, written out."]]},{"start":62.691500000000005,"say":"The first line that came back is the status line. It says which version of the language the server spoke, and then a number. Two hundred means it worked.","live":["question","command","stage","client","server","outgoing","incoming"],"does":[[64.712,"status is shown on the screen, written out."],[70.377,"status (the \"200\" part) is indicated — a transient flash."]]},{"start":72.2775,"say":"Then a few lines about the reply itself. This one says the thing coming back is a web page.","live":["question","command","status","stage","client","server","outgoing","incoming"],"does":[[73.16,"kind is shown on the screen, written out."]]},{"start":78.782,"say":"Then one blank line, and then the page, which is what a browser would draw for you. Here it simply gets printed.","live":["question","command","status","kind","stage","client","server","outgoing","incoming"],"does":[[80.901,"page is shown on the screen, written out."],[86.334,"command is hidden from the screen — left the board."],[86.334,"kind is hidden from the screen — left the board."],[86.334,"page is hidden from the screen — left the board."],[86.334,"question is hidden from the screen — left the board."],[86.334,"stage is hidden from the screen — left the board."],[86.334,"client is hidden from the screen — stage left the board."],[86.334,"server is hidden from the screen — stage left the board."],[86.334,"outgoing is hidden from the screen — stage left the board."],[86.334,"incoming is hidden from the screen — stage left the board."],[86.334,"status is hidden from the screen — left the board."]]},{"start":87.53399999999999,"say":"Four separate things had to happen before any of that text existed, and they happen in the same order every single time.","live":[],"does":[[87.53399999999999,"head_plan is shown on the screen, written out."],[88.01,"plan is shown on the screen, written out."]]},{"start":94.856,"say":"First the name in the address had to become a number. Then a connection had to be opened between the two machines. Then the request had to be written out in a very particular form. And then the reply came back, carrying a code.","live":["plan","head_plan"],"does":[[95.576,"plan (the \"A name becomes a number\" part) is emphasized."],[98.2,"plan (the \"A connection is opened\" part) is emphasized."],[98.2,"plan (the \"A name becomes a number\" part) is no longer emphasized."],[102.124,"plan (the \"A connection is opened\" part) is no longer emphasized."],[102.124,"plan (the \"A request is written out\" part) is emphasized."],[106.489,"plan (the \"A reply comes back with a code\" part) is emphasized."],[106.489,"plan (the \"A request is written out\" part) is no longer emphasized."]]},{"start":108.00699999999999,"say":"None of those four is mysterious, and by the end you will have watched each one happen on your own screen. Let us begin with the number.","live":null,"does":[[108.00699999999999,"plan (the \"A reply comes back with a code\" part) is no longer emphasized."],[111.44300000000001,"claim is shown on the screen, written out."],[116.06629166666666,"claim is hidden from the screen — left the board."],[116.06629166666666,"head_plan is hidden from the screen — left the board."],[116.06629166666666,"plan is hidden from the screen — left the board."]]}]},{"title":"Names and Numbers","start":117.10795833333334,"end":245.52614583333332,"objects":{"address":"a Math [green] that says \"$upright(\"93.184.216.34\")$\"","cmd_v":"a Math [green] that says \"$upright(\"curl -v https://example.com\")$\"","connected":"a Math [text] that says \"$upright(\"* Connected to example.com\")$\"","dns":"a Table [text] that says \"name address example.com 93.184.216.34 example.org 198.51.100.7 example.net 203.0.113.42\" (rows=(('name', 'address'), ('example.com', '93.184.216.34'), ('examp…, header=True)","head_lookup":"a Heading that says \"Turning a Name Into a Number\"","head_numbers":"a Heading that says \"Machines Have Numbers\"","head_watch":"a Heading that says \"Watching the Lookup Happen\"","hop_back":"an Arrow [green] labelled \"upright(\"address\")\" drawn in relay (start=(4.4, 1.5), end=(3.2, 1.5))","hop_on":"an Arrow [gray] drawn in relay (start=(7.6, 2.5), end=(8.8, 2.5))","hop_out":"an Arrow [gray] drawn in relay (start=(3.2, 2.5), end=(4.4, 2.5))","host_a":"a Point [gray] labelled \"upright(\"198.51.100.7\")\" drawn in net (location=(1.8, 5.4))","host_b":"a Point [gray] labelled \"upright(\"203.0.113.42\")\" drawn in net (location=(2.6, 1.5))","host_c":"a Point [gray] labelled \"upright(\"192.0.2.15\")\" drawn in net (location=(9.8, 5.6))","ip_defn":"a Panel that says \"An IP address is the number that identifies one machine on the network. Every piece of every message carries the address it is going to.\"","me":"a Polygon [blue] labelled \"upright(\"you\")\" drawn in relay (vertices=((0.3, 1.0), (3.0, 1.0), (3.0, 3.0), (0.3, 3.0)), fill_opacity=0.18)","net":"a Figure (x_range=(0.0, 12.0), y_range=(0.0, 7.0), aspect=(12.0, 7.0))","registry":"a Polygon [gray] labelled \"upright(\"name servers\")\" drawn in relay (vertices=((9.0, 1.0), (11.7, 1.0), (11.7, 3.0), (9.0, 3.0)), fill_opacity=0.18)","relay":"a Figure (x_range=(0.0, 12.0), y_range=(0.0, 4.0), aspect=(6.0, 2.0))","resolver":"a Polygon [yellow] labelled \"upright(\"resolver\")\" drawn in relay (vertices=((4.6, 1.0), (7.4, 1.0), (7.4, 3.0), (4.6, 3.0)), fill_opacity=0.18)","target":"a Point [green] labelled \"upright(\"93.184.216.34\")\" drawn in net (location=(9.1, 1.9))","trying":"a Math [text] that says \"$upright(\"* Trying 93.184.216.34:\") 443$\""},"beats":[{"start":117.10795833333334,"say":"The name you typed is for you. It is a convenience for humans, and the machines in between do not use it at all. What they use is a number.","live":[],"does":[[117.10795833333334,"head_numbers is shown on the screen, written out."],[117.10795833333334,"net is shown on the screen, written out."]]},{"start":126.60145833333334,"say":"Every machine that can be reached from the internet has one of these addresses. Here are four of them: four machines, somewhere in the world.","live":["net","head_numbers"],"does":[[131.26795833333335,"host_a is shown on the screen, written out."],[131.47850767703775,"host_b is shown on the screen, written out."],[132.45819549100634,"host_c is shown on the screen, written out."],[133.42151895559877,"target is shown on the screen, written out."]]},{"start":135.64145833333333,"say":"One of them is the one we are after, the machine holding the page we asked for. It is this one, in green.","live":["net","head_numbers","host_a","host_b","host_c","target"],"does":[[141.70195833333332,"target is indicated — a transient flash."]]},{"start":143.10295833333333,"say":"Written down, an address is four small numbers with dots between them. Each one runs from zero up to two hundred and fifty five, which is as much as eight bits can hold.","live":null,"does":[[143.45095833333335,"net moves to a new place on the board."],[143.45095833333335,"address is shown on the screen, written out."]]},{"start":153.72195833333333,"say":"That number is the destination, and it is the only part of the address the network in between cares about. The name itself never travels anywhere.","live":["address","net","head_numbers","host_a","host_b","host_c","target"],"does":[[154.77895833333335,"ip_defn is shown on the screen, written out."],[162.08695833333334,"address is hidden from the screen — left the board."],[162.08695833333334,"head_numbers is hidden from the screen — left the board."],[162.08695833333334,"ip_defn is hidden from the screen — left the board."],[162.08695833333334,"net is hidden from the screen — left the board."],[162.08695833333334,"host_a is hidden from the screen — net left the board."],[162.08695833333334,"host_b is hidden from the screen — net left the board."],[162.08695833333334,"host_c is hidden from the screen — net left the board."],[162.08695833333334,"target is hidden from the screen — net left the board."]]},{"start":163.28695833333333,"say":"Which leaves us a problem. You typed a name. The wire wants a number. Something has to do the translation, and it has to happen before anything else.","live":[],"does":[[163.28695833333333,"head_lookup is shown on the screen, written out."],[169.38795833333333,"dns is shown on the screen, written out."]]},{"start":173.12295833333334,"say":"That something is called the domain name system. Picture it as one enormous table with two columns: names on the left, addresses on the right.","live":["head_lookup"],"does":[[178.10295833333333,"dns is shown on the screen, written out."],[178.45939004948104,"dns is shown on the screen, written out."],[179.53290941547993,"dns is shown on the screen, written out."]]},{"start":183.45145833333333,"say":"No single machine holds that table. It is cut into pieces, spread over thousands of machines, and each piece is kept by whoever owns those particular names.","live":null,"does":[[191.47395833333331,"dns (the \"example.com\" part) is indicated — a transient flash."]]},{"start":193.91945833333335,"say":"So your computer asks a machine nearby, called a resolver. If the resolver does not know the answer, it asks the machines that do, and passes back what it gets.","live":null,"does":[[193.91945833333335,"relay is shown on the screen, written out."],[193.91945833333335,"me is shown on the screen, written out."],[197.22795833333333,"resolver is shown on the screen, written out."],[198.0334760182208,"hop_out is shown on the screen, written out."],[201.69795833333333,"registry is shown on the screen, written out."],[201.95162392818864,"hop_on is shown on the screen, written out."],[203.06795833333334,"hop_back is shown on the screen, written out."]]},{"start":205.51395833333333,"say":"And the answer is remembered for a while, both on your machine and on the resolver. That is why the first visit to a site is a little slower than the second.","live":["relay","head_lookup","me","resolver","hop_out","registry","hop_on","hop_back"],"does":[[209.60095833333332,"resolver is indicated — a transient flash."],[214.31495833333332,"dns is hidden from the screen — left the board."],[214.31495833333332,"head_lookup is hidden from the screen — left the board."]]},{"start":215.51495833333334,"say":"You do not have to take any of this on faith. Add one flag to the command, dash v for verbose, and the program narrates its own work as it goes.","live":["relay","me","resolver","hop_out","registry","hop_on","hop_back"],"does":[[215.51495833333334,"head_watch is shown on the screen, written out."],[218.69595833333335,"cmd_v is shown on the screen, written out."]]},{"start":224.81145833333335,"say":"The first thing it says is the number it looked up, and the door it is knocking on at that address. Four four three is called a port, and it is the one kept for encrypted web traffic.","live":["relay","me","resolver","hop_out","registry","hop_on","hop_back","cmd_v","head_watch"],"does":[[226.16995833333334,"trying is shown on the screen, written out."],[232.46195833333334,"trying (the \"443\" part) is indicated — a transient flash."]]},{"start":236.54545833333333,"say":"Then it says connected. From that moment the name is finished with, and everything that follows happens between two numbers.","live":["relay","me","resolver","hop_out","registry","hop_on","hop_back","cmd_v","trying","head_watch"],"does":[[237.58995833333336,"connected is shown on the screen, written out."],[244.48447916666663,"cmd_v is hidden from the screen — left the board."],[244.48447916666663,"connected is hidden from the screen — left the board."],[244.48447916666663,"head_watch is hidden from the screen — left the board."],[244.48447916666663,"relay is hidden from the screen — left the board."],[244.48447916666663,"me is hidden from the screen — relay left the board."],[244.48447916666663,"resolver is hidden from the screen — relay left the board."],[244.48447916666663,"hop_out is hidden from the screen — relay left the board."],[244.48447916666663,"registry is hidden from the screen — relay left the board."],[244.48447916666663,"hop_on is hidden from the screen — relay left the board."],[244.48447916666663,"hop_back is hidden from the screen — relay left the board."],[244.48447916666663,"trying is hidden from the screen — left the board."]]}]},{"title":"Opening a Line","start":245.52614583333332,"end":389.33045833333324,"objects":{"ack":"an Arrow [yellow] labelled \"upright(\"ACK\")\" drawn in wire (start=(2.2, 3.3), end=(7.8, 2.7))","client_line":"a Line [blue] labelled \"upright(\"client\")\" drawn in wire (start=(2.0, 0.6), end=(2.0, 7.4), dashed=True)","cmd_s":"a Math [green] that says \"$upright(\"curl -v https://example.com\")$\"","head_hand":"a Heading that says \"Three Messages, Then a Line\"","head_https":"a Heading that says \"The S in HTTPS\"","head_pieces":"a Heading that says \"The Network Moves Pieces\"","order":"a Table [text] that says \"sent 1 2 3 4 arrived 1 3 2 4\" (rows=(('sent', '1', '2', '3', '4'), ('arrived', '1', '3', '2', '4')))","server_line":"a Line [green] labelled \"upright(\"server\")\" drawn in wire (start=(8.0, 0.6), end=(8.0, 7.4), dashed=True)","step_one":"a Math [text] that says \"$upright(\"SYN: may I talk?\")$\"","step_three":"a Math [text] that says \"$upright(\"ACK: yes\")$\"","step_two":"a Math [text] that says \"$upright(\"SYN, ACK: yes, and may I?\")$\"","syn":"an Arrow [yellow] labelled \"upright(\"SYN\")\" drawn in wire (start=(2.2, 6.5), end=(7.8, 5.9))","synack":"an Arrow [red] labelled \"upright(\"SYN, ACK\")\" drawn in wire (start=(7.8, 4.9), end=(2.2, 4.3))","tcp_defn":"a Panel that says \"TCP numbers the pieces, asks again for the ones that go missing, and hands them over in the order they were sent. What the two programs see is one clean stream of text in each direction.\"","tls_cert":"a Math [text] that says \"$upright(\"* certificate verified\")$\"","tls_done":"a Math [text] that says \"$upright(\"* TLS handshake, done\")$\"","wire":"a Figure (x_range=(0.0, 10.0), y_range=(0.0, 8.0))"},"beats":[{"start":245.52614583333332,"say":"We know where to send things now. That still is not a conversation, because the network in between does not carry conversations. It carries small pieces, one at a time, and it promises nothing about them.","live":[],"does":[[245.52614583333332,"head_pieces is shown on the screen, written out."],[254.0361458333333,"order is shown on the screen, written out."]]},{"start":258.87414583333333,"say":"Send four pieces in this order. Each one is passed from machine to machine along whatever path happens to be free, so they arrive in whatever order they manage.","live":["head_pieces"],"does":[[266.72214583333334,"order is shown on the screen, written out."]]},{"start":269.55164583333334,"say":"Here the third piece overtook the second. And sometimes a piece never arrives at all, and nothing along the way even notices.","live":null,"does":[[270.42214583333333,"order (the \"3\" part) is indicated — a transient flash."]]},{"start":278.3246458333333,"say":"So something has to number the pieces, keep track of which ones landed, ask again for the missing ones, and hand the whole thing over in order. That something is TCP, and it is the reason you have never had to think about any of this.","live":null,"does":[[288.41414583333335,"tcp_defn is shown on the screen, written out."],[292.95264583333335,"head_pieces is hidden from the screen — left the board."],[292.95264583333335,"order is hidden from the screen — left the board."],[292.95264583333335,"tcp_defn is hidden from the screen — left the board."]]},{"start":294.15264583333334,"say":"Setting one of those connections up takes three short messages, and nothing useful is said in any of them. Time runs downward here.","live":[],"does":[[294.15264583333334,"head_hand is shown on the screen, written out."],[296.3001458333333,"wire is shown on the screen, written out."],[296.3001458333333,"client_line is shown on the screen, written out."],[296.48079436889816,"server_line is shown on the screen, written out."]]},{"start":302.6941458333333,"say":"The client sends the first one. It means: I would like to talk to you. It is called SYN, short for synchronise.","live":["wire","head_hand","client_line","server_line"],"does":[[303.9251458333333,"syn is shown on the screen, written out."],[308.6151458333333,"wire moves to a new place on the board."],[308.6151458333333,"step_one is shown on the screen, written out."]]},{"start":311.73464583333333,"say":"The server sends the second, which means: I heard you, and I would like to talk to you as well.","live":["step_one","wire","head_hand","client_line","server_line","syn"],"does":[[312.9651458333333,"synack is shown on the screen, written out."],[314.6951458333333,"step_two is shown on the screen, written out."]]},{"start":317.9421458333333,"say":"And the client sends the third: I heard that. Now both ends know the other one can hear them, and both know where to start counting. That is the three way handshake.","live":["step_one","step_two","wire","head_hand","client_line","server_line","syn","synack"],"does":[[319.1961458333333,"ack is shown on the screen, written out."],[320.5431458333333,"step_three is shown on the screen, written out."]]},{"start":329.9426458333333,"say":"It costs one full round trip before a single useful byte moves. That is why a server on the far side of the world feels slow even when it is not busy at all.","live":["step_one","step_two","step_three","wire","head_hand","client_line","server_line","syn","synack","ack"],"does":[[331.48714583333333,"syn is indicated — a transient flash."],[331.7270104030144,"synack is indicated — a transient flash."],[332.20461746785867,"ack is indicated — a transient flash."],[340.1246458333333,"head_hand is hidden from the screen — left the board."],[340.1246458333333,"step_one is hidden from the screen — left the board."],[340.1246458333333,"step_three is hidden from the screen — left the board."],[340.1246458333333,"step_two is hidden from the screen — left the board."]]},{"start":341.3246458333333,"say":"One more thing happens when the address begins with https. Before any useful words are exchanged, the two ends agree on how to encrypt everything that follows.","live":["wire","client_line","server_line","syn","synack","ack"],"does":[[341.3246458333333,"head_https is shown on the screen, written out."],[344.03014583333334,"cmd_s is shown on the screen, written out."]]},{"start":352.6636458333333,"say":"That agreement is a short exchange of its own, and it gets reported when it finishes. From here on, anybody watching the wire in between sees only noise.","live":["wire","client_line","server_line","syn","synack","ack","cmd_s","head_https"],"does":[[356.19314583333335,"tls_done is shown on the screen, written out."]]},{"start":363.61914583333333,"say":"The server also hands over a certificate: a document signed by an organisation your machine already trusts, saying that this really is the machine for that name. It gets checked, and here it passed.","live":["wire","client_line","server_line","syn","synack","ack","cmd_s","tls_done","head_https"],"does":[[373.9061458333333,"tls_cert is shown on the screen, written out."]]},{"start":376.4446458333333,"say":"If that check fails, the program stops and tells you, and your request is never sent. So now we have an address, and an open, private, orderly line. Time to say something.","live":["wire","client_line","server_line","syn","synack","ack","cmd_s","tls_done","tls_cert","head_https"],"does":[[388.2887916666666,"cmd_s is hidden from the screen — left the board."],[388.2887916666666,"head_https is hidden from the screen — left the board."],[388.2887916666666,"tls_cert is hidden from the screen — left the board."],[388.2887916666666,"tls_done is hidden from the screen — left the board."],[388.2887916666666,"wire is hidden from the screen — left the board."],[388.2887916666666,"client_line is hidden from the screen — wire left the board."],[388.2887916666666,"server_line is hidden from the screen — wire left the board."],[388.2887916666666,"syn is hidden from the screen — wire left the board."],[388.2887916666666,"synack is hidden from the screen — wire left the board."],[388.2887916666666,"ack is hidden from the screen — wire left the board."]]}]},{"title":"What Gets Said","start":389.33045833333324,"end":562.3371874999999,"objects":{"codes":"a Table [text] that says \"code what it means 2xx it worked 3xx it is somewhere else 4xx the request was wrong 5xx the server broke\" (rows=(('code', 'what it means'), ('2xx', 'it worked'), ('3xx', 'it i…, header=True)","head_methods":"a Heading that says \"The Method Is a Promise\"","head_req":"a Heading that says \"What the Client Says\"","head_res":"a Heading that says \"What the Server Says Back\"","method_note":"a Panel that says \"Nothing forces a server to keep these meanings. Almost every server does, and the whole web is written on the assumption that they hold.\"","methods":"a Table [text] that says \"method what it means GET send me this thing POST here is something new PUT replace it with this DELETE remove it\" (rows=(('method', 'what it means'), ('GET', 'send me this thing'), ('…, header=True)","parts":"a Table [text] that says \"part says here method what to do GET path which thing $upright(\"/\")$ version which rules HTTP/1.1\" (rows=(('part', 'says', 'here'), ('method', 'what to do', 'GET'), ('p…, header=True)","req_accept":"a Math [text] that says \"$upright(\"Accept: */*\")$\"","req_agent":"a Math [text] that says \"$upright(\"User-Agent: curl/8.4.0\")$\"","req_blank":"a Text [text] that says \"Then one blank line, and then a body, if there is one.\"","req_host":"a Math [text] that says \"$upright(\"Host: example.com\")$\"","req_line":"a Math [text] that says \"$upright(\"GET / HTTP/1.1\")$\"","res_body":"a Text [text] that says \"Then a blank line, and then the thing itself.\"","res_len":"a Math [text] that says \"$upright(\"content-length: 1256\")$\"","res_status":"a Math [text] that says \"$upright(\"HTTP/1.1\") thin 200 thin upright(\"OK\")$\"","res_type":"a Math [text] that says \"$upright(\"content-type: text/html\")$\""},"beats":[{"start":389.33045833333324,"say":"The connection is open, and both ends can now send plain text at each other. What the client sends first is called a request, and it always begins with a single line like this.","live":[],"does":[[389.33045833333324,"head_req is shown on the screen, written out."],[398.42145833333325,"req_line is shown on the screen, written out."]]},{"start":401.31995833333326,"say":"There are three things on it, always in that order: what to do, which thing to do it to, and which version of the rules we are speaking.","live":["req_line","head_req"],"does":[[402.05145833333324,"parts is shown on the screen, written out."],[404.66345833333327,"parts is shown on the screen, written out."],[405.73145833333325,"parts is shown on the screen, written out."],[407.50845833333324,"parts is shown on the screen, written out."]]},{"start":410.25595833333324,"say":"Get is the method, and it means send me this thing. The path is a single slash, which means the front page. The version is one point one, which almost everything still speaks.","live":null,"does":[[411.11545833333327,"parts (the \"row=2\" part) is emphasized."],[414.36545833333327,"parts (the \"row=2\" part) is no longer emphasized."],[414.36545833333327,"parts (the \"row=3\" part) is emphasized."],[417.8724583333332,"parts (the \"row=3\" part) is no longer emphasized."],[417.8724583333332,"parts (the \"row=4\" part) is emphasized."]]},{"start":422.68645833333323,"say":"Under that first line come the headers. Each one is a name, a colon, and a value, and each is one extra fact about the request.","live":null,"does":[[422.68645833333323,"parts (the \"row=4\" part) is no longer emphasized."],[424.5094583333332,"req_host is shown on the screen, written out."],[425.19698080726346,"req_agent is shown on the screen, written out."],[426.0345845469255,"req_accept is shown on the screen, written out."]]},{"start":432.46995833333324,"say":"Host says which site you meant, because one machine often answers for hundreds of them. User agent says who is asking. Accept says what kinds of thing you are willing to receive.","live":["req_line","req_host","req_agent","req_accept","head_req"],"does":[[432.87645833333323,"req_host is indicated — a transient flash."],[438.64545833333324,"req_agent is indicated — a transient flash."],[441.18845833333324,"req_accept is indicated — a transient flash."]]},{"start":445.2019583333332,"say":"Then one blank line. That is how the far end knows the headers are finished. After it comes the body, if the request has one, and a get request usually does not.","live":null,"does":[[446.1534583333332,"req_blank is shown on the screen, written out."],[456.53245833333324,"head_req is hidden from the screen — left the board."],[456.53245833333324,"parts is hidden from the screen — left the board."],[456.53245833333324,"req_accept is hidden from the screen — left the board."],[456.53245833333324,"req_agent is hidden from the screen — left the board."],[456.53245833333324,"req_blank is hidden from the screen — left the board."],[456.53245833333324,"req_host is hidden from the screen — left the board."],[456.53245833333324,"req_line is hidden from the screen — left the board."]]},{"start":457.7324583333332,"say":"The method is the interesting part, because it is a promise about what the request is going to do.","live":[],"does":[[457.7324583333332,"head_methods is shown on the screen, written out."],[460.6124583333333,"methods is shown on the screen, written out."]]},{"start":463.90545833333323,"say":"Get asks for something and changes nothing. You can send it a hundred times over and the server is the same afterwards.","live":["head_methods"],"does":[[464.25345833333324,"methods is shown on the screen, written out."]]},{"start":471.61095833333326,"say":"Post carries something with it and expects the server to be different afterwards. Put replaces a thing with what you sent. Delete removes it.","live":null,"does":[[471.95945833333326,"methods is shown on the screen, written out."],[476.4754583333332,"methods is shown on the screen, written out."],[479.13445833333327,"methods is shown on the screen, written out."]]},{"start":481.2324583333332,"say":"Those meanings are a convention rather than a law. Nothing stops a server doing something else entirely. Almost none of them do, and everything from your browser to a search engine is built on that.","live":null,"does":[[482.74145833333324,"method_note is shown on the screen, written out."],[493.36445833333323,"head_methods is hidden from the screen — left the board."],[493.36445833333323,"method_note is hidden from the screen — left the board."],[493.36445833333323,"methods is hidden from the screen — left the board."]]},{"start":494.5644583333332,"say":"Now the other direction. The reply has exactly the same shape: one first line, then headers, then a blank line, then the body.","live":[],"does":[[494.5644583333332,"head_res is shown on the screen, written out."],[499.85845833333326,"res_status is shown on the screen, written out."]]},{"start":504.83545833333324,"say":"But that first line carries a number where the request had a method, and the number is the whole point. Two hundred means it worked.","live":["res_status","head_res"],"does":[[511.38345833333324,"res_status (the \"200\" part) is indicated — a transient flash."]]},{"start":513.2029583333333,"say":"The numbers come in families, and the family is the first digit. Two hundreds mean it worked. Three hundreds mean the thing is somewhere else. Four hundreds mean the request was wrong. Five hundreds mean the server itself broke.","live":null,"does":[[514.3054583333333,"codes is shown on the screen, written out."],[517.5674583333332,"codes is shown on the screen, written out."],[520.0284583333332,"codes is shown on the screen, written out."],[523.4424583333332,"codes is shown on the screen, written out."],[526.2984583333332,"codes is shown on the screen, written out."]]},{"start":529.9054583333332,"say":"So four zero four is not a mysterious error message. It sits in the four hundreds family, and it means you asked for something that is not there.","live":null,"does":[[536.2214583333332,"codes (the \"row=4\" part) is emphasized."]]},{"start":540.3509583333332,"say":"Then come the reply's own headers: what kind of thing this is, and how long it is.","live":null,"does":[[540.3509583333332,"codes (the \"row=4\" part) is no longer emphasized."],[543.7644583333332,"res_type is shown on the screen, written out."],[544.9944583333332,"res_len is shown on the screen, written out."]]},{"start":546.8254583333332,"say":"And after the blank line, the thing itself: the page, the picture, the data. That is the entire protocol. A first line, some headers, a blank line, a body, in both directions.","live":["res_status","res_type","res_len","head_res"],"does":[[547.8124583333333,"res_body is shown on the screen, written out."],[561.2955208333333,"codes is hidden from the screen — left the board."],[561.2955208333333,"head_res is hidden from the screen — left the board."],[561.2955208333333,"res_body is hidden from the screen — left the board."],[561.2955208333333,"res_len is hidden from the screen — left the board."],[561.2955208333333,"res_status is hidden from the screen — left the board."],[561.2955208333333,"res_type is hidden from the screen — left the board."]]}]},{"title":"Seeing It for Yourself","start":562.3371874999999,"end":715.6468124999999,"objects":{"closing":"a Panel that says \"Every layer named here is text on a wire. One command prints that text, which is why it is the quickest way to learn what any of it actually does.\"","cmd_v":"a Math [green] that says \"$upright(\"curl -v https://example.com\")$\"","extras":"a Table [text] that says \"command what changes curl -I headers only, no body curl -d name=ada sends a body, method becomes POST curl -L follows a redirect\" (rows=(('command', 'what changes'), ('curl -I', 'headers only, no bod…, header=True)","flag_note":"a Panel that says \"A flag is an option, written with a dash in front of it. Each one changes what gets sent, or what gets printed, and nothing else.\"","head_more":"a Heading that says \"Three More Commands\"","head_recap":"a Heading that says \"Four Steps, All Visible\"","head_watch":"a Heading that says \"Watching the Whole Conversation\"","prefixes":"a Table [text] that says \"mark who is speaking $upright(\"*\")$ curl, about its own work $upright(\">\")$ a line we sent out $upright(\"<\")$ a line that came back\" (rows=(('mark', 'who is speaking'), ('$upright(\"*\")$', 'curl, about i…, header=True)","recap":"a Block [text] that says \"A name becomes a number. A connection is opened. A request is written out. A reply comes back with a code.\"","t_get":"a Math [text] that says \"$upright(\"> GET / HTTP/1.1\")$\"","t_host":"a Math [text] that says \"$upright(\"> Host: example.com\")$\"","t_open":"a Math [text] that says \"$upright(\"* Connected to example.com\")$\"","t_status":"a Math [text] that says \"$upright(\"< HTTP/1.1 200 OK\")$\"","t_type":"a Math [text] that says \"$upright(\"< content-type: text/html\")$\""},"beats":[{"start":562.3371874999999,"say":"All of this is invisible by default. One flag makes every bit of it visible: dash v, for verbose.","live":[],"does":[[562.3371874999999,"head_watch is shown on the screen, written out."],[565.1701874999999,"cmd_v is shown on the screen, written out."]]},{"start":570.4951874999999,"say":"Now every line that gets printed is marked, and the mark tells you who said it.","live":["cmd_v","head_watch"],"does":[[572.8871874999999,"prefixes is shown on the screen, written out."]]},{"start":576.4126875,"say":"A star means the program talking about its own work. A greater than sign means the line went out to the server. A less than sign means it came back.","live":null,"does":[[576.9351874999999,"prefixes is shown on the screen, written out."],[580.5581874999999,"prefixes is shown on the screen, written out."],[583.9481874999999,"prefixes is shown on the screen, written out."]]},{"start":586.8351875,"say":"So read it from the top. A star: the connection is open, which is the handshake we just watched, all three messages of it.","live":null,"does":[[589.4941875,"t_open is shown on the screen, written out."]]},{"start":596.5841874999999,"say":"Two lines going out: the request line we took apart, and the host header underneath it.","live":["cmd_v","t_open","head_watch"],"does":[[599.0691874999999,"t_get is shown on the screen, written out."],[600.9841875,"t_host is shown on the screen, written out."]]},{"start":603.3256875,"say":"Two lines coming back: the status line with its two hundred, and one of the reply's own headers.","live":["cmd_v","t_open","t_get","t_host","head_watch"],"does":[[605.5081875,"t_status is shown on the screen, written out."],[608.7131874999999,"t_type is shown on the screen, written out."]]},{"start":610.2066874999999,"say":"That is the whole conversation, in order, on your own screen. The lookup, the connection, the request, the code. Each thing we have talked about is one line of this.","live":["cmd_v","t_open","t_get","t_host","t_status","t_type","head_watch"],"does":[[616.1051874999999,"t_open is indicated — a transient flash."],[617.1841874999999,"t_get is indicated — a transient flash."],[618.2411874999999,"t_status is indicated — a transient flash."],[622.4091874999999,"cmd_v is hidden from the screen — left the board."],[622.4091874999999,"head_watch is hidden from the screen — left the board."],[622.4091874999999,"prefixes is hidden from the screen — left the board."],[622.4091874999999,"t_get is hidden from the screen — left the board."],[622.4091874999999,"t_host is hidden from the screen — left the board."],[622.4091874999999,"t_open is hidden from the screen — left the board."],[622.4091874999999,"t_status is hidden from the screen — left the board."],[622.4091874999999,"t_type is hidden from the screen — left the board."]]},{"start":623.6091875,"say":"Three more commands are worth knowing, and each one is the same request with one thing changed.","live":[],"does":[[623.6091875,"head_more is shown on the screen, written out."],[623.9571874999999,"extras is shown on the screen, written out."]]},{"start":629.8396874999999,"say":"Dash capital I asks for the headers and nothing else. It sends the HEAD method, so the server answers with the status line and the headers but no body. It is the quick way to ask whether something is there at all.","live":["head_more"],"does":[[632.1851875,"extras is shown on the screen, written out."],[635.4481874999999,"extras (the \"row=2\" part) is emphasized."]]},{"start":645.2661874999999,"say":"Dash d sends a body. Adding it changes the method to POST all by itself, and whatever you write after the flag is what goes into the body.","live":null,"does":[[646.2181874999999,"extras is shown on the screen, written out."],[649.2601874999999,"extras (the \"row=2\" part) is no longer emphasized."],[649.2601874999999,"extras (the \"row=3\" part) is emphasized."]]},{"start":654.9566874999999,"say":"And dash capital L follows a redirect. When the reply comes back in the three hundreds, the new address is sitting in a header called location, and this makes the program go there and ask again.","live":null,"does":[[656.7561874999999,"extras is shown on the screen, written out."],[663.8031874999999,"extras (the \"row=3\" part) is no longer emphasized."],[663.8031874999999,"extras (the \"row=4\" part) is emphasized."]]},{"start":668.5596874999999,"say":"That last one is a status code doing real work, rather than being an error you read about.","live":null,"does":[[668.5596874999999,"flag_note is shown on the screen, written out."],[668.5596874999999,"extras (the \"row=4\" part) is no longer emphasized."],[674.5156874999999,"extras is hidden from the screen — left the board."],[674.5156874999999,"flag_note is hidden from the screen — left the board."],[674.5156874999999,"head_more is hidden from the screen — left the board."]]},{"start":675.7156875,"say":"So, back to the four steps we started with, and every one of them has now been on the screen.","live":[],"does":[[675.7156875,"head_recap is shown on the screen, written out."],[676.9231874999999,"recap is shown on the screen, written out."]]},{"start":682.1556874999999,"say":"The name became a number, and you watched the lookup. A connection was opened, and you watched the handshake. A request went out, and you read it line by line. A reply came back carrying a code, and you read that too.","live":["recap","head_recap"],"does":[[682.7481874999999,"recap (the \"A name becomes a number\" part) is emphasized."],[686.3231874999999,"recap (the \"A connection is opened\" part) is emphasized."],[686.3231874999999,"recap (the \"A name becomes a number\" part) is no longer emphasized."],[689.9111874999999,"recap (the \"A connection is opened\" part) is no longer emphasized."],[689.9111874999999,"recap (the \"A request is written out\" part) is emphasized."],[693.4981874999999,"recap (the \"A reply comes back with a code\" part) is emphasized."],[693.4981874999999,"recap (the \"A request is written out\" part) is no longer emphasized."]]},{"start":697.7206874999999,"say":"None of it is magic and none of it is hidden. It is four steps and a great deal of plain text, and one command will print that text for you.","live":null,"does":[[697.7206874999999,"recap (the \"A reply comes back with a code\" part) is no longer emphasized."],[702.4921874999999,"closing is shown on the screen, written out."]]},{"start":707.1401874999999,"say":"That is the internet, from the bottom up. Go and run the command on something you use every day, and read what comes back.","live":["recap","closing","head_recap"],"does":[[714.6051458333333,"closing is hidden from the screen — left the board."],[714.6051458333333,"head_recap is hidden from the screen — left the board."],[714.6051458333333,"recap is hidden from the screen — left the board."]]}]}]},"durationSeconds":716,"chapters":[{"title":"One Question and One Answer","startSeconds":0,"narration":"You have done this thousands of times. You type an address, something happens for about a second, and a page appears. Almost none of that is visible, and almost all of it is simple. Underneath there are only two machines and two messages. One machine wants something. That is this one, and we call it the client. The other machine has the thing and is waiting to be asked for it. That is the server: a computer in a building somewhere, switched on all the time, doing nothing else. The client sends a request. One short piece of text saying what it wants, and a few facts about itself. The server sends back a reply. A short piece of text saying how it went, and then the thing itself. That is the whole shape of it, and everything else is detail about those two arrows. So let us do exactly that, using a program called curl. It plays the part of the client, once, and prints whatever comes back. Here is the whole command. The first line that came back is the status line. It says which version of the language the server spoke, and then a number. Two hundred means it worked. Then a few lines about the reply itself. This one says the thing coming back is a web page. Then one blank line, and then the page, which is what a browser would draw for you. Here it simply gets printed. Four separate things had to happen before any of that text existed, and they happen in the same order every single time. First the name in the address had to become a number. Then a connection had to be opened between the two machines. Then the request had to be written out in a very particular form. And then the reply came back, carrying a code. None of those four is mysterious, and by the end you will have watched each one happen on your own screen. Let us begin with the number."},{"title":"Names and Numbers","startSeconds":117.10795833333334,"narration":"The name you typed is for you. It is a convenience for humans, and the machines in between do not use it at all. What they use is a number. Every machine that can be reached from the internet has one of these addresses. Here are four of them: four machines, somewhere in the world. One of them is the one we are after, the machine holding the page we asked for. It is this one, in green. Written down, an address is four small numbers with dots between them. Each one runs from zero up to two hundred and fifty five, which is as much as eight bits can hold. That number is the destination, and it is the only part of the address the network in between cares about. The name itself never travels anywhere. Which leaves us a problem. You typed a name. The wire wants a number. Something has to do the translation, and it has to happen before anything else. That something is called the domain name system. Picture it as one enormous table with two columns: names on the left, addresses on the right. No single machine holds that table. It is cut into pieces, spread over thousands of machines, and each piece is kept by whoever owns those particular names. So your computer asks a machine nearby, called a resolver. If the resolver does not know the answer, it asks the machines that do, and passes back what it gets. And the answer is remembered for a while, both on your machine and on the resolver. That is why the first visit to a site is a little slower than the second. You do not have to take any of this on faith. Add one flag to the command, dash v for verbose, and the program narrates its own work as it goes. The first thing it says is the number it looked up, and the door it is knocking on at that address. Four four three is called a port, and it is the one kept for encrypted web traffic. Then it says connected. From that moment the name is finished with, and everything that follows happens between two numbers."},{"title":"Opening a Line","startSeconds":245.52614583333332,"narration":"We know where to send things now. That still is not a conversation, because the network in between does not carry conversations. It carries small pieces, one at a time, and it promises nothing about them. Send four pieces in this order. Each one is passed from machine to machine along whatever path happens to be free, so they arrive in whatever order they manage. Here the third piece overtook the second. And sometimes a piece never arrives at all, and nothing along the way even notices. So something has to number the pieces, keep track of which ones landed, ask again for the missing ones, and hand the whole thing over in order. That something is TCP, and it is the reason you have never had to think about any of this. Setting one of those connections up takes three short messages, and nothing useful is said in any of them. Time runs downward here. The client sends the first one. It means: I would like to talk to you. It is called SYN, short for synchronise. The server sends the second, which means: I heard you, and I would like to talk to you as well. And the client sends the third: I heard that. Now both ends know the other one can hear them, and both know where to start counting. That is the three way handshake. It costs one full round trip before a single useful byte moves. That is why a server on the far side of the world feels slow even when it is not busy at all. One more thing happens when the address begins with https. Before any useful words are exchanged, the two ends agree on how to encrypt everything that follows. That agreement is a short exchange of its own, and it gets reported when it finishes. From here on, anybody watching the wire in between sees only noise. The server also hands over a certificate: a document signed by an organisation your machine already trusts, saying that this really is the machine for that name. It gets checked, and here it passed. If that check fails, the program stops and tells you, and your request is never sent. So now we have an address, and an open, private, orderly line. Time to say something."},{"title":"What Gets Said","startSeconds":389.33045833333324,"narration":"The connection is open, and both ends can now send plain text at each other. What the client sends first is called a request, and it always begins with a single line like this. There are three things on it, always in that order: what to do, which thing to do it to, and which version of the rules we are speaking. Get is the method, and it means send me this thing. The path is a single slash, which means the front page. The version is one point one, which almost everything still speaks. Under that first line come the headers. Each one is a name, a colon, and a value, and each is one extra fact about the request. Host says which site you meant, because one machine often answers for hundreds of them. User agent says who is asking. Accept says what kinds of thing you are willing to receive. Then one blank line. That is how the far end knows the headers are finished. After it comes the body, if the request has one, and a get request usually does not. The method is the interesting part, because it is a promise about what the request is going to do. Get asks for something and changes nothing. You can send it a hundred times over and the server is the same afterwards. Post carries something with it and expects the server to be different afterwards. Put replaces a thing with what you sent. Delete removes it. Those meanings are a convention rather than a law. Nothing stops a server doing something else entirely. Almost none of them do, and everything from your browser to a search engine is built on that. Now the other direction. The reply has exactly the same shape: one first line, then headers, then a blank line, then the body. But that first line carries a number where the request had a method, and the number is the whole point. Two hundred means it worked. The numbers come in families, and the family is the first digit. Two hundreds mean it worked. Three hundreds mean the thing is somewhere else. Four hundreds mean the request was wrong. Five hundreds mean the server itself broke. So four zero four is not a mysterious error message. It sits in the four hundreds family, and it means you asked for something that is not there. Then come the reply's own headers: what kind of thing this is, and how long it is. And after the blank line, the thing itself: the page, the picture, the data. That is the entire protocol. A first line, some headers, a blank line, a body, in both directions."},{"title":"Seeing It for Yourself","startSeconds":562.3371874999999,"narration":"All of this is invisible by default. One flag makes every bit of it visible: dash v, for verbose. Now every line that gets printed is marked, and the mark tells you who said it. A star means the program talking about its own work. A greater than sign means the line went out to the server. A less than sign means it came back. So read it from the top. A star: the connection is open, which is the handshake we just watched, all three messages of it. Two lines going out: the request line we took apart, and the host header underneath it. Two lines coming back: the status line with its two hundred, and one of the reply's own headers. That is the whole conversation, in order, on your own screen. The lookup, the connection, the request, the code. Each thing we have talked about is one line of this. Three more commands are worth knowing, and each one is the same request with one thing changed. Dash capital I asks for the headers and nothing else. It sends the HEAD method, so the server answers with the status line and the headers but no body. It is the quick way to ask whether something is there at all. Dash d sends a body. Adding it changes the method to POST all by itself, and whatever you write after the flag is what goes into the body. And dash capital L follows a redirect. When the reply comes back in the three hundreds, the new address is sitting in a header called location, and this makes the program go there and ask again. That last one is a status code doing real work, rather than being an error you read about. So, back to the four steps we started with, and every one of them has now been on the screen. The name became a number, and you watched the lookup. A connection was opened, and you watched the handshake. A request went out, and you read it line by line. A reply came back carrying a code, and you read that too. None of it is magic and none of it is hidden. It is four steps and a great deal of plain text, and one command will print that text for you. That is the internet, from the bottom up. Go and run the command on something you use every day, and read what comes back."}]}}