Web Server

A web server is a program that answers

Strip away the racks and the cloud logos and a web server is one program doing one thing in a loop: wait for someone to connect, read what they asked for, send text back, and go back to waiting.

The word server means two things and people mix them up constantly. There is the machine sitting in a data centre, and there is the program listening on a port. Here we mean the program — and it runs just as happily on the laptop in front of you.

One visit, four steps

1. ListenThe program claims a port — 8084 in our example — and waits. A port is just a numbered door on the machine, so two programs cannot hold the same one.
2. Read the requestA browser connects and sends a few lines of text. The first line says which method and which path it wants, for example a GET of the path /.
3. DecideThe server works out what to send: read a file from disk, run some code, query a database. Our example always sends the same file, which is the only reason it fits in twenty lines.
4. Respond and closeIt writes a status line, a few headers, one blank line and the content, then closes the connection and loops round for the next visitor.

A web server in one shell script

Here is a complete web server written in bash. It opens a port, reads the request line, logs it, and sends index.html back as a proper HTTP response. Twenty lines, no libraries, and a browser cannot tell the difference.

Two commands do the heavy lifting. netcat (nc) speaks the network, and mkfifo makes a named pipe so the answer we write can be fed back into the connection nc is holding open.

#!/bin/bash

PORT=8084
PIPE=simple-web-server-pipe

rm -f "$PIPE"
mkfifo "$PIPE"

response() {
    read -r method path _
    echo "[$(date)] $method $path" >&2
    index=$(cat index.html)
    printf 'HTTP/1.1 200 OK\r\nContent-Type: text/html; charset=utf-8\r\nConnection: close\r\n\r\n%s' \
      "$index"
}

while true; do
    response < "$PIPE" | nc -l "$PORT" > "$PIPE"
done

Line by line

1. PORT=8084The port to listen on. Anything above 1024 works without administrator rights, and 8084 is unlikely to collide with something you already run.
2. PIPE=simple-web-server-pipeThe name of the pipe file. It appears in the folder like an ordinary file, but nothing is ever stored in it — it only passes data between two commands.
3. mkfifo "$PIPE"Creates the named pipe (a FIFO). The line above deletes any leftover from a previous run, because mkfifo refuses to overwrite an existing file.
4. read -r method path _Reads the first line of the request and splits it into method, path and the rest. Everything after it — the browser's headers — is ignored by this server.
5. echo "[$(date)] $method $path" >&2Prints a log line. The redirection sends it to standard error so it appears on your terminal instead of being sent to the browser as part of the page.
6. index=$(cat index.html)Reads index.html from the current folder into a variable. This is why the server must be started from the folder that holds the file.
7. printf 'HTTP/1.1 200 OK\r\n…'Writes the response: a status line, the content type, a header saying the connection ends here, then a blank line and the page. The blank line is what separates headers from content, and HTTP demands carriage-return-and-newline on every one of them.
8. response < "$PIPE" | nc -l "$PORT" > "$PIPE"The trick that makes it work. nc reads our response from the pipe and writes the browser's request back into it, so the two halves feed each other.
9. while true; do … doneThe loop. nc handles one connection and exits, so without this the server would answer a single visitor and stop.

Run it yourself

Save the script as server.sh, put an index.html next to it, and start it. Leave that terminal running and use a second one to make requests.

# 1. create an index.html in the same folder
echo '<h1>Hello</h1>' > index.html

# 2. save the script above as server.sh and start it
bash server.sh

# 3. open it in a browser (or check from a second terminal)
open http://localhost:8084
curl http://localhost:8084

Watch the first terminal while you reload the browser: one log line appears per request. Stop the server with Control-C, and delete the pipe file afterwards if it is still there. On Linux the nc command may need an extra -p flag before the port; on macOS the version shipped with the system works as written.

What actually travels down the wire

Open the connection and you find plain text you could have typed yourself. This is the whole reason the shell server is possible — and the reason you can debug a web problem by reading, rather than guessing.

The request the browser sends

GET / HTTP/1.1
Host: localhost:8084
User-Agent: Mozilla/5.0
Accept: text/html

• The first line is the only part our server reads: the method (GET means fetch, POST means send something), the path, and the protocol version.
• Host says which site is wanted. One server can hold hundreds of domains on one address, and this header is how it tells them apart.
• The rest are the browser introducing itself: what software it is, what formats it can read, what languages it prefers. A server may use them or ignore them.

The response the server sends

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Connection: close

<h1>Hello</h1>

• The status line: the protocol version and a code saying how it went. 200 means the request succeeded.
• Content-Type tells the browser how to read what follows. Send HTML with the wrong type and the browser will show your tags as text instead of a page.
• The charset part is not optional in practice. Without it, any non-English text is likely to arrive as garbled characters.
• Then one blank line, and everything after it is the content. That blank line is the entire border between the headers and the page.

Status codes worth knowing

What this server cannot do

The example is honest teaching material and a poor service. Knowing exactly why it falls short is what a real server has to solve for you.

• One visitor at a time. nc handles a single connection, so while it is busy everyone else waits.
• It ignores the path. Ask for anything at all and you still get index.html — there is no routing and no second page.
• It sends everything as HTML. Images, stylesheets and downloads all need their own content type, and this server has exactly one.
• No HTTPS. Everything travels in the clear, so any network in between can read it or change it on the way.
• No safety at all. It never checks the size of a request, never times out, and would happily read a file path someone else chose if you extended it carelessly.

Caddy: a real web server that stays readable

Caddy is a modern web server in a single executable file. It is worth learning right after the shell example because it keeps the same feeling — a few readable lines and it is running — while solving everything the shell version cannot.

Its headline feature is automatic HTTPS. Give it a real domain name and it obtains a certificate, installs it, renews it before it expires, and redirects visitors from http to https. On other servers that is a chore you repeat every few months.

Installing it

# macOS
brew install caddy

# Ubuntu / Debian
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
sudo apt update && sudo apt install caddy

# Docker alone is enough
docker run -p 8080:80 -v "$PWD":/usr/share/caddy caddy

Serving a folder in one command

No configuration file needed. Go to the folder holding your index.html and run one command — this is the fastest way to show a page to someone on the same network.

# serve this folder's index.html on port 8080
caddy file-server --listen :8080

# add a directory listing for folders without an index.html
caddy file-server --listen :8080 --browse

The Caddyfile

For anything permanent you write a Caddyfile. It is a plain text file, and a whole site can be four lines long.

example.com {
	root * /var/www/html
	file_server
	encode gzip zstd
}

1. example.com {The site address. Write a real domain here and Caddy arranges the HTTPS certificate on its own; write localhost and it serves plain http for local work.
2. root * /var/www/htmlWhere the files live on disk. The star means it applies to every path of this site.
3. file_serverActually serve those files. Without this line Caddy answers the request with nothing.
4. encode gzip zstdCompress responses before sending them. One word, and pages arrive noticeably faster on slow connections.

Reverse proxy: passing the request on

Most sites are not files on disk; they are an application listening on some local port. A reverse proxy is a server that takes the public request and hands it to that application, then returns its answer. One line does it.

getpes.com {
	reverse_proxy pes-web:3000
}

dl.getpes.com {
	root * /srv/files
	file_server browse
}

This is also how one machine hosts many sites. Only one program may hold port 443, so Caddy holds it, reads which domain each visitor asked for, and forwards each one to a different application behind it.

The commands you need

Docker: Caddy in a box, then real sites on the internet

Docker packs a program together with everything it needs to run — libraries, settings, even a small operating system — into a single Docker image, and runs that image as a container. A container behaves the same on your laptop and on a rented server, so the old excuse "but it works on my machine" disappears.

Caddy is already a single file, so why bother? Because a real server soon runs more than one thing — a site, an app, a database. Docker gives each one its own box that you start, stop, update and delete with the same few commands, without installing anything else on the server or disturbing its neighbours.

Four words to know first

1. ImageA frozen package of a program and everything it needs. You download one (caddy:2) or build your own from a recipe file called a Dockerfile. It never changes while it runs.
2. ContainerAn image that is running. One image can start many containers, and deleting a container never touches the image — the next start is fresh.
3. Port mappingA container has its own ports that nobody outside can see. -p 8080:80 connects port 8080 of your computer to port 80 inside the container, so a browser can reach it.
4. VolumeThe files a container writes inside itself vanish when it is deleted. A volume, or one of your folders mounted into it, is storage that stays — your pages, your settings and Caddy's certificates live there.

Installing Docker

On a Mac or a Windows PC, install Docker Desktop from docker.com and start it once; after that the docker command works in the terminal. On a Linux server, Docker's official script installs everything in one line. Both check commands should answer before you go on.

# macOS · Windows: install Docker Desktop from docker.com and start it once

# Ubuntu / Debian server: Docker's official install script
curl -fsSL https://get.docker.com | sh

# check that both commands answer
docker --version
docker compose version

Caddy in one command

Go to a folder that holds an index.html and run the line below. The first time, Docker downloads the Caddy image; after that it starts in about a second. Open http://localhost:8080 and your page is there.

# serve this folder at http://localhost:8080 (Control-C stops it)
docker run --rm -p 8080:80 -v "$PWD":/usr/share/caddy caddy:2

1. docker runCreate a container from an image and start it.
2. --rmDelete the container as soon as it stops, so stopped containers do not pile up. The image stays, so the next run is just as fast.
3. -p 8080:80Connect port 8080 of your computer to port 80 inside the container, where Caddy listens. You choose the left number; the right one is fixed by the image.
4. -v "$PWD":/usr/share/caddyShow the current folder ($PWD) inside the container as /usr/share/caddy — the folder the official image serves by default. Edit a file here and the next reload shows the change.
5. caddy:2The image to run: the official Caddy image, any 2.x version. Without the :2 tag Docker takes latest, which may one day jump to a new major version under your feet.

Press Control-C to stop it. This is perfect for a quick look, but the command is long and forgotten the moment the terminal closes. For anything permanent you write it down in a file — which is the next project.

The commands you will use