One Protocol to Rule Them All

HTTP protocol seems to have replaced almost all other popular (at one time) protocols. When the web got CGI, it got powerful, and it has started killing other protocols.

In the last 33 years, we have seen an unprecedented consolidation of standards that's almost the opposite of what happens in the tech world otherwise.

We have forgotten the early competitors to HTTP (Gopher, HyperCard, etc.), but HTTP protocol has spread its web all over and has encroached the space other standards had carved out.

Here is a non-comprehensive table to refresh your memory:

Why is HTTP winning? I considered five different reasons:

1. Binary vs. Text protocols
2. Persistent connections
3. Generic vs. specific protocol
4. Programmability
5. Web mafia

Binary vs. Text

I initially thought that the primary reason HTTP has done so well was because it's a text protocol; has managed to replace binary protocols.

Binary protocols are challenging to understand, implement and debug. I worked with SMPP, SNMP, CIMD, etc. We have mostly found HTTP APIs to do what these protocols did.

I also saw a move from SS7 ISUP to SIP standard for calls even though SS7 was moving to the IP layer with SIGTRAN.

But then I saw that HTTP APIs were rapidly replacing even simpler text protocols like POP. Moreover, HTTP itself is now a binary protocol with HTTP/2. GRPC over TCP is also an example of a binary protocol that's becoming popular.

Persistent connections

The next suspect in my mind was the requirement of persistent connections. HTTP 1.0 famously wanted the connection to be closed after the server responded. Not keeping connections open made it very easy for utilities like curl to become very useful.

It's notoriously difficult to maintain persistent connections, and that may have played a role in HTTP becoming more popular than POP, FTP, or MQTT.

Every time someone stops a page from loading, I think about how the developer writing that server would have ignored SIGPIPE and how they would have discovered the need.

This theory had legs, but we are also replacing UDP-based protocols (that don't even make any connections) with equivalent HTTP solutions. HTTP 1.1 onwards persistent connections became a popular performance measure for browsers, so the persistent connection theory was incomplete. There is more to the story.

Generic vs. specific protocol

I realized how HTTP is a generic protocol for transporting and interlinking information is similar to Intel's X86 general-purpose CPUs and modern general-purpose computers. Specially designed chips and computers exist, but many have been replaced with general-purpose microprocessors.

We know that some specific protocols like SMTP are still thriving (for now), but we look for HTTP-based solutions first for most scenarios.

Programmability

When I said CGI gave web superpowers, it directly benefitted HTTP. No other protocol delivers dynamically rendered content/behavior. Most of them try to transfer specific data types based on particular client requests.

With its headers and a small set of verbs (GET/POST, etc.), HTTP encouraged programmability. This is why HTTP (or web) is now used for emulating what other protocols were trying to do. Calendaring, video streaming, or file transfer; everything is possible on the web.

Web Mafia

Have you heard of Paypal Mafia? This fantastic group of entrepreneurs trusted and supported each other and built some of the most well-known silicon valley startups. HTTP had access to similar friends. Applications like browsers have become so good that they are now replacing operating systems. HTTP was a direct beneficiary of browser popularity.

HTML, CSS, JavaScript, and HTTP are all part of the web mafia that has evolved continuously and has gained popularity.

The following table documents the nature of protocols, and we can see how a combination of these five reasons made HTTP rule the world:

Learnings

We can learn a lot from the success of HTTP. Here are some things I learned:

Start Simple - HTTP started as a simple protocol and became complex only as usage and popularity increased. Many of us make complex products without confirming a product-market fit. I believe this is true for both products and protocols. There is something to learn from HTTP.

Keep evolving - HTTP 1.0 and HTTP 3 look entirely different. The same should be valid for our products and startups. Keep building on your first version, grow your product in iterations.

Diversify - Start with one product, become good at solving one problem, and then look at adjacent services and products you can offer.