Developing on the Web is Better Than Ever – and Not as Good as It Could Be

<Puts on my grumpy old developer hat:> Back in my day, we didn’t have “front end” and “back end” developers — it was just making things on the web!

I joke, a little, but there’s some truth to the observation. The web development world has gotten very complex — to this point where we no longer expect people to understand it holistically. We’ve built these tremendously powerful and terrifically numerous development tools, and often they’re used both frequently and casually with little regard to their cost. Projects get so big, so bloated, and hard to get into that we feel forced to divide ourselves into ever-smaller domains.

No longer does anyone simply type in a few HTML tags into edit.com and FTP it onto the internet. Now we transpile, pack, tree-shake, bundle, containerize, and generally use up all our laptop’s memory. The web feels like a tarpit where it’s no longer possible to “just whip something up.”

Everything is terrible! Back in my day, we didn’t need all that! We had simple tools! That was all we needed.

We didn’t need more than that. But we wanted more.

Let me take a step back:

On the “Pro” Side: Abstraction is a Barrier

The development tools, languages, and constructs we use are all intended to serve one goal: to let us accomplish sophisticated things without having to worry about how it’s done. For example, we use compilers or interpreters so that we can run high-level code. Instead of writing machine code or punch cards, we can simply write console.log(“hello world”).

This is powerful: we can write web apps in ECMAScript (of various versions,) CoffeeScript, Rust (really! It can compile into WebAssembly,) or TypeScript. And I’m sure, many others. We can automatically remove parts of the code that aren’t used to reduce bundle size.

Just a few years ago, I wouldn’t be able to justify investment in those things for many of my projects, but now we can just grab a tool and poof done!

On the “Con” Side: Abstraction is a Barrier

But this ability doesn’t come without its costs.

For example, there’s a frustration I commonly have nowadays with Docker. It feels like every time I use Docker to start some dependency, I immediately have to wonder “Okay, how do I configure this container?”

I’ll wonder: Is it really running? I can’t connect on port XYZ. Is Docker forwarding that port? Is that service in the container listening on its side of that port? Where even are the configuration files for the service inside the container? What Docker volumes are mapped? Where does the service output its logs and error messages?

I’ve used Docker enough now that I know that these are the questions to ask. But, if you’re not familiar with the tool and just ran docker-compose up and it didn’t work — in that case, you’re pretty well stuck.

I may have started with, “Hey let’s start this Ruby on Rails app,” but with tools like Docker that can shift to mostly-unrelated questions like “Where the heck does AcmeDatabase store its network configuration file?”

Docker tries to abstract away several concerns about running services: packaging, dependency management, distribution, execution. The problem is that it is a particularly leaky abstraction. While it makes it “easy” to launch a service, you still need to understand the service and its configuration and operation.

I feel like many of the web tools are like that. They provide a lot of power but don’t free you from having to understand the underlying concept, whereas I don’t have any need to worry about processor architecture, cache line size, register allocators, or anything like that.

Good or Not?

When I encounter developers that have been working — either professionally or as a hobbyist — since roughly the turn of the century, there’s a question I like to ask them. Which is easier: building things today with this flood of tooling, or building it back in the day?

The answer, pretty consistently, is: Nobody prefers the “good” old days.

This is, of course, an informal survey, but the consistency is illuminating. Some people do hedge their bets a bit and point out what things are better or worse. Some people also (rightly) point out that it’s a bit of a trick question.

When we dig in further, I usually hear something like this:

  • It’s easier to build sophisticated web apps nowadays.
  • But it was easier to learn and get started back then.

A Comparison of the Simple Things

Consider a simple static website:

Years ago, you’d type HTML (and maybe some early CSS) into an editor (perhaps Vim, Emacs, or at least as commonly, just Notepad.) You could use the FTP client that came with your OS install to upload it to the 10MB of web space you get from your ISP.

That’s not bad! There’s a selection of development tools there that one can grasp pretty easily.

Today, it doesn’t look much different. Notepad still exists, and you can still type HTML into it. You are probably more likely to use something like SFTP to upload it, but if you have a web host you can still do that. More static site hosting options are available today than ever before. Or, with a little bit more complexity, you could host it on S3 and have a level of scalability that the 90s had barely dreamt of. Consider that the C10K problem was once a very big deal and now… most developers just don’t have to care.

Or Complicated Things

The differences between then and now really become stark when you compare more ambitious projects.

It wasn’t too hard to get started writing PHP code, for example. You just needed regular HTML plus a special tag <!--?php?--> where you could execute dynamic things.

The LAMP stack) was and still is a pretty solid platform to build software on. Well-understood tools. But let’s not pretend it’s easy to install correctly if you’re unfamiliar with it. Installing from source is a nightmare. The transitive dependencies will keep you busy compiling things like image format libraries for days, and that’s before you’ve figured out MySQL permissions. Even if your Linux distribution has those built, there’s usually some amount of wrangling to get the installed MySQL to talk to PHP or for Apache to do virtual hosting correctly.

Today, there are other options besides LAMP, but you can still use it. And you can just get a Docker container, mount your source code as a volume, and run it. If you can figure out Docker. And for servers? Just push it to Heroku and use their database add-on.

Building So Much More

Docker and Heroku are distracting and intimidating, too. And people rightly point out that many projects don’t need the sort of power many of the modern tools provide.

But some do, and the bar for smooth user interactions is much much higher now. For example, MapQuest used to not support scrolling. Instead, it had buttons you’d click to get a map square to the north, south, east, or west. When Google Maps came out, I remember that many of us in the office were absolutely floored at how smoothly you could explore.

Webpack, TypeScript, Docker, CSS preprocessors, big frameworks, and the rest of it… these things make a big, hairy, hard-to-learn mess. And the things I’ve been building are the best they’ve ever been. Many modern products out in the world are incredibly ambitious (for example, Miro and Figma.) Incredibly, we can take for granted the tools that make this possible. Teams are out there today doing things I wouldn’t even have considered attempting 15 years ago.

The development experience absolutely needs improvement. The tools we have don’t work together well (try to put VS Code, TypeScript, Jest, and NodeJS all loading the same set of modules!) but I’m not going to throw out the baby with the bathwater. Even if the baby is ugly.

I’m eager to see where the web is in another decade. Oh, and get off my lawn.