There is too much to do

I just finished reading Drug Cartels Do Not Exist by Oswaldo Zavala, which has to be one of the best books I’ve read in years. It’s about the creation of the “narco” and drug cartels, not as things that exist in their own right, but as part of state policy. Some of this is old news if you’ve read Graeber’s stuff or things about the Middle East, where structures and institutions that we take for granted, like bureaucracies, sectarianism, and narcos, are so often completely swallowed up and become creations of the state^.

Meanwhile, I’m knee-deep in a network murder mystery. One of my datacenters is plagued by more timeouts than a sporadic internet connection. I’m also looking at issues with the Erlang scheduler, where the rise of 256-core CPUs like Bergamo is causing problems with scheduler stealing algorithms. Tokio and Go seem to have “solved” this by having multiple workers steal half of a queue instead of one task at a time, but this still feels like a no-brainer, especially when Zen 5 chips come online with performance and efficiency cores. Maybe the scheduler stealing model is fundamentally bad, but that just increases the amount of space I have to search for a solution I like.

I’m also obsessed with this Linux-is-dead idea, where the combination of the warehouse-scale computer has forced us to partition the pieces we traditionally thought of as operating systems into different ones, network attached storage being the most obvious one. Everything has grown faster, while x86 CPUs ^{have largely stagnated. Datacenter software deals with cooling, power, and networking, the three components that now constrain our computing. Linux is bypassed at most levels, most serious applications do some form of kernel bypass on the network, while others want to talk directly to the block device (Ceph, Crucible, etc.). If it is not bypassed, Linux has invited huge elephants into the room, such as eBPF, where the giant LLVM monster now sits on our couches. None of these things are necessarily bad, but where does this leave Linux, if not a thin shim? And if there is a thin shim, can we return to the days of tiny, dare I say, “micro” kernels?}

This leads to my dilemma: there are simply too many things to do. Our towers of abstraction have become so high that exploring any level is dizzying. To solve my network problem, I have to triage the operating system, the rack, the suite, the network planes I manage. To understand the little piece of Iraq I care about, I have to understand how we produce knowledge about Iraq, which means I have to understand how we produce knowledge about failed states in general. To see how people understand disappearances and the forms of mourning, I have to see how people responded to state-enforced disappearances in Uruguay and Argentina. Each step forward forces me to tap into a raging river of knowledge and study that others contribute to, which then creates this feeling of vertigo.

I have come to believe that software is inextricably linked to the people who develop it. Cantrill’s statement about software as an oral tradition is not only correct, it is perhaps too weak a claim. I will tentatively advance the idea that software is not just an oral tradition, but a set of rituals that force you to think about the world in a specific way. Good software engineers begin to tackle problems by building abstractions, almost to the point where they have difficulty seeing any other way of solving problems. Our rituals become our practices, which only then start the flywheel of understanding. We as software engineers strive to develop a mechanical sympathy, to connect ourselves to a near-community with the machine. What purpose do rituals serve in religion if not to be repeated, which, once they become self-evident, binds us to a specific way of seeing the world?

I get it – saying that “software is like religion” sounds like one of those half-baked “everything is everything” ideas. But hear me out. There’s something to it that we shouldn’t understand software as a specific set of practices, but rather as a tradition. Just as Zavala’s book reveals how “narcos” are constructs of state policy rather than independent entities, our software practices and rituals are not just technical choices, but constructs of our techculture. Studying specific practices, like programming languages, models, etc., is more like studying a specific type of prayer, but none of these rituals and practices exist in isolation. They exist in repetition, in tension with all the other rituals, which can only be described as a tradition.

And that brings us back to the fact that there is too much to do. I don’t think I’ll come to a satisfying conclusion, but I’m going to take a stab at sticking my head in the pool of knowledge and see what I can come up with.