--- /dev/null
+<!--#include virtual="/doctype.txt"-->
+<html>
+
+<!--#set var="pagetitle" value="Why I dislike systemd"-->
+<!--#include virtual="/head.shtml"-->
+
+<body>
+
+<!--#include virtual="/header.shtml"-->
+
+<!--#include virtual="/navigation.shtml"-->
+
+<div id="content">
+
+<h1>Why I dislike systemd</h1>
+
+<p>
+As a Linux sysadmin in the 2010s, it's hard not to have an opinion on
+systemd. But what I find baffling about it is how divisive it is;
+nearly everyone (or at least the most vocal crowd) seems to either love
+it or hate it. When I tell people that systemd was the catalyst for my
+defection to OpenBSD last year, their usual reaction is to assume that
+I am part of the "hate it" group. Nope.
+</p>
+
+<p>
+In truth, systemd itself was a very small part of the reason I jumped
+ship. Its introduction made me realise two important things. First, the
+design problems with modern Linux run deeper than any one piece of
+software, I just hadn't noticed until I had a fresh one to learn.
+Second, and this is specific to Debian, the "universal operating
+system" mantra is fundamentally flawed; you cannot support all use
+cases when confronted with two alternatives that each function best
+when adopted to the exclusion of all others.
+</p>
+
+<p>
+But you didn't come here to read my life story. Back to systemd.
+</p>
+
+<p>
+Let me be clear. systemd is not the work of the devil, it probably
+doesn't contain any NSA backdoors, and it isn't the worst piece of
+software the big Linux distributions are shipping by default. It does
+address some real problems with Linux, and its approach does have its
+merits.
+</p>
+
+<p>
+It's often said that a half-truth is worse than a lie. In the case of
+systemd, a half-improvement is worse than a complete flop. People are
+focusing on the features they want to use and ignoring the ones that
+will come back to bite them later.
+</p>
+
+<p>
+No, my biggest problem with systemd is that it is repeating the
+mistakes of System V all over again. It's just given them a change of
+clothes to appeal to the 21st century.
+</p>
+
+<h2>A bit of history</h2>
+
+<p>
+In order to understand how we got here, we need to cast our minds (or,
+for people under the age of 50 like myself, our imaginations) back to
+the mid-1960s. Time-sharing operating systems were a new and innovative
+concept, and many people still used batch processing as the established
+method for getting things done. Many different organisations were
+experimenting with many different approaches to time-sharing. One of
+these was Multics.
+</p>
+
+<p>
+Multics was the conceptual ancestor of all UNIX and UNIX-like systems.
+The problems that made Multics a failure were not its core design
+principles, but the design and implementation of its parts. This is
+much more effectively communicated using a concrete example, from <a
+href="http://www.utterpower.com/10132011-dennis-ritchie-bell-labs-rip-c-language-and-unix/">
+a paper</a> by Dennis Ritchie first presented in 1979.
+</p>
+
+<blockquote
+cite="http://www.utterpower.com/10132011-dennis-ritchie-bell-labs-rip-c-language-and-unix/">
+<p>
+The very convenient notation for IO redirection, using the `>’ and
+`<‘ characters, was not present from the very beginning of the PDP-7
+Unix system, but it did appear quite early. Like much else in Unix, it
+was inspired by an idea from Multics. Multics has a rather general IO
+redirection mechanism embodying named IO streams that can be
+dynamically redirected to various devices, files, and even through
+special stream-processing modules. Even in the version of Multics we
+were familiar with a decade ago, there existed a command that switched
+subsequent output normally destined for the terminal to a file, and
+another command to reattach output to the terminal. Where under Unix
+one might say
+</p>
+
+<pre><code>ls >xx</pre></code>
+
+<p>
+to get a listing of the names of one’s files in xx, on Multics the
+notation was
+</p>
+
+<pre><code>iocall attach user_output file xx
+list
+iocall attach user_output syn user_i/o</code></pre>
+
+<p>
+Even though this very clumsy sequence was used often during the Multics
+days, and would have been utterly straightforward to integrate into the
+Multics shell, the idea did not occur to us or anyone else at the time.
+I speculate that the reason it did not was the sheer size of the
+Multics project: the implementors of the IO system were at Bell Labs in
+Murray Hill, while the shell was done at MIT. We didn’t consider making
+changes to the shell (it was their program); correspondingly, the
+keepers of the shell may not even have known of the usefulness, albeit
+clumsiness, of iocall. (The 1969 Multics manual lists iocall as an
+`author-maintained,’ that is non-standard, command.) Because both the
+Unix IO system and its shell were under the exclusive control of
+Thompson, when the right idea finally surfaced, it was a matter of an
+hour or so to implement it.
+</p>
+</blockquote>
+
+<p>
+What we have here is, fundamentally, a trade-off. Ritchie's speculation
+on the reason for the trade-off aside, Multics tried to be everything
+to everyone by providing the most general interface possible. As a
+result, it was inefficient to develop and cumbersome to use. It got the
+job done by making the 99% majority of use cases more difficult in
+order to make the 1% minority simpler. This is a design pattern I think
+is being repeated in systemd.
+</p>
+
+<p>
+Most importantly for the point I'm making, an interface like iocall is
+so obviously wrong to us <strong>because we have seen what a better
+interface looks like</strong>. To the designers of Multics, it was a
+perfectly reasonable approach to solving a very real problem. That is
+not the same thing as being a good solution.
+</p>
+
+<h2>The commercialisation of UNIX</h2>
+
+<p>
+Fast-forward to the 1980s. AT&T wanted to commercialise UNIX, ended
+the practice of distributing source code along with their binaries, and
+began implementing features they thought would make their product sell.
+The culmination of these efforts is the System V family of operating
+systems, of which Linux is a member (albeit as a clone, not a direct
+descendant). For the purposes of this discussion, I'll be referring to
+System III as a member of the System V family, as the distinction isn't
+too relevant.
+</p>
+
+<p>
+One of the things System V inherited from Research UNIX was an init
+that provided inflexible process monitoring, and invoked an
+ever-growing shell script (/etc/rc) to perform service startup. Its
+developers decided that the best way to tackle this problem was to
+invent a new configuration language which would enable init to perform
+more complex and customisable tasks, along with a way of allowing
+vendors to drop in their own service startup routines and have them run
+automatically at boot.
+</p>
+
+<p>
+Sound familiar? It should. The fact that System V's init failed so
+spectacularly at scaling to the problems of today, while modern BSD
+systems continue to manage just fine by way of incremental improvements
+to the pre-existing components, should serve as a lesson in itself.
+</p>
+
+<h2>Introducing systemd</h2>
+
+<p>
+Okay, enough with the history lesson already. You came here to read a
+rant. So here it is.
+</p>
+
+<h3>It shuffles complexity around</h3>
+
+<p>
+A common point raised in favour of systemd is that its unit files are
+"simpler" than init scripts. Superficially, this is true. What's really
+going on, though, is that the complexity has been moved from the logic
+into the language.
+</p>
+
+<p>
+In stark contrast to Bourne shell, so many of systemd's unit directives
+are highly specialised building blocks which aren't easily reusable.
+This isn't necessarily a bad thing, except that because systemd units
+aren't real programs, there's no way to escape to more basic primitives
+as required. Inevitably, what this means is that more and more
+directives will be added over time as more and more use cases are
+discovered.
+</p>
+
+<p>
+Don't believe me? Wait until 2025, and if the number of unit directives
+hasn't at least tripled compared to today (and people aren't working
+around their absence by invoking wrapper scripts from ExecStart), I'll
+buy you a drink. Hint: At least one Debian package (memcached) is
+already invoking a wrapper script from ExecStart.
+</p>
+
+<p>
+If you start with complex logic in a simple language, you can always
+simplify your implementation. If you start with simple logic in a
+complex language, you're stuck with that complexity as soon as anyone
+else starts using it.
+</p>
+
+<h3>It babysits its users</h3>
+
+<p>
+This complaint is far from unique to systemd, but is pandemic to a
+disturbingly increasing number of programs, especially on Linux. Why is
+<tt>systemctl edit</tt> even a thing?
+</p>
+
+<p>
+No, you're just a <strong>user</strong>. You couldn't possibly manage
+the job of invoking an editor on a configuration file without bricking
+your computer, so here's a wrapper that does it for you. Oh, you edited
+the file directly anyway? Here, let me helpfully <strong>tell you that
+you should run <tt>systemctl daemon-reload</tt>, but perform the action
+you requested using the old configuration anyway</strong>.
+</p>
+
+<p>
+Speaking of which, why is <tt>systemctl daemon-reload</tt> even a
+thing? When did filesystems become so uncool that not only are they
+demoted to second-class citizens beneath some other primary source of
+truth, but you don't even reload files when you <strong>know</strong>
+they've been changed?
+</p>
+
+<p>
+Now, I'm well aware that I'm giving an oversimplification of the issue
+here. There are deeper design issues that make <tt>systemctl edit</tt>
+quite different from opening an editor on a unit file, but my point is
+that <strong>the user doesn't care</strong>. All they see is an overly
+complex CLI that does little other than edit a file. However, this does
+make a convenient segue into my next point...
+</p>
+
+<h3>It takes the Multics approach to interoperability</h3>
+
+<p>
+One of the fundamental differences between Multics and UNIX is the way
+programs interact. Multics had programs that were developed
+independently, and while they were each workable solutions that could
+be made to complement each other when necessary, they were far from
+efficient at doing so. UNIX turned this on its head, with one of its
+most basic design principles to be to "write programs to work
+together".
+</p>
+
+<p>
+Now, systemd does rely heavily on D-Bus, and in doing so makes itself
+controllable by other applications. You might consider that "working
+together", but I see systemd as more akin to iocall than UNIX's shell
+redirection; it provides facilities other programs can use without
+integrating into their workflow.
+</p>
+
+<p>
+The previous example is an ideal illustration of this. Using an editor
+on unit files directly will work, but it's clumsy, and you need to
+remember to invoke <tt>daemon-reload</tt> after you're done. Yes, it
+works, but that's not the point. The point is that it's inefficient and
+counterintuitive to edit unit files except through systemd's own
+interfaces. That is not what I would call "working together".
+</p>
+
+<p>
+I'll repeat what I said before, because it's important: To the
+designers of Multics, it was a perfectly reasonable approach to solving
+a very real problem. That is not the same thing as being a good
+solution.
+</p>
+
+<h3>It's unpredictable</h3>
+
+<p>
+Log into a computer running systemd. Try to find the answer to "which
+units are going to be started on next boot?". This is not quite the
+same thing as "which units were started on this boot?", and because of
+the complex dependency relationships between units, it's actually a
+very difficult question to answer. This is bad if you're a
+security-conscious sysadmin.
+</p>
+
+<p>
+The failing of dependency-driven service initialisation is that it
+works far better in theory than it does in practice. In theory, you
+just have a directed graph of nodes, and you would like those depended
+upon by default.target to start. In practice, that's not how systems
+administration works.
+</p>
+
+<p>
+As a sysadmin, when I reboot a system, I want to be able to ask "is foo
+going to start up after a reboot?". I want to know, with reasonable
+confidence, what the state of my system is going to be when it comes
+onto the network. This is one area where sysvinit outperforms systemd
+immeasurably.
+</p>
+
+<p>
+And yes, you can avoid the whole issue just by masking any units you
+don't want to start, but that's missing the point. Again, the point I'm
+making isn't that it doesn't work, just that it doesn't work
+efficiently. Your tools should make your job easier, not get in your
+way.
+</p>
+
+<h3>Its priorities are warped</h3>
+
+<p>
+One of the "features" of systemd is that it allows you to boot a system
+without needing a shell at all. This seems like such a senseless
+manoeuvre that I can't help but think of it as a knee-jerk reaction to
+the perception of Too Much Shell in sysv init scripts.
+</p>
+
+<p>
+In exactly which universe is it reasonable to assume that you have a
+running D-Bus service (or kdbus) and a filesystem containing unit
+files, all the binaries they refer to, all the libraries they link
+against, and all the configuration files any of them reference, but
+that you lack that most ubiquitous of UNIX binaries, /bin/sh?
+</p>
+
+<p>
+The use case often cited for this is managing services inside a
+container. I don't see why the init on my desktop needs to be
+complicated and restricted for the sake of a feature used by a minority
+of people with specialised use cases. By all means, write a tool for
+bootstrapping containers that doesn't rely on a shell, but don't
+shoehorn that into a one-size-fits-all init.
+</p>
+
+<p>
+What makes this especially annoying, though, is that systemd also
+includes a dumbed-down shell-like parser to handle "EnvironmentFile"s
+(which usually don't actually set environment variables when sourced
+from a shell, but that's the way systemd's parser treats them). Also,
+service units have a pseudo-shell syntax for argument lists. One
+particularly bizarre feature that breaks expectations for users of
+pretty much any other software is that $FOO is word-split into multiple
+arguments, whereas ${FOO} isn't.
+</p>
+
+<p>
+Is avoiding ever having to execute a binary really so useful in the
+real world that these complications are justified?
+</p>
+
+<h2>Conclusions</h2>
+
+<p>
+I said it in my introduction, and I'll say it again: systemd's approach
+has its merits. But its shortcomings are representative of a philosophy
+that seems to be in fashion at the moment on Linux: ignoring history
+and assuming we can do it better now, instead of familiarising
+ourselves with historical mistakes and learning from them.
+</p>
+
+<p>
+There is no one thing about it that makes it bad software, but taken as
+a whole, I believe it creates more problems than it solves. Sadly, many
+of those problems will take years to become apparent, by which time all
+the major Linux distributions will have fully integrated it. That mess
+will be left up to some poor soul in the 2040s to deal with, who will
+hopefully learn from these mistakes and break the vicious cycle.
+</p>
+
+<p>
+While this article was focused on systemd, it is really just one
+example of an endemic trend on Linux, and across System V-family
+systems in general. It's easy to look at an existing tool, identify its
+deficiencies, write a replacement and label it as progress. It takes
+considerably more skill to look at an existing tool, identify its
+strong points, and modify it to work better. That is where Linux stands
+to learn a lot from BSD.
+</p>
+
+<p>
+If you think I'm full of shit, feel free to rant at me about it on
+Twitter or something, but don't expect me to care. I've made my choice
+to move to OpenBSD, and I couldn't care less whether systemd works well
+for others or not. I'm simply documenting my experiences because I feel
+there are important lessons to be learned which I'd rather not forget
+about, and maybe others would like to read as well.
+</p>
+
+</div>
+
+<!--#include virtual="/footer.shtml"-->
+
+</body>
+
+</html>
projects / www.git / commitdiff