PEACEFUL systemd thread - GNU/Linux
Users browsing this thread:
|
|||
|
So I think most people in there would argues that systemd is a shitty software. It might be, but in most discussions I came across regarding this topic on IRC, I found myself to be in the position to defend systemd, because most people arguments are badly expressed, or simply false.
I would like to start a PEACEFUL and comprehensive discussion about systemd there, in order to help people construct a relatively well-though opinion. To do so, I think the way to go is to limit ourselves to technical arguments ONLY, and for each arguments against systemd, one one should provide at least one argument in favor of this feature. Because I tend to be pretty vague, and have difficulties to explain what I want, I'll start by one of the first thing that bother me with systemd: binary logging: systemd replaces syslog by translating kernel logs into its own format, that is saved in binary data, and accessible through a tool named 'journalctl'. Logging in binary form sucks for me, at it prevent you from investigating with standard tools, accessing logs through a single file sitting in /var/log. it means that you NEED a tool that can read these logs if you want to investigate them. Standard tools made to operate on text streams won't suffice. Binary data can also be corrupted easily. Add 1 byte padding, and the whole file might become unreadable (I hope they have a mchanism to prevent that, but again it add complexity over something that was pretty simple). Text files can't really get corrupted, as if you add garbage in the middle of a text file, your eyes can simply just ignore that part and continue reading after that. Hopefully, there is a way to forward journald logs to syslog, that will process them in text form, and write them back to files in /var/log. This is just not efficient and coherent at all, as the logs will go this way: Code: [kernel]---text--->[systemd]---binary--->[journald]---text--->|syslog]---text--->[/var/log]So much useless step and conversion, giving it even more possibility to corrupt data. The reasonning behind this decision might make sense though, as it was done to give the possibility to "structure" the logs, and sort them in an easier manner by the logger. Like, imagine the binary format written looks like that: Code: struct log_entry {For systemd, reading such a structure is easier, more efficient, and more reliable than parsing a text log entry. It provides a simple mechanism to search, order and classify logs, which could be real helpful in some situations. Binary data is also way easier to deal with in terms of code, making systemd "simple" in this case (not real code here, but that's what I'll do if I was Lennart): Code: f = fopen("binary.log", "a+");Same goes for reading... Pretty simple! Sadly, I doubt systemd would do it in such a simple, and lovely way, thus killing the whole advantage of doing so. |
|||
|
|||
|
-- <mort> choosing a terrible license just to be spiteful towards others is possibly the most tux0r thing I've ever seen |
|||
|
|||
(30-10-2016, 01:05 PM)jkl Wrote: So I think most people in there would argues that systemd is a shitty software. It might be, but in most discussions I came across regarding this topic on IRC, I found myself to be in the position to defend systemd, because most people arguments are badly expressed, or simply false. Shitty discussions on the internet? You must be kidding me. About the binary logging, does journalctl provide a simple method of sorting the logs after priority/date? |
|||
|
|||
(30-10-2016, 01:05 PM)jkl Wrote: http://blog.darknedgy.net/technology/2015/10/11/0/As usual, you always post the most awesome links. This was quite a long and technical read but one about things I didn't get anywhere else on the internet. Thanks! |
|||
|
|||
(30-10-2016, 05:17 PM)pranomostro Wrote: About the binary logging, does journalctl provide a simple method of sorting the logs after priority/date? Keep in mind that the structure I gave was just an example usage, I didn't actually read systemd code to get this. journalctl does however provide a way to retrieve logs after a certain date, or by priority, so I assumed these informations are present in the log binary format. |
|||
|
|||
(30-10-2016, 05:49 AM)z3bra Wrote: Logging in binary form sucks for me, at it prevent you from investigating with standard tools, accessing logs through a single file sitting in /var/log. it means that you NEED a tool that can read these logs if you want to investigate them. Standard tools made to operate on text streams won't suffice. Binary data can also be corrupted easily. I completely agree, there's too much that can go wrong with binary logging. I will grant that in the time that I used a setup with systemd, it gave me no problems with log corruption, and I found journalctl to be easy to use. However, my experience does not remove the fact that the possibility is there. My major beef with systemd is the homogenization of control it has over the system. It seems to me that Pöttering and company are looking for issues (whether true or perceived) and instead of contributing to existing tools, or creating tools to remedy these issues, they increase the scope of systemd to take over that problem. This creates the culture and attitude that systemd detractors rail against, in where the ever-growing tree of dependencies causes more software to require systemd in order to function properly. |
|||
|
|||
|
Seeing the beast systemd ease, there are probably multiple mechanism against log corruption, but still, the fact they *NEED* to be accessed through journalctl bother me. I've been using archlinux at work for 2 years now, and everytime something goes wrong, I'm finding myself googling things like "journalctl get sshd logs", or "journalctl increave verbosity". It's definitely not user friendly, or suitable when you don't know what to search.
When there is an issue on a server, you possibly don't know what's happenning. What I usually do (on centos), is skimming through dmesg, /var/log/messages, /var/log/secure and so on to try and discover an issue. With journalctl, it's just too complex... You have no idea *which* logs are displayed to you, so even if you don't see anything abnormal, you can't be sure you checked everything. On a side note: Anybody can argue about another "non-feature" of systemd? Binary logging is being discussed here but I'm pretty sure there are other topics grinding people's gears ;) |
|||
|
|||
(02-11-2016, 05:38 AM)z3bra Wrote: On a side note: Anybody can argue about another "non-feature" of systemd? Binary logging is being discussed here but I'm pretty sure there are other topics grinding people's gears ;) Well since you asked, this is more of an annoyance and gripe then a hard point: The recent addition of systemd-mount really grinds my gears. It goes back to my previous post where the scope creep of systemd really bothers me as it tries to take over more and more core utilities and functions. |
|||
|
|||
|
Well, this is definitely not a behavior I'd encourage, but from their point of view, the idea of controlling EVERYTHING that happens at boot time might look appealing, and they probably realise that they'll have to fight in order to force people to comply with their API. The simplest way they have to get around it is to rewrite everything themselve, so they can implement everything they want, however they want. This is a big task, but they seem to prefer quantity over quality, so this decision makes sense for them.
Now I don't think systemd-mount brings any new feature that would justify this... |
|||
|
|||
|
From my point of view systemd is cluttering the linux ecosystem with a near-second kernel, at least if it keeps growing as it is right now.
From a technical view, this is both bad for quality and for testing. The more you combine everything together to a big blop the less testing you can do against it and the more testing is done on the end-user. This is not a good decision,- to say it nicely. |
|||
|
|||
|
I imagined the shell was first written to manage and start the other commands of the system. But it seems that it was first for some kind of interactive use:
https://en.wikipedia.org/wiki/Unix_shell#Early_shells http://www.multicians.org/shell.html But even if it was not, the shell can still be considered as a monolithic C program made to boot the system by running external commands, configurable enough to cover all use cases user may encounter. It rely on external commands, and compose them. From this point of view it looks that systemd is replacing the shell, by calling external commands (with a 'd' at the end, like "journald") and composing them to make an init system. Is this really the shortcomings of popular shells that gave a rise to systemd? (03-11-2016, 01:45 PM)TheAnachron Wrote: near-second kernel Oh, overlapping with kernel features too, some sort of kernel-level shell, then? |
|||
|
|||
(03-11-2016, 03:11 PM)josuah Wrote: Is this really the shortcomings of popular shells that gave a rise to systemd? Interesting idea. About integration of commands: I think the reason traditional unix enthusiasts oppose systemD is that it does not want to integrate into the existing unix system. I have wanted to write a little blog post about this, but in short unix is not about making another unix (which systemD may very well be), but integrating into the existing unix by using the usual data transfer and transformation methods. |
|||
|
|||
|
After all this time, I managed to make my experience with systemd less "worse" than it was. At work, we start using centos 7 (irony, I'm in charge of doing that, and thus explain systemd to my colleagues), and I must say that things don't change that much (mostly because red hat worked hard so it does not).
After forcing myself to use it (on my local work machine as well), here are a few thoughs:
So overall, yeah, systemd works, and you will not notice it most of the time. You will have to learn a new system infrastructure though, that is, IMO, not well though or standardized (Even if you know how service configuration is done, you will still have to learn how networkd is configured, or resolved...) I think systemd is growing to fast to polish all ots rough edges, and even if it will grow stable enough to, one day, be the single dependency to build a fully fledged distro, it will be hard to manage. Free the init! |
|||
|
|||
(15-06-2017, 06:16 AM)z3bra Wrote: I'd argue that it's just different. I had the pleasure of showing someone fairly new to FreeBSD how to write a service file and that is just as hacky and full of arbitrary variables. The only thing I really don't like writing is systemd timers. I just want my cron back :( |
|||
|
|||
|
While I avoid systemd as much as possible, I will agree that it "Just Works" for basic operations. Hell, for a workstation distro it's probably the easiest choice as most of the time you don't have to touch anything.
The problems I have are usually server-level, with things like systemd-resolved and those awful binary logs. Though I'm slowly moving my servers and personal apps to OpenBSD or Docker containers, I still use CentOS for a few things. I'll admit that for me it's an ideological thing. Since my professional experience is in the Windows enterprise (kill me), I know the evils of bloated, all-encompassing systems. Systemd's progress of eating all the things (resolve, cron, mounting, etc) is bad for the Linux environment. |
|||
|
|||
|
I also work with windows based machines and am fully satisfied with it, but I digress (that could deserve a topic on its own).
To first answer azk, I consider freebsd, openbsd ans sysv to be too complex as well, as there is too much abstraction between what you write and what is actually done. I'd head more toward things like runit or daemontools, where you can guess how to configure things on your own, without bothering about how it works internally. When you got the following arborescence: Code: /servicesIt's easy to understand how to create a new service, as the content of "run" is simply Code: sshd -D -f /etc/ssh/sshd_configeasier than anything else really. No complex variable, no surprise on what will run, no hidden framework to source... Just a shell script, that you can run on its own to test it |
|||
|
|||
(15-06-2017, 02:35 PM)z3bra Wrote: easier than anything else really. No complex variable, no surprise on what will run, no hidden framework to source... Just a shell script, that you can run on its own to test it However there are two things you are missing:
|
|||
|
|||
(15-06-2017, 02:49 PM)venam Wrote:(15-06-2017, 02:35 PM)z3bra Wrote: easier than anything else really. No complex variable, no surprise on what will run, no hidden framework to source... Just a shell script, that you can run on its own to test it The init systems I mentioned are indeed not perfect (none of them is actually, which is why we're facing a war of inits). First of all, I don't think it's the init job to handle services. Its role is to bring up the system. What happens next is irrelevant to the "init". For services, you're right, today, only systemd can start a service when a special path is mounted, or a socket is created. But is it the right way to do it? In the end, is a mount point semantically different from a "network" service? IMO, they should all be treaten the same, using a mechanism similar to the need(8) idea (see https://web.archive.org/web/200104281211...ot-scripts) basically, your script could be this (note that mpd flags are random here, I don't know.them): Code: need network |
|||
|
|||
(15-06-2017, 03:36 PM)z3bra Wrote: First of all, I don't think it's the init job to handle services. Its role is to bring up the system. What happens next is irrelevant to the "init".And this is where the controversy is, there's no clear definition of the limits of the bootstraping process, the edges are hazy. You could have an init system that just barely gets you runing by giving you a tty or you could have an init system that practically toss you in a graphical environment all ready to go. The edges of the *initialization* are not clear all of that is based on personal opinions. Also, having the PID 1 makes it easier to handle services. |
|||
|
|||
Quote:having the PID 1 makes it easier to handle services. I disagree with you there. It is only said so because orphaned programs get adopted by PID 1. Proper service handling can be done by any process, as long as their children don't double fork themselves (which they shouldn't do on their own anyway). |
|||
|
|||
(15-06-2017, 04:02 PM)z3bra Wrote: I disagree with you there. It is only said so because orphaned programs get adopted by PID 1. Proper service handling can be done by any process, as long as their children don't double fork themselves (which they shouldn't do on their own anyway). Yes, it doesn't have to but it's easier to do that from PID1. In the case where it's not it could still fulfill the following:
Thus it's subject to signals sent to that group. And if the service manager dies (which would not happen with PID1) then all the services will become orphan. Those are not too big of an issue, you're right, it doesn't have to be PID 1. Here's a thread that discusses this a bit: https://unix.stackexchange.com/questions...472#197472 Another one that talks about orphaned process not necessarily being reparented to PID 1 and thus could be reparented to that service-manager https://unix.stackexchange.com/questions...361#177361 The discussion is going a bit off-rails, sorry for that. |
|||
|
|||
|
No problem, we're discussing about init systems, which systemd is.
The fact PID 1 cannot die is false. It is a process like any other which is subject to segfaults, oom and the likes. You statement that: Quote:if the service manager dies (which would not happen with PID1) then all the services will become orphan. is a bit irrelevant, because it COULD die. If it does, you'll have more problems than sshd not running though, but it could. If you service manager is badly written, and thus die, then moving all this logic to PID 1 won't fix the crashes. It will only weaken PID 1. As for daemons being orphaned, it shouldn't happen if your daemon manager exits correctly (eg, cleanup by killing all children or whatever). If it does though, then yes, you will have orphaned processes. But it is IMO better than a kernel panic! Regarding the environment, I agree that it should be reset by the service manager before firing up the service. As for the ID of the service manager, they are technically root:root by default (like PID 1), and each service is free to launch themselves as separate users to drop their privileges. IMO, there is no valid technical reason to keep service management in PID 1. Not even for orphaned processes, as you will adopt them without any context (it could as well be a simple user process, you can treat all of them as daemons). To get back to the topic, this is one of the first flaws systemd has regarding service management. The internal logic seems to be okay though, but it is not managed at the good level. |
|||
|
|||
(16-06-2017, 05:00 AM)z3bra Wrote: is a bit irrelevant, because it COULD die. If it does, you'll have more problems than sshd not running though, but it could. If you service manager is badly written, and thus die, then moving all this logic to PID 1 won't fix the crashes. It will only weaken PID 1.Indeed I agree. (16-06-2017, 05:00 AM)z3bra Wrote: IMO, there is no valid technical reason to keep service management in PID 1. Not even for orphaned processes, as you will adopt them without any context (it could as well be a simple user process, you can treat all of them as daemons).After further reading (the links in my previous comment), I have to agree that there were a lot of solutions and aspects I hadn't thought of before. Especially the fact that the parent of orphaned processes doesn't have to be PID1 and is implementation specific and that reaping zombies isn't a requirement of the init anymore. Init could be, as far as the service manager is concerned, only responsible to respawn it in case of failure. (16-06-2017, 05:00 AM)z3bra Wrote: To get back to the topic, this is one of the first flaws systemd has regarding service management. The internal logic seems to be okay though, but it is not managed at the good level.Then, according to you, it's a matter of separation of tasks and delegation through child-processes? |
|||
|
|||
|
Privilege/task separation is great indeed. If the service manager dies, your OS should keep working without a hitch. Neware not falling too deep in the delegation though, and spawning 3 processes per service like runit does for example
|
|||
|
|||
|
I'm unable to give any technical input regarding systemd as I don't know much about init systems. Here are a few links that might be useful:
http://without-systemd.org/wiki/index.ph...st_systemd https://suckless.org/sucks/systemd/ One thing I've noticed is how different distros handle systemd services. For example when installing mpd on Arch you will be required to enable the service manually whereas on Debian it will enable it automatically. I'm personally in favour of manually enabling/starting services but in Debian's defense at times it's very convenient. The problem with this for me is that it assumes you want the service running at startup, which is reasonable in most cases but not all. Another problem is that by default the program you just installed could've pulled in some other services such as avahi, and now they're also enabled. I guess that's not really a problem with systemd but how different distros incorporate it. Systemd unit files are stored in sub-folders, which is a nice way to organize things but it could confuse some users. For a user having service scripts in either /etc/rc.d, /etc/init.d, or /etc/sv is much more straightforward. |
|||