Using Directories as Configuration Fomats

Using Directories as Configuration Fomats - Printable Version
+- nixers (https://nixers.net)
+-- Forum: Development & Graphics (https://nixers.net/Forum-Development-Graphics)
+--- Forum: Programming On Unix (https://nixers.net/Forum-Programming-On-Unix)
+--- Thread: Using Directories as Configuration Fomats (/Thread-Using-Directories-as-Configuration-Fomats)

Using Directories as Configuration Fomats - pkal - 11-09-2019

I've recently wrote about the subject here, but I wanted to ask this community specifically if there were any thoughts about this. To recap, I think there's a value in using the directory hierarchy as a key-value hierarchy for program configurations. I realize that there are downsides, but I think that especially in a unix-setting it would be worth playing with the boundary where directories and and where files begin.

So what opinions do other people here have? Could this help improve the situation with configuration files or is just another case of XKCD #927?

RE: Using Directories as Configuration Fomats - budRich - 11-09-2019

Good luck sorting out the dotfiles standards on linux. Many projects doesn't even respect XDG_CONFIG_HOME. But if you use it for your own applications and you like it it's fine, I've seen worse (looking at you thunar).

I think many of us who are annoyed by the whole dotfile mess on linux, have some sort of symlink system, be it with stow or just a script (personally i have a list with relative paths to a bunch of configs in a git repo that are often renamed and the target path in column 2- , that i parse and symlink, create dirs and stuff in a script). I don't think this issue will ever be sorted out, and it's not that big of a deal, as long as it is DOTfiles/dirs and not normal visible dirs in ~. Sometimes i wish I had a single (+root) user system, with no /home/* directories at all with configs in /etc etc ;)

One disadvantage i see in your system, is comments. I kind of like to add chunks from the man page in some configs or just making sections in long files. Another is getting an overlook of the values, since one would need to read the files to see the values (if they aren't boolean).

Nice webpage and blog, subbed and saved.

RE: Using Directories as Configuration Fomats - Halfwit - 12-09-2019

There's a page on the Arch Wiki (https://wiki.archlinux.org/index.php/XDG_Base_Directory) outlining how to achieve a pretty clean home dir. I have a pretty clean setup, mostly everything lives in ~/local/[bin|cache|data|cfg|pkg|src|run]/, which are set by XDG_CONFIG stuff. Mostly I source it in my shell profile, leaving one dot (.zshenv) and the compulsory ones like .ssh/ which will never go away. (That's basically to say, it's possible to sort out dotfiles on Linux, I've done it.)

I've taken to using mostly the ndb file format from plan9 for configurations in programs I've written, which have a very nice feature for includes. Mostly I've been well served by them, The example shown at http://man.cat-v.org/9front/6/ndb accomplishes relatively the same as what you've accomplished here, so I can state that your approach would likely be really nice to use, and definitely it's more query-friendly to have something that is predictable for each program, rather than a bespoke offering for every fucking thing. But, yeah. xkcd standards.

RE: Using Directories as Configuration Fomats - pkal - 12-09-2019

(11-09-2019, 10:53 PM)budRich Wrote: Good luck sorting out the dotfiles standards on linux. Many projects doesn't even respect XDG_CONFIG_HOME.

Oh, I'm not even trying, but on that note, I was shown this tools a while ago, that manipulates system calls, to redirect read-write requests from ~/.something to ~/.config/something

(11-09-2019, 10:53 PM)budRich Wrote: One disadvantage i see in your system, is comments.

I thought about this one, and there are two ways I could see it being solved. Either one takes the more complex path towards messing with extensions, and if "this/that/opt" is a key, then "this/that/opt.cmt" is the corresponding comment file, that would be respected by a editor, or oneself if one prefers to edit it manually. Or to avoid extensions, one could just choose "this/that/.opt", saying all hidden files are comments, regardless of their name.

(11-09-2019, 10:53 PM)budRich Wrote: Another is getting an overlook of the values, since one would need to read the files to see the values (if they aren't boolean).

Code:
find . -type f | xargs head

;^)

But you're right, that's why I suggested having explicit programs, that could either be something like sysctl or a more interactive tool like tig, aptitude or ranger. I'm planning to make one for emacs, to see how it could work.

(12-09-2019, 02:07 AM)Halfwit Wrote: I've taken to using mostly the ndb file format from plan9 for configurations in programs I've written, which have a very nice feature for includes. Mostly I've been well served by them

Did you implement it yourself, or is there some library for this?

RE: Using Directories as Configuration Fomats - budRich - 12-09-2019

(12-09-2019, 03:54 AM)zge Wrote: one could just choose "this/that/.opt", saying all hidden files are comments, regardless of their name.

That sounds great. And one could have this/.opt , with "global" notes. It is kind of a cool system, one benefit is also if one would like the same value for different keys you can create symlinks, especially nice for "theming".

Code:
head blue

-> #0000FF ,

Code:
ln -fs blue i3/bordercolor polybar/backgroundcolor

...

Halfwit Wrote:and the compulsory ones like .ssh/ which will never go away.

yeah and .bashrc and .inputrc and .profile and .vimrc and .xinitrc and .moonchild productions/ whatever you know. I know about that (excellent) Arch wikipage, but have never really bothered setting since it will not be 100%, and then even if it got setup, we install some random nasty program and one needs to constantly get annoyed by this. So i chose to have my own directory structured the way i want, and then symlink them to their stupid default locations and always have hidden icons hidden in thunar.

I know i once saw i guy who had made a script that was running as a cronjob, and removing all new files-(whitelist) from ~ .most of these stupid files contain the default settings and if removed it will not effect the program, other then it will always create a new file with default settings that we never change anyways.

RE: Using Directories as Configuration Fomats - z3bra - 12-09-2019

I like the idea! and you're certainly not the first one to think about it. The best example I can find is the linux kernel itself that can be tweaked dynamically from userland using a filesystem.
There is also the WMII window manager that work this way iirc.
On the paper, it is great.

In practice though...

Typing words in a file is easier than creating a whole tree, ir navigate it.
Sharing snippets online is also much easier than tarballs of your conf.
It is also worse performance wise, as you get much more overhead due to high IO.
The one main advantage I see though (and a huge one!) us dynamic reload on specific settings by monitoring the filesystem, which you can't easily do with a single conf file.

RE: Using Directories as Configuration Fomats - Halfwit - 13-09-2019

> Did you implement it yourself, or is there some library for this?
I'm using a Go lib for it, https://github.com/mischief/ndb

Also worth noting that practically speaking most everything on plan9 is controlled via files, as are some other offerings in the wild.
The biggest win with that is you can audit the state with trivial means (a `cat` here or there, even). The biggest burden is that in some cases, the nomenclature is intractable. (The /net stack uses numerical fids, and good luck knowing which is which process without peeking). The whole ndb system (network database) is queriable and configurable at runtime, and the files update to reflect the state. It's quite nice, you can simply upload the file in question for support as it stands, in the current _running_ state. But, again, it can be intractable and overly hard to dial in on the specific setting you need, and while there are man pages and comments and everything to aid with discovery, sometimes you yearn for just a big, fat config file of everything you can grep.

RE: Using Directories as Configuration Fomats - pkal - 13-09-2019

(12-09-2019, 07:28 PM)z3bra Wrote: I like the idea! and you're certainly not the first one to think about it. The best example I can find is the linux kernel itself that can be tweaked dynamically from userland using a filesystem.

Yeah, that was one thing that I was thinking about when writing about this, which is why I say

Quote:Hence the main difference here is that we are not working with virtual file systems, but just regular/any file system.

(12-09-2019, 07:28 PM)z3bra Wrote: There is also the WMII window manager that work this way iirc.

iirc, wmii is a remake of rio, right? So it's also the same idea of projecting the internal state of the program onto the file system.

(12-09-2019, 07:28 PM)z3bra Wrote: Sharing snippets online is also much easier than tarballs of your conf.

I agree, although I think that sharing tarballs would be the wrong approach. One could have a program that serializes a configuration in a human readable format, to share with others. Would be limited by binary data though.

(12-09-2019, 07:28 PM)z3bra Wrote: The one main advantage I see though (and a huge one!) us dynamic reload on specific settings by monitoring the filesystem, which you can't easily do with a single conf file.

I haven't thought of this, good point. I will add it to the text.

RE: Using Directories as Configuration Fomats - josuah - 12-01-2020

The OpenBSD approach is to have one language per configuration format, using a common lexer (.c code written by hand but similar across programs, splitting keywords and special characters), but specific parser (using yacc).

The advantage is to have a single file, with macro (var = "xxx"), include support, and a syntax efficient for the actual format to read. OTOH, this requires YACC.

The DJB Way [1] is to have multiple files, closer to what you zge seems to be doing, but each file still a "table", that containt the bulk of the data (like in http://cr.yp.to/djbdns/tinydns-data.html), or individual (like in the various qmail-* programs, each reading a subset of /var/qmail/control/* files).

Then, there are more formal syntax, like .ini, .json, .toml, .yaml (;_;), .hcl... That are built like a tree, or programing languages (.h header file to recompile, external conf.lua for project in C, .py for .py project...).

I had this idea that a configuration built like a tree could continue the filesystem tree:

If /etc/myprog/test.json is a file, /etc/myprog/server/runners/memory would open that file then go to ."server"."runner"."memory" within json (jq syntax).
If /etc/myprog/test/server is a file, /etc/myprog/server/runners/memory would open that file then go to ."runner"."memory" with json (jq syntax).

RE: Using Directories as Configuration Fomats - josuah - 12-01-2020

Regarding XKCD 927 (that should be a word in the dictionnary), I use the include mechanism to split configuration files, and do not fear to generate configuration files whenever I want to write "echo example.org >/etc/myprog/server".

For instance my /etc/mail/smtpd.conf for my indicus.z0.is node:

name = indicus.z0.is
include "/etc/mail/smtpd.common.conf"
include "/etc/mail/smtpd.mailer.conf"

If I had more than 2 VPS, I could generate this file. Making use of the various existing configuration file mechanisms permits to make the situation really not too bad, instead of piling layers on top of others.

Adding layers is how you end up in atrocities like https://manpages.debian.org/jessie/exim4-config/update-exim4.conf.8.en.html

OTOH, when program entirely lack a layer of "configuration format", let's be free to add one!

Qemu for instance, full command-line. Or pure-ftpd: a filesystem-based format just like the one you described (on debian)!