<![CDATA[nixers - Programming On Unix]]> https://nixers.net/ Fri, 08 May 2026 00:29:31 +0000 MyBB <![CDATA[text coordinate systems]]> https://nixers.net/Thread-text-coordinate-systems Thu, 27 Jun 2024 18:13:44 +0000 ckester]]> https://nixers.net/Thread-text-coordinate-systems
https://zed.dev/blog/zed-decoded-text-co...te-systems

Much of this echoes Finseth's classic book on implementing emacs-style editors, but with updates to reflect our Unicoded world.

(Many people will avoid dealing with these issues by simply dropping in whatever text widget their favorite library provides, but I enjoy cooking from scratch.)]]>
https://zed.dev/blog/zed-decoded-text-co...te-systems

Much of this echoes Finseth's classic book on implementing emacs-style editors, but with updates to reflect our Unicoded world.

(Many people will avoid dealing with these issues by simply dropping in whatever text widget their favorite library provides, but I enjoy cooking from scratch.)]]> <![CDATA[[Assembly] Need a good resources to start]]> https://nixers.net/Thread-Assembly-Need-a-good-resources-to-start Fri, 25 Mar 2022 01:11:12 +0000 strawman2511]]> https://nixers.net/Thread-Assembly-Need-a-good-resources-to-start <![CDATA[Command-line interface on GUI programs.]]> https://nixers.net/Thread-Command-line-interface-on-GUI-programs Thu, 24 Jun 2021 23:12:01 +0000 seninha]]> https://nixers.net/Thread-Command-line-interface-on-GUI-programs I have found, however, an application that makes use of the standard input for the user to type commands: gnubg, GNU's backgammon game.

Here's a sample video. On the video, I first type "new game" to start a new game, then "roll" to roll the dice, "help" to get a dialog window listing known commands, and "move" to move the tokens. Then, I type ^D (Control+D) to send EOF and close the game.

Here's the video.

This command-based interface is not available when the user opens gnubg(1) from the application menu/start menu, only when the stdin is connected to a terminal.

However, the commands are fundamental to the game operations: menu entries and the icons on the toolbar are bound to commands (clicking in one of them has the same effect as entering a command on stdin). You can also run a game from a script either by pipeing it to the stdin; or by running gnubg(1) with the -c option; or from the menu bar on File/Open Commands.

I found that interface really interesting, because it can be extended and composed with other applications. The user can run gnubg(1) from a script and call dmenu to run commands from it and/or get a list of commands within dmenu (mimicking HUD on the Unity DE from Ubuntu).

MacOS' global menu bar can also be mimicked. We can have a global menu bar in which clicking on a option on the menu bar sends commands to the stdin of the process of the active window.

This kind of interface: a regular user interface that also interprets command is nothing new on TUI (aka roguelike interfaces): Vim and emacs have a command-line interpreter at the bottom of their windows. Ranger, lf and other file managers can also interpret commands while also being controlled by usual keybindings.

With this kind of interface we can bind commands to menu entries, icons, and hotkeys with separate programs like dmenu and sxhkd.

Something like a command-line interface is being implemented in QT5 applications for KDE. The user can enter commands in a dmenu-like prompt, those commands are interpreted by the toolkit, which will run the corresponding action for the application.

That kind of GUI interface, so unexplored, influenced me in projecting two different, but related applications: control and xfiles. Xfiles will be a GUI file manager that reads commands from the stdin. Control will have server and client processes. The server is a macOS-like bar with global menu that changes its menus based on the current active window. The client is a wrapper; calling "control xfiles" will open xfiles with its stdin bound to control, allowing the user to control xfiles (ie., send commands to its stdin) from the global menu bar. Control will know which process stdin corresponds to which window by means of startup notification.

Control and xfiles are still in my mind, and will be there for a long time, as I do not have time to work on them (my uni is returning next semester).

What do you think of a command-line interface (CLI) for GUI applications?
And what do you think in implementing CLI in GUI by means of the standard interface (as gnubg does)?
Are CLI and GUI unrelated or can they be used together?
What do you think of KDE way of implementing CLI in GUI by means of the QT toolkit?
What solution is better: including CLI in GUI via stdin or via toolkit?

I think that implementing CLI in GUI, while not caring about implementing menu bars, toolbars and keybindings in the application, is the UNIXy way of GUI interfaces: let the actual interface to be done by other programs (dmenu, sxhkd, macos-like global menu, etc), concentrate only on interpreting a little language and separate the interface from the engine.]]> I have found, however, an application that makes use of the standard input for the user to type commands: gnubg, GNU's backgammon game.

Here's a sample video. On the video, I first type "new game" to start a new game, then "roll" to roll the dice, "help" to get a dialog window listing known commands, and "move" to move the tokens. Then, I type ^D (Control+D) to send EOF and close the game.

Here's the video.

This command-based interface is not available when the user opens gnubg(1) from the application menu/start menu, only when the stdin is connected to a terminal.

However, the commands are fundamental to the game operations: menu entries and the icons on the toolbar are bound to commands (clicking in one of them has the same effect as entering a command on stdin). You can also run a game from a script either by pipeing it to the stdin; or by running gnubg(1) with the -c option; or from the menu bar on File/Open Commands.

I found that interface really interesting, because it can be extended and composed with other applications. The user can run gnubg(1) from a script and call dmenu to run commands from it and/or get a list of commands within dmenu (mimicking HUD on the Unity DE from Ubuntu).

MacOS' global menu bar can also be mimicked. We can have a global menu bar in which clicking on a option on the menu bar sends commands to the stdin of the process of the active window.

This kind of interface: a regular user interface that also interprets command is nothing new on TUI (aka roguelike interfaces): Vim and emacs have a command-line interpreter at the bottom of their windows. Ranger, lf and other file managers can also interpret commands while also being controlled by usual keybindings.

With this kind of interface we can bind commands to menu entries, icons, and hotkeys with separate programs like dmenu and sxhkd.

Something like a command-line interface is being implemented in QT5 applications for KDE. The user can enter commands in a dmenu-like prompt, those commands are interpreted by the toolkit, which will run the corresponding action for the application.

That kind of GUI interface, so unexplored, influenced me in projecting two different, but related applications: control and xfiles. Xfiles will be a GUI file manager that reads commands from the stdin. Control will have server and client processes. The server is a macOS-like bar with global menu that changes its menus based on the current active window. The client is a wrapper; calling "control xfiles" will open xfiles with its stdin bound to control, allowing the user to control xfiles (ie., send commands to its stdin) from the global menu bar. Control will know which process stdin corresponds to which window by means of startup notification.

Control and xfiles are still in my mind, and will be there for a long time, as I do not have time to work on them (my uni is returning next semester).

What do you think of a command-line interface (CLI) for GUI applications?
And what do you think in implementing CLI in GUI by means of the standard interface (as gnubg does)?
Are CLI and GUI unrelated or can they be used together?
What do you think of KDE way of implementing CLI in GUI by means of the QT toolkit?
What solution is better: including CLI in GUI via stdin or via toolkit?

I think that implementing CLI in GUI, while not caring about implementing menu bars, toolbars and keybindings in the application, is the UNIXy way of GUI interfaces: let the actual interface to be done by other programs (dmenu, sxhkd, macos-like global menu, etc), concentrate only on interpreting a little language and separate the interface from the engine.]]> <![CDATA[What is your coding style?]]> https://nixers.net/Thread-What-is-your-coding-style Tue, 25 May 2021 14:43:41 +0000 seninha]]> https://nixers.net/Thread-What-is-your-coding-style I have read Rob Pike's, suckless', some 9front developers', and NetBSD and OpenBSD coding styles.
Those documents inspired me to write down all the subjective practices I have in my mind into objective conventions. And thus I wrote my coding style.

What do you think of those coding styles?
Do they enforce good practice or are they too normative to be feasible?
Do you have a coding style you follow in your projects? If yes, why not writing it down?]]> I have read Rob Pike's, suckless', some 9front developers', and NetBSD and OpenBSD coding styles.
Those documents inspired me to write down all the subjective practices I have in my mind into objective conventions. And thus I wrote my coding style.

What do you think of those coding styles?
Do they enforce good practice or are they too normative to be feasible?
Do you have a coding style you follow in your projects? If yes, why not writing it down?]]> <![CDATA[POSIX Shell Programming Challenge]]> https://nixers.net/Thread-POSIX-Shell-Programming-Challenge Mon, 12 Apr 2021 13:36:27 +0000 mort]]> https://nixers.net/Thread-POSIX-Shell-Programming-Challenge
I thought it would be fun to have a small shell programming challenge on here. The challenge is to iterate over all the files in a directory and print their path, without looking at the internet for guidance. We will then try to come up with ways in which proposed solutions are incorrect.

Here are the criteria:
  • Iterate over all the files in the directory "test" (assume it exists) and print their names
  • Include hidden files, but exclude "." and ".."
  • All legal file names must be supported, so the only characters you may assume are not in a file name are "/" and the 0 byte
  • Don't be recursive, directories should be treated as files
  • The body of the loop must be in the same process as the rest of the script
  • You may only use POSIX shell syntax, no bashisms

Here's an example:

Code:
for f in test/*; do
    echo "$f"
done

And some possible critiques:
  • The solution only considers visible files, it wouldn't see a file prefixed with "."
  • The solution doesn't work if there are no (non-hidden) files in the directory; the loop will end up printing "test/*"

Looking forwards to seeing what solutions and non-solutions y'all come up with 🤓

EDIT: The idea behind the challenge is to end up with a shell loop over all the file names in a directory; printing the file names is just an example use case. Imagine replacing "echo" with some shell function or anything else. Also, it's not important whether the file name is prefixed with the path to the directory or not, because adding or removing a path prefix is trivial and not the point of the challenge.]]>
I thought it would be fun to have a small shell programming challenge on here. The challenge is to iterate over all the files in a directory and print their path, without looking at the internet for guidance. We will then try to come up with ways in which proposed solutions are incorrect.

Here are the criteria:
  • Iterate over all the files in the directory "test" (assume it exists) and print their names
  • Include hidden files, but exclude "." and ".."
  • All legal file names must be supported, so the only characters you may assume are not in a file name are "/" and the 0 byte
  • Don't be recursive, directories should be treated as files
  • The body of the loop must be in the same process as the rest of the script
  • You may only use POSIX shell syntax, no bashisms

Here's an example:

Code:
for f in test/*; do
    echo "$f"
done

And some possible critiques:
  • The solution only considers visible files, it wouldn't see a file prefixed with "."
  • The solution doesn't work if there are no (non-hidden) files in the directory; the loop will end up printing "test/*"

Looking forwards to seeing what solutions and non-solutions y'all come up with 🤓

EDIT: The idea behind the challenge is to end up with a shell loop over all the file names in a directory; printing the file names is just an example use case. Imagine replacing "echo" with some shell function or anything else. Also, it's not important whether the file name is prefixed with the path to the directory or not, because adding or removing a path prefix is trivial and not the point of the challenge.]]> <![CDATA[Favourite Scripting Languages]]> https://nixers.net/Thread-Favourite-Scripting-Languages Wed, 24 Mar 2021 09:40:11 +0000 sulami]]> https://nixers.net/Thread-Favourite-Scripting-Languages
The go-to language for this in the past has been shell scripts, but I keep getting annoyed by weird bashisms, the way it handles errors, the way it handles any kind of non-string data (numbers are already a pain, and don't get me started on associative arrays). Which means I now have a hodgepodge of languages, including Ruby, Racket, and Janet, (and more bash) with no clear idea of which to use when.

Ruby has the backtick syntax which allows running shell commands without a lot of hassle. Contrary to Python, where subprocess includes a lot of ceremony for basic tasks.

Janet is explicitly meant for scripting, but (1) I haven't really gotten into it, and (2) I have some doubts about the ecosystem and portability.

Racket has been quite nice, as it has a very fully featured standard library & package ecosystem, and is reasonably widely supported.

I have also looked at:

- Elvish: seems to be more focussed on being a rich interactive shell, and somewhat niche
- Python: see the subprocess note above
- Oil: looks promising, but not sure about stability yet
- Haskell with shebang: just too much in every way
- Common Lisp with Roswell: might be an okay option? Haven't really tried

Another consideration I have not explored yet is the ability to build a library of useful utility functions, i.e. a decent module system.

All of this is a long-winded way of saying:

What do you use for scripting, and why?]]>
The go-to language for this in the past has been shell scripts, but I keep getting annoyed by weird bashisms, the way it handles errors, the way it handles any kind of non-string data (numbers are already a pain, and don't get me started on associative arrays). Which means I now have a hodgepodge of languages, including Ruby, Racket, and Janet, (and more bash) with no clear idea of which to use when.

Ruby has the backtick syntax which allows running shell commands without a lot of hassle. Contrary to Python, where subprocess includes a lot of ceremony for basic tasks.

Janet is explicitly meant for scripting, but (1) I haven't really gotten into it, and (2) I have some doubts about the ecosystem and portability.

Racket has been quite nice, as it has a very fully featured standard library & package ecosystem, and is reasonably widely supported.

I have also looked at:

- Elvish: seems to be more focussed on being a rich interactive shell, and somewhat niche
- Python: see the subprocess note above
- Oil: looks promising, but not sure about stability yet
- Haskell with shebang: just too much in every way
- Common Lisp with Roswell: might be an okay option? Haven't really tried

Another consideration I have not explored yet is the ability to build a library of useful utility functions, i.e. a decent module system.

All of this is a long-winded way of saying:

What do you use for scripting, and why?]]> <![CDATA[ffmpeg script to convert files]]> https://nixers.net/Thread-ffmpeg-script-to-convert-files Mon, 15 Mar 2021 03:22:36 +0000 pfr]]> https://nixers.net/Thread-ffmpeg-script-to-convert-files
I found this on the web looking for answers

PHP Code:
<?php 

#!/bin/bash

for i in *.mkv; do

    ffmpeg -i "$i" -codec copy "${i%.*}.mp4"

done

So, my question is, how do I add other file types to the fist line so that if I have a folder which contains .mkv & .webm files then both file types will be converted to .mp4.

Lastly, as this will create newly converted files, how can I have all old files deleted within this same script?]]>
I found this on the web looking for answers

PHP Code:
<?php 

#!/bin/bash

for i in *.mkv; do

    ffmpeg -i "$i" -codec copy "${i%.*}.mp4"

done

So, my question is, how do I add other file types to the fist line so that if I have a folder which contains .mkv & .webm files then both file types will be converted to .mp4.

Lastly, as this will create newly converted files, how can I have all old files deleted within this same script?]]> <![CDATA[how many projects per repository?]]> https://nixers.net/Thread-how-many-projects-per-repository Tue, 23 Feb 2021 01:12:06 +0000 freem]]> https://nixers.net/Thread-how-many-projects-per-repository
I wanted to ask a question: what do you think about "mono-repos", aka: one repo for a bunch of projects.
One of the alternatives, which seems to me the more common solution, is to have one repo per project (say, "project-repo").
And the other alternative, which is also quite common, is to have projects divided into sub-projects: repos which depends on other repos, this mostly happens within big projects (say, "split-repo"?).

Personally, I've worked with mono-repos at work. It ended that, because of some peoples lack of skill, mono-repo was split, with repos depending on other repos (split-repo, since our codes had to work together anyway, and even shared code I wrote as libraries). Well, it was not really a sub-project division, but rather a per-dev division, since the unskilled one kept all his projects into a single repo (but at least could not pollute other's repos...).

I have also contributed to "project-repos", and am still contributing to some "split-repos", which are kind of "encouraged" by git's submodule (shitty) system.

My personal preference clearly goes to mono-repo, because it makes things easy as a dev: I have code I share between my projects. I put it in a subdirectory, each project have own directory. They even usually have their own branches (since I use git), that, when they have a changeset ready, gets rebased (so that I can keep a sane history) and then merged into master.
This makes things pretty easy from both my side (which I consider most important one, since I'm the one working on it) and on user's side which have to bother about less dependencies.
But, this approach also have disadvantages:
* it makes it hard to point others at a specific project, since everything is in the same repo,
* if a project grows too much, it can become quite heavy, which can become a problem depending on connnection,
* history is "polluted", as in, it will not only be the history of the single part you're interested in. I try to workaround that by using prefix in commits' title, and with git it can be solved by specifying the directory you want history of, but that's still not easy/practical,
* it requires discipline, in order to keep things organized. In my case, I forbid myself to touch multiple project's source-code in a single commit. Note that I don't consider the build system as code, to the contrary, I think it makes a lot of sense to have build system considered a project in it's own.

The "project-repo" approach, which is probably the most common one, requires less discipline, but require more user's intervention, since it is then required to clone all required repositories.

Both of those approaches have a big issue, though. If the project have some sub-parts with heavy files, like a game, there are technical issues when you try to clone or update the repo: bandwidth requirements, even if you don't intend to work on graphical, audio, or gameplay parts (usually called assets, but a recent discussion made me think it's a bad idea to call those assets, because that implies that the engine is worth nothing. Programmers are worth nothing...).
This problems is solved by the "split-repo" approach, but this approach usually also introduces difficulties when someone tries to work with the whole project.

So, which approach do you use?
Why?
In all situations?]]>
I wanted to ask a question: what do you think about "mono-repos", aka: one repo for a bunch of projects.
One of the alternatives, which seems to me the more common solution, is to have one repo per project (say, "project-repo").
And the other alternative, which is also quite common, is to have projects divided into sub-projects: repos which depends on other repos, this mostly happens within big projects (say, "split-repo"?).

Personally, I've worked with mono-repos at work. It ended that, because of some peoples lack of skill, mono-repo was split, with repos depending on other repos (split-repo, since our codes had to work together anyway, and even shared code I wrote as libraries). Well, it was not really a sub-project division, but rather a per-dev division, since the unskilled one kept all his projects into a single repo (but at least could not pollute other's repos...).

I have also contributed to "project-repos", and am still contributing to some "split-repos", which are kind of "encouraged" by git's submodule (shitty) system.

My personal preference clearly goes to mono-repo, because it makes things easy as a dev: I have code I share between my projects. I put it in a subdirectory, each project have own directory. They even usually have their own branches (since I use git), that, when they have a changeset ready, gets rebased (so that I can keep a sane history) and then merged into master.
This makes things pretty easy from both my side (which I consider most important one, since I'm the one working on it) and on user's side which have to bother about less dependencies.
But, this approach also have disadvantages:
* it makes it hard to point others at a specific project, since everything is in the same repo,
* if a project grows too much, it can become quite heavy, which can become a problem depending on connnection,
* history is "polluted", as in, it will not only be the history of the single part you're interested in. I try to workaround that by using prefix in commits' title, and with git it can be solved by specifying the directory you want history of, but that's still not easy/practical,
* it requires discipline, in order to keep things organized. In my case, I forbid myself to touch multiple project's source-code in a single commit. Note that I don't consider the build system as code, to the contrary, I think it makes a lot of sense to have build system considered a project in it's own.

The "project-repo" approach, which is probably the most common one, requires less discipline, but require more user's intervention, since it is then required to clone all required repositories.

Both of those approaches have a big issue, though. If the project have some sub-parts with heavy files, like a game, there are technical issues when you try to clone or update the repo: bandwidth requirements, even if you don't intend to work on graphical, audio, or gameplay parts (usually called assets, but a recent discussion made me think it's a bad idea to call those assets, because that implies that the engine is worth nothing. Programmers are worth nothing...).
This problems is solved by the "split-repo" approach, but this approach usually also introduces difficulties when someone tries to work with the whole project.

So, which approach do you use?
Why?
In all situations?]]> <![CDATA[Why is modular code so hard to implement?]]> https://nixers.net/Thread-Why-is-modular-code-so-hard-to-implement Mon, 09 Nov 2020 05:51:24 +0000 jkl]]> https://nixers.net/Thread-Why-is-modular-code-so-hard-to-implement YouTube downloader recently and I made an observation: While it is rather easy to dynamically load libraries during runtime, writing statically linked software that makes it easy to add new “libraries” (classes) is rather complicated. There seems to be no obvious way in most programming languages to automatically embed all files in a certain directory.

My original plan was to simulate the default behavior of Go: Adding a new site to the list of supported sites would only require to add one file, the compiler would pick it up automatically. In Rust (yes, I am a software masochist, why do you ask?), this won’t work: I’ll still need to specify the new module in another file. (Arguably, this is already a relatively easy approach, but it took me a while to figure it out.)

So I wonder why most languages don’t have Go’s “plug-in” compilation system that wouldn’t require additional files. Is this really an unusual use case?]]> YouTube downloader recently and I made an observation: While it is rather easy to dynamically load libraries during runtime, writing statically linked software that makes it easy to add new “libraries” (classes) is rather complicated. There seems to be no obvious way in most programming languages to automatically embed all files in a certain directory.

My original plan was to simulate the default behavior of Go: Adding a new site to the list of supported sites would only require to add one file, the compiler would pick it up automatically. In Rust (yes, I am a software masochist, why do you ask?), this won’t work: I’ll still need to specify the new module in another file. (Arguably, this is already a relatively easy approach, but it took me a while to figure it out.)

So I wonder why most languages don’t have Go’s “plug-in” compilation system that wouldn’t require additional files. Is this really an unusual use case?]]> <![CDATA[Benchmarking Pthread]]> https://nixers.net/Thread-Benchmarking-Pthread Tue, 03 Nov 2020 19:02:47 +0000 venam]]> https://nixers.net/Thread-Benchmarking-Pthread I've been recently wondering about the effect of different pthread implementations, especially when it comes to scheduling between threads either CPU bound or not, different scheduling algorithms, and most importantly if using process contention scope or system contention scope affects normal usage "snappiness" and how much.

I've been trying to find benchmarks Ă  la TPC or SPEC, but I've only found this 20+ year old paper, Benchmarking Pthreads Performance.
According to it, process bound threads are faster to create, on all 4 machines tested, than system bound ones. However, there's no indication of the effect it would have on the normal computing. You could only guess what it would do.

I find this especially important in a world where single processes now have hundreds of threads for themselves, and if ran in system contention scope could starve other processes.
Especially that Linux only supports PTHREAD_SCOPE_SYSTEM, see man 3 pthread_attr_setscope.

I'm wondering if anyone has more info on this or interesting benchmarks to share, especially from companies switching from Linux to FreeBSD or other types of system with different pthread default behavior. Or software that have modified their software to take advantage of certain facilities.


Brief background info:
There are two types of threads: kernel-level threads and user-level threads.
Kernel-level threads are responsible for kernel stuff and handling system calls.
User-level threads are the threads user create.

User-level threads need to be mapped with kernel-level threads to be able to do anything useful. This could either be a one-to-one, one-to-many, or many-to-many relation. On most Unix-like system it's a one-to-one relation, for every thread there's a kernel thread (other than Solaris which is many-to-one).
See https://en.wikipedia.org/wiki/Light-weight_process

The kernel schedules the priority of the kernel-level thread according to whatever scheme it has in place.
What schedules the user-level thread is the thread library, in our case pthread.

There are two big ways to take care of threads, either all threads from all processes are considered equals, system contention scope. Or threads from a single process are grouped together and considered a single thread block, all processes being scheduled as is, process contention scope.

So if you have a 100 unit of processing time, 5 threads per process, and 3 processes, that would be.
System contention scope: 6.66 units per thread regardless of the process.
Process contention scope: 33.3 units per process, internally the process schedule these 33.3 between 5 threads.

Obviously CPU bounding also plays a role, along with the type of pthread scheduling policy.

Additional link: https://www.icir.org/gregor/tools/pthrea...uling.html]]> I've been recently wondering about the effect of different pthread implementations, especially when it comes to scheduling between threads either CPU bound or not, different scheduling algorithms, and most importantly if using process contention scope or system contention scope affects normal usage "snappiness" and how much.

I've been trying to find benchmarks Ă  la TPC or SPEC, but I've only found this 20+ year old paper, Benchmarking Pthreads Performance.
According to it, process bound threads are faster to create, on all 4 machines tested, than system bound ones. However, there's no indication of the effect it would have on the normal computing. You could only guess what it would do.

I find this especially important in a world where single processes now have hundreds of threads for themselves, and if ran in system contention scope could starve other processes.
Especially that Linux only supports PTHREAD_SCOPE_SYSTEM, see man 3 pthread_attr_setscope.

I'm wondering if anyone has more info on this or interesting benchmarks to share, especially from companies switching from Linux to FreeBSD or other types of system with different pthread default behavior. Or software that have modified their software to take advantage of certain facilities.


Brief background info:
There are two types of threads: kernel-level threads and user-level threads.
Kernel-level threads are responsible for kernel stuff and handling system calls.
User-level threads are the threads user create.

User-level threads need to be mapped with kernel-level threads to be able to do anything useful. This could either be a one-to-one, one-to-many, or many-to-many relation. On most Unix-like system it's a one-to-one relation, for every thread there's a kernel thread (other than Solaris which is many-to-one).
See https://en.wikipedia.org/wiki/Light-weight_process

The kernel schedules the priority of the kernel-level thread according to whatever scheme it has in place.
What schedules the user-level thread is the thread library, in our case pthread.

There are two big ways to take care of threads, either all threads from all processes are considered equals, system contention scope. Or threads from a single process are grouped together and considered a single thread block, all processes being scheduled as is, process contention scope.

So if you have a 100 unit of processing time, 5 threads per process, and 3 processes, that would be.
System contention scope: 6.66 units per thread regardless of the process.
Process contention scope: 33.3 units per process, internally the process schedule these 33.3 between 5 threads.

Obviously CPU bounding also plays a role, along with the type of pthread scheduling policy.

Additional link: https://www.icir.org/gregor/tools/pthrea...uling.html]]> <![CDATA[Alternative programming languages and their usability on *nix]]> https://nixers.net/Thread-Alternative-programming-languages-and-their-usability-on-nix Fri, 23 Oct 2020 07:36:53 +0000 opfez]]> https://nixers.net/Thread-Alternative-programming-languages-and-their-usability-on-nix I've been trying out a bunch of languages the past months, and wanted to start some discussion on the usage of other languages than traditional ones (C/++, Perl, Python, etc.).
One of the most interesting ones is Common Lisp. I've written a phlog post on programming in it on the gopher://g.nixers.net, but the main take away is that it is incredibly fun to work with. It's cool because it's not as simple as recompiling your project when you make changes, you can individually compile parts of your program, like a function, and try it out in the REPL afterwards. The downside is that it is very much dpendent on the REPL and it's hard to get a nice binary to send to others. I also find it not very *nix friendly as it isn't good to work with stdin and stdout like most (good) Unix tools.
Another one is Zig. My best description of it is "a modern C". It is basically a huge fork of C, but it's very different. The preprocessor, for example, is gone. One of the most interesting things with Zig is the ability to execute code in compile time. Asserts, if/else checks, anything can be executed at compile time, making it great for debugging. It's completely compatible with the C ABI, so you can use any of your favorite C libraries with it. Therefore, it has great *nix compatibility. Hooray!
Other languages I'm a bit interested in is Julia, with it's multiple dispatch-oriented design and Scheme, being a more simplistic Lisp with the ability to time travel in your programs (?!).

What languages are you most excited about, and how are they to work with in *nix?

Edit: Got an idea for a follow up question: What language do you think future *nix tools will be written in? Will they be written in C, or something more modern like Go?]]> I've been trying out a bunch of languages the past months, and wanted to start some discussion on the usage of other languages than traditional ones (C/++, Perl, Python, etc.).
One of the most interesting ones is Common Lisp. I've written a phlog post on programming in it on the gopher://g.nixers.net, but the main take away is that it is incredibly fun to work with. It's cool because it's not as simple as recompiling your project when you make changes, you can individually compile parts of your program, like a function, and try it out in the REPL afterwards. The downside is that it is very much dpendent on the REPL and it's hard to get a nice binary to send to others. I also find it not very *nix friendly as it isn't good to work with stdin and stdout like most (good) Unix tools.
Another one is Zig. My best description of it is "a modern C". It is basically a huge fork of C, but it's very different. The preprocessor, for example, is gone. One of the most interesting things with Zig is the ability to execute code in compile time. Asserts, if/else checks, anything can be executed at compile time, making it great for debugging. It's completely compatible with the C ABI, so you can use any of your favorite C libraries with it. Therefore, it has great *nix compatibility. Hooray!
Other languages I'm a bit interested in is Julia, with it's multiple dispatch-oriented design and Scheme, being a more simplistic Lisp with the ability to time travel in your programs (?!).

What languages are you most excited about, and how are they to work with in *nix?

Edit: Got an idea for a follow up question: What language do you think future *nix tools will be written in? Will they be written in C, or something more modern like Go?]]> <![CDATA[What can a password contain]]> https://nixers.net/Thread-What-can-a-password-contain Tue, 20 Oct 2020 13:03:44 +0000 venam]]> https://nixers.net/Thread-What-can-a-password-contain Let me throw a brainstorming session on a question I've always wondered about.
Can a unix password contain special characters, especially spaces.
I know that this is feasible, as I've tested it on the command line. It can contain anything utf-8, even non-printable characters as long as it doesn't have the default erase or kill characters.

However, the next question is: Which utilities are going to break and which ones are going to still function properly.
Another question is, what happens when a password doesn't initially have an erase or kill character and the terminal settings are changed to make one of the character one of these. Will it then be impossible to type the password as it would erase or stop the readline? What about sending the password via other types of inputs than readline, does it still work?

Let's have a go at testing this, the downsides and upsides.]]> Let me throw a brainstorming session on a question I've always wondered about.
Can a unix password contain special characters, especially spaces.
I know that this is feasible, as I've tested it on the command line. It can contain anything utf-8, even non-printable characters as long as it doesn't have the default erase or kill characters.

However, the next question is: Which utilities are going to break and which ones are going to still function properly.
Another question is, what happens when a password doesn't initially have an erase or kill character and the terminal settings are changed to make one of the character one of these. Will it then be impossible to type the password as it would erase or stop the readline? What about sending the password via other types of inputs than readline, does it still work?

Let's have a go at testing this, the downsides and upsides.]]> <![CDATA[Netflix from the command line?]]> https://nixers.net/Thread-Netflix-from-the-command-line Thu, 15 Oct 2020 23:35:50 +0000 pfr]]> https://nixers.net/Thread-Netflix-from-the-command-line
I am wondering if there are yet any ncurses/cli/tui netflix apps? I have done a bit of a search but elgoog isn't showing me much.

I was thinking, I can log into Spotify using their API and listen to music from my terminal using ncspot, surely there is an ncurses netflix tool that would allow me to browse through thumbnails (possibly displayed with w3m-img or even sixel) and then simply launch streams in mpv? Provided Netflix has an API..

Someone please tell me this app exists already because it would change my life :)

So far the only thing I've come across that looks interesting is QtDRMStreamViewer but this still uses the the Qt Web engine and requires glibC to run it.]]>
I am wondering if there are yet any ncurses/cli/tui netflix apps? I have done a bit of a search but elgoog isn't showing me much.

I was thinking, I can log into Spotify using their API and listen to music from my terminal using ncspot, surely there is an ncurses netflix tool that would allow me to browse through thumbnails (possibly displayed with w3m-img or even sixel) and then simply launch streams in mpv? Provided Netflix has an API..

Someone please tell me this app exists already because it would change my life :)

So far the only thing I've come across that looks interesting is QtDRMStreamViewer but this still uses the the Qt Web engine and requires glibC to run it.]]> <![CDATA[xnotify: popup a notification on your screen]]> https://nixers.net/Thread-xnotify-popup-a-notification-on-your-screen Mon, 21 Sep 2020 14:20:29 +0000 seninha]]> https://nixers.net/Thread-xnotify-popup-a-notification-on-your-screen xnotify.
It is a notification launcher that reads notification from the stdin and displays it on your screen.
You can create a fifo in order to echo stuff into it.

For example:

Code:
echo "Hello World" > /tmp/xnotify.fifo

Would create a notification with “Hello World” as its content.

Here's a demo gif that illustrates its usage.

[Image: 93690391-49b11780-fac7-11ea-96da-ab850dca1b9f.gif]

To launch xnotify, add the following to your ~/.xinitrc:

Code:
rm -f /tmp/xnotify.fifo
mkfifo /tmp/xnotify.fifo
xnotify </tmp/xnotify.fifo 3<>/tmp/xnotify.fifo &

You can use it together with tiramisu, an application that outputs notifications to the stdout.

I still have to improve the text and image placement on the notification window, so any idea is welcome.

So, what do you think?]]> xnotify.
It is a notification launcher that reads notification from the stdin and displays it on your screen.
You can create a fifo in order to echo stuff into it.

For example:

Code:
echo "Hello World" > /tmp/xnotify.fifo

Would create a notification with “Hello World” as its content.

Here's a demo gif that illustrates its usage.

[Image: 93690391-49b11780-fac7-11ea-96da-ab850dca1b9f.gif]

To launch xnotify, add the following to your ~/.xinitrc:

Code:
rm -f /tmp/xnotify.fifo
mkfifo /tmp/xnotify.fifo
xnotify </tmp/xnotify.fifo 3<>/tmp/xnotify.fifo &

You can use it together with tiramisu, an application that outputs notifications to the stdout.

I still have to improve the text and image placement on the notification window, so any idea is welcome.

So, what do you think?]]> <![CDATA[Git updater shell script]]> https://nixers.net/Thread-Git-updater-shell-script Thu, 10 Sep 2020 05:30:13 +0000 jkl]]> https://nixers.net/Thread-Git-updater-shell-script $HOME/github and (randomly) other folders below $HOME. Keeping them updated is relatively annoying though, so I automatized it.

You might want to modify $BLACKLIST and $GITDIRS, especially if you don’t use the bosh for scripting and/or you keep your repositories elsewhere. (On standard POSIX shells, $GITDIRS won’t work like this. Sorry. I might rewrite that line some day.)

Usage: ./gitup.sh.

Code:
#!/opt/schily/bin/bosh
# Update all Git repositories in/below $HOME.

# ---- VARIABLES

# Contains a list of (substrings of) directories to skip.
# It makes sense to not interfere with Go's own version control,
# and on macOS the ~/Library folder should not be traversed
# either.
BLACKLIST="$HOME/go/ $HOME/Library"

# Contains a list of Git repository directories.
# Currently, this is collected automatically, but it can be a
# manual list as well. Note that "find" on non-Schily shells
# usually does not have a "-call" parameter.
GITDIRS=`find $HOME/git -maxdepth 3 -name '.?*' -prune -o -type d -call '[ -e "$1/.git" ]' {} \; -prune -print`

# ---- CODE

repocount=0
for DIR in ${GITDIRS}; do
    # Blacklist check:
    allowed=1
    for BAN in ${BLACKLIST}; do
        substrings=`expr "$DIR" : "$BAN"`
        if [ $substrings -ne 0 ] ; then
            # This directory is in/below a blacklisted one.
            allowed=0
            break
        fi
    done

    if [ $allowed -eq 0 ]; then
        continue
    fi

    # This directory is promised to be not blacklisted.
    # Process it.
    repocount=`expr $repocount + 1`
    cd $DIR

    # Check the remote state:
    git fetch -q

    UPSTREAM=${1:-'@{u}'}
    LOCAL=`git rev-parse @`
    REMOTE=`git rev-parse "$UPSTREAM"`
    BASE=`git merge-base @ "$UPSTREAM"`

    if [ $LOCAL = $REMOTE ]; then
        # This repository is up-to-date.
        # Do nothing with it.
        continue
    elif [ $LOCAL = $BASE ]; then
        # The remote version is newer.
        echo "Updating Git repository: $DIR"
        git submodule sync -q
        git pull -q
        git submodule update --init --recursive
        git log $LOCAL..$REMOTE --date=short --pretty=format:"     - New [%ad] : %s"
    else
        # Something is different.
        echo "The Git repository in $DIR needs to be merged."
        echo "This cannot be done automatically."
    fi
done

Enjoy. Or modify as you need.]]> $HOME/github and (randomly) other folders below $HOME. Keeping them updated is relatively annoying though, so I automatized it.

You might want to modify $BLACKLIST and $GITDIRS, especially if you don’t use the bosh for scripting and/or you keep your repositories elsewhere. (On standard POSIX shells, $GITDIRS won’t work like this. Sorry. I might rewrite that line some day.)

Usage: ./gitup.sh.

Code:
#!/opt/schily/bin/bosh
# Update all Git repositories in/below $HOME.

# ---- VARIABLES

# Contains a list of (substrings of) directories to skip.
# It makes sense to not interfere with Go's own version control,
# and on macOS the ~/Library folder should not be traversed
# either.
BLACKLIST="$HOME/go/ $HOME/Library"

# Contains a list of Git repository directories.
# Currently, this is collected automatically, but it can be a
# manual list as well. Note that "find" on non-Schily shells
# usually does not have a "-call" parameter.
GITDIRS=`find $HOME/git -maxdepth 3 -name '.?*' -prune -o -type d -call '[ -e "$1/.git" ]' {} \; -prune -print`

# ---- CODE

repocount=0
for DIR in ${GITDIRS}; do
    # Blacklist check:
    allowed=1
    for BAN in ${BLACKLIST}; do
        substrings=`expr "$DIR" : "$BAN"`
        if [ $substrings -ne 0 ] ; then
            # This directory is in/below a blacklisted one.
            allowed=0
            break
        fi
    done

    if [ $allowed -eq 0 ]; then
        continue
    fi

    # This directory is promised to be not blacklisted.
    # Process it.
    repocount=`expr $repocount + 1`
    cd $DIR

    # Check the remote state:
    git fetch -q

    UPSTREAM=${1:-'@{u}'}
    LOCAL=`git rev-parse @`
    REMOTE=`git rev-parse "$UPSTREAM"`
    BASE=`git merge-base @ "$UPSTREAM"`

    if [ $LOCAL = $REMOTE ]; then
        # This repository is up-to-date.
        # Do nothing with it.
        continue
    elif [ $LOCAL = $BASE ]; then
        # The remote version is newer.
        echo "Updating Git repository: $DIR"
        git submodule sync -q
        git pull -q
        git submodule update --init --recursive
        git log $LOCAL..$REMOTE --date=short --pretty=format:"     - New [%ad] : %s"
    else
        # Something is different.
        echo "The Git repository in $DIR needs to be merged."
        echo "This cannot be done automatically."
    fi
done

Enjoy. Or modify as you need.]]>