Recently I've been seeing a lot of programs on GitHub that are rewritten versions of common UNIX command-line programs. These programs usually claim to be faster, and feature lots of colors. Some I've stumbled upon have been fd, sd, exa, and lsd. Funnily they are all written in rust. fd, and sd seem interesting, and quite useful, on the other hand, I have some negative opinions on exa and lsd. Exa offers a ton of features that I feel would be useful in something like a vifm, ranger, etc, but not in a "ls" style program. Also, if you want a super colorful ls output, use a LS_COLORS generator. lsd adds Font Awesome icons and more coloring options, but these don't seem necessary when most people just use ls to quickly view a directory's contents, as intended.

We've had a sort of similar discussion here.

There are multiple aspects to this question.
Are those fun helpers? I bet they are, they're colorful, usually more intuitive than anything else. The authors had fun writing them, maybe learned something along the way (isn't it usually like that that those tools appear).

Is it necessary to rewrite some core utilities? Well, in a certain way because it teaches the author about the ins and outs of what is necessary to achieve a similar tool to a core one it may introduce them to why the utility is currently the way it is, or how they could help in fixing it.

Are those tools the must have for everyone? For each their own. I'd rather stick with core utilities for the simple reason that they are available everywhere but I don't have an issue testing something novel written in a fancy language.

NB: Your thread was moved from the community section to the shell scripting one.

It's worth remembering that the GNU coreutils were replacement utilities once too, and still can often be used as such. So IMO it's hard to say anything about them in general, if anything it's a plus that they can exist, and most of the time don't have to be exchanged all at once.

There are two sides of that. I like to extend the core utilities (never GNU - avoid GNU like hell!) by useful tools like better text editors and file managers, but I would never replace them. I need the basic POSIX commands as standards-compliant as possible. Shell scripts which are tailored to my personal desktop are not shell scripts which are written well.

I think reinventing the wheel can be very beneficial in these cases.

While the original coreutils are very powerful and have certainly advanced over the years, they have mostly just added new features and flags. Writing new, incompatible tools has a very important advantage: they can rethink the user experience and interface.

Even if we completely disregard the performance, using something like fd over find is simply much nicer. You can mostly do the same things, though find can probably do more things and is more versatile in scripting. But if I just want to find a file with "foo" in the name, `fd foo` is easier to use, even if I at this point know automatically that `find -iname '*foo*'` would do the same thing. And usually I'm only interested in source files, so fd automatically filters eg. .gitignored files away. Same with rg.

Someone could point out that this could be achieved with shell aliases, and for a long time I did that. But if you're okay with config management for all machines you use, you can practically with the same effort just install the new fancy tool, and get the other benefits as well. Thought, if I do require something more complex (usually with scripts), I usually fall back to coreutils, even if the fancy ones had the same features with different syntax. For complex things learned habits are more important to me. But not every search needs to be complex.

Interface design matters, even with CLI tools.

This trend hasn't stopped since that post has been written. We now see, more often than ever, countless new tools popping up in the news, all doing the same behavior as the old one but with more "fancy" or "user-friendly" features. Or just plainly, the only selling feature being that it was rewritten in the language fashion of the day.

As mentioned, we also had a previous thread with a similar topic: helper/wrapper scripts.

What's your opinion on these? Have you tried some of these new tools? Have some of them helped in places? Is it hard to install them and/or keep track of their changes? Are they maintained properly and will they be maintained on the long run? Which one is your favorite between them?

I have, indeed, aliased cat to bat (by the author of fd) on at least one of my systems. The combination of cat and less is useful for me. Other than that, I still need to find a good reason to move away from find, ls and sed.

I think I am too used to the core utils to move away from them. I'm not fond of colors and interactive filters anyway, so I'm fine with keeping them. Their behavior is also more predictable because defined by POSIX.

I do think however that some tools should be part of the core utilities, and thus "extend" my core utils rather than replace them:

sponge (moreutils)
Reads ALL input before writing to output. Useful for "inplace" operations.

pick (mptre)
Simple interactive selector for the terminal. I know that I said I'm not fond of "interactive" commands, but they are sometimes handy, and I'd rather have a dedicated tool for that, rather than have it built into every single tool, each with a different interface.
I use it mostly to select hostnames out of the nearly 2000 hosts we have at work.
This is a much simpler version of my work script (I retrieve host list using an API with curl(1)):
abduco (brain-dump)
Session manager, replicating the "detach" feature of tmux. This is the replacement for the now outdated "dtach" (which still works great).

(23-11-2020, 06:13 AM)z3bra Wrote: pick (mptre)
Simple interactive selector for the terminal.

I tend to use fzf for tasks like these. It's really fast and provides a great search. I don't know how it compares to pick though, gotta check it out.

Generally, fzy is probably faster.

It looks great, thanks for the recommendation. Is the vim integration as good as fzf's? Looks like the plugin implementation is a lot smaller.

I have stopped using Vim quite a few years ago. You might want to try it yourself. ;-)

What do you use instead? Emacs? Vis?

That varies a lot. However, GNU Emacs is the only one where fzy integration makes any sense.

I think replacement for core utilities are not a problem, but as z3bra, I prefer tools that extends this set and I tend to rely a lot on the standard ones.

One example of extension I have built (probably the only one worth speaking about) is a tool to merge lines that share a set of identical columns. By merging I mean putting them on the same line, without removing the columns.
An "old" version of it is here: https://www.deadbeef.fr/projects/tools/f....1.md.html (this is the markdown files that I use to generate the man page).

Other than that, I would not be against a complete tool set that does what core utilities do (but differently), but with a more consistent API, and why not, expose a way to actually allow a shell to extract auto-completion through an option instead of relying on external files.
This basically imply that I don't care about stand-alone replacements for the core utilities, since by definition, they won't share consistent API, and thus "force" the user to learn every single tool. And I don't have time to waste with non-reusable knowledge.

I know, plan9 had that, and I should try it, someday...

I mostly agree with the stated opinions here, extending coreutils is fine, but replacing is a no-no.

One other reason I’m hesitant to jump onto all those “like X but nicer (and in Rust)” tools is that by getting used to these, I forget how to use coreutils. I’m frequently SSHing into production boxes at work, where there are only coreutils, so knowing how to use them effectively is somewhat important when things are on fire.

That being said, I only use fd and ripgrep from the new school of tools. I wouldn’t know how to do ripgrep in coreutils (find + grep I guess), and fd really just to avoid typing out that annoyingly long find syntax every time.

In general, I think its great. nothing is perfect and we should always strive to make things better. That's open source. Coreutils are great, but I think there will come a time when it will e replaced with more feature rich utils while still remaining small.

(13-03-2019, 08:55 PM)gmk Wrote: exa, and lsd

Funnily enough, I have exa aliased to lsd. I just like the colours and I don't really need the icons like in lsd.

(23-11-2020, 05:34 AM)jkl Wrote: aliased cat to bat

I use bat also, but I didn't alias cat to bat because on xterm, If I need to select/copy multiple lines from a file I need to use cat because bat includes the line numbers.

"bat -p" should solve that.

(25-11-2020, 07:52 AM)jkl Wrote: "bat -p" should solve that.

Ahh thanks! Might just alias cat to bat -p then.

Speaking of replacement coreutils, I just came across zoxide which is a replacement for cd. So far I love it.

Sadly, zoxide does not support my shell.

(26-11-2020, 07:04 AM)jkl Wrote: Sadly, zoxide does not support my shell.

Which shell ?

tcsh.
Supporting it is not prioritized either.

(23-11-2020, 09:38 AM)opFez Wrote: What do you use instead? Emacs? Vis?

So I'd never really looked into vis before now, but it looks interesting. Anybody here use it over vim? I've also stumbled across kakoune which is another modal editor, but I honestly don't need all those features.
I'd be interested to hear any opinions on these editors from anyone who uses them.

Warning: wall of text ahead ! TL;DR at the end.

I've been using vis full-time to replace vim for the last few years, and can't see myself switching back to vim now. I'm convinced vis is more powerful than vim for text editing, for a couple reasons. Before I get into these, you must first understand how I used vim:

No plugins. I was one of those weirdo that though vim was best used with no plugins, in order to truly master the editor. My .vimrc was only a few lines, mostly to disable syntax HL (personnal preference), and set sane default for my needs. I made heavy use of macros for bulk edits, and though it was the best feature an editor could have to make arbitrary edits in a large scale way. I used recursive macros like a boss and looked down on everyone using plugins while not even trying to do stuff directly with "vanilla" vim.

In my quest to minimalism, I found vis(1) which was announced on the suckless mailing list, and advertised as "90% of vim features, with 10% of the code". This sounded delicious, and I started using it. Despite some bugs, it filled the promise, and some missing stuff were on the way to get implemented (like macros), so I used it more and more alongside vim, which I only used for some specific use cases (like digraphs, macros, which were not builtin at the time). The git log of vis was showing up new features and fixes every week, making it even more usable and powerful.

I spent more and more time in vis, loving it more and more everyday.

Then a new feature came in: sam's structural regular expressions. This is when vim's editing capabilities started feeling painful to me.

Sam is a Plan 9 text editor built around the "structural regular expressions" concept. To put it simply, you don't edit a full text buffer anymore, but rather small portions of it, that you pre-select using regular expressions. Then your commands will be applied to the "dot", which is your selection. As an example, take the following buffer:


Let's first fix that ugly typo. You'll first need to set the "dot" (current selection) to that word. Then, you can apply a command to this selection ('d' for delete, 'c' for change, 'i' for insert, … You know them as they somehow match ed and vi commands):


The 'x' command sets the "dot" using a regexp: then the 'c' command will replace the "dot" with the provided text. The above command is similar to the "substitute" command in vi, except that it's actually made of two separate commands: select + change, which is much more modular. Note that commands are bound to the same keys as the letter commands, which means that the above change can also be done interactively with:


Let's now show how vim and vis differ for a few use cases.

Using the text above, I want to do these 3 tasks:

1. add "really" before all occurence of "want" (prepend)
2. emphasis "really" by uppercasing it (change)
3. delete word "simple" on line 3 (delete)

With vim command mode:

As you can see, vim leverages on the "substitute" command to do all three "verbs": prepend, change, delete. This doesn't follows the "vim grammar" that makes its normal mode so "powerful". But anyway, vim is not made for command line edition, so let's see the same edits in normal mode (commented out):


As you can see, this is fairly different from vim. For each action, I use the corresponding sam command (insert, change, delete) rather than the "substitute" command everytime.
You'll note that I selected "want" only once (":x/want/"). This is because all commands apply to the whole buffer, unless specified, and the 'x' command (which means eXtract), will select ALL occurence of the search, so the command specified afterward applies to each occurence, kind of like a "for" loop.
For the last command, I limited the 'x' command to like 3, so it only select occurence of the word "simple" on line 3.

Let's see the normal mode way of doing it, using multicursor instead of macros:


So the way you'd do it with vis is by using multiple cursors. There are multiple ways to create them, and the ^N used here is not the most efficient (and you can't "repeat" it by using 3^N for example, it's a feature limitation). This is however much more easy to figure out what's going on, because all selections are updated in real time. This also make the selection interactive, and you can decide for each match wether or not to include it (hence why I didn't search "really want", and just "really", because I can press ^X to skip the one I don't want to edit).

However in this case, you'd rather use ":x/want" to create all multiple cursors at the same time, which is much more efficient. The biggest avantage to it hover macros is that you get a realtime feedback of the changes you do. And of course, the "undo/redo" work just as expected: per cursor. Same goes for yank/paste ! Each cursor has its own register for yanked text.

So yeah, I've switched to vis(1) fulltime, and whenever I have to use vim instead, I am disapointed. I miss the better commands, and I miss the multiple cursors. And just in case you're affraid to "loose" your vim editing skill: don't be scared, you won't loose them. After years of using vis, I can still kick some ass with my powerful vim editing skills. Just less efficiently than with vis ! It's kind of like typing again on QWERTY after years on DVORAK. You can still do it, but it feels awkward…

Oh, and did I mention that vis has a '|' command, to use external tools to edit selections ? Try to figure out the following:


Ok vim has it built-in… The real question is: WHY BUILTIN ??!


TL;DR: vis multicursor and structural regexp are a must, making vim feels old and clunky in comparison.

Edit: Screencast of these edits on both vim and vis : http://0x0.st/i7cA.webm.
Note that I forgot mid edit how to change the primary selection when multiple cursors are used, hence the popped up manpage (answer: ^D to cycle through selections, note the bold 'y' at the end of the word, indicating the primary cursor position).

(30-11-2020, 08:48 AM)z3bra Wrote: Oh, and did I mention that vis has a '|' command, to use external tools to edit selections ? Try to figure out the following:

Code:

:x/caesar/ |tr a-m n-z

Which is probably the most interesting thing borrowed from Sam/Acme.

This is indeed pretty useful, but I don't use it as much as the structural regexps and multi-dot changes.

Vis looks great! I'll try it out, see how it goes.