I, and I imagine many others, have a wide collection of miscellaneous scripts to perform various tasks. "Scripts" in this case is defined as interpreted/not compiled/fast startup over runtime performance, shorter than ~150 LoC, and interacting heavily with the file system and/or other processes.

The go-to language for this in the past has been shell scripts, but I keep getting annoyed by weird bashisms, the way it handles errors, the way it handles any kind of non-string data (numbers are already a pain, and don't get me started on associative arrays). Which means I now have a hodgepodge of languages, including Ruby, Racket, and Janet, (and more bash) with no clear idea of which to use when.

Ruby has the backtick syntax which allows running shell commands without a lot of hassle. Contrary to Python, where subprocess includes a lot of ceremony for basic tasks.

Janet is explicitly meant for scripting, but (1) I haven't really gotten into it, and (2) I have some doubts about the ecosystem and portability.

Racket has been quite nice, as it has a very fully featured standard library & package ecosystem, and is reasonably widely supported.

I have also looked at:

- Elvish: seems to be more focussed on being a rich interactive shell, and somewhat niche
- Python: see the subprocess note above
- Oil: looks promising, but not sure about stability yet
- Haskell with shebang: just too much in every way
- Common Lisp with Roswell: might be an okay option? Haven't really tried

Another consideration I have not explored yet is the ability to build a library of useful utility functions, i.e. a decent module system.

All of this is a long-winded way of saying:

What do you use for scripting, and why?

I've always used Perl for scripts when they get long enough to be annoying to write in shell.
It's supported and installed by default on almost all Unix-like machines and has one of the most extensive libraries. Additionally, the libraries all come with manpages, similar to what is done with C, so they integrate well with the system.

It depends.

For system automation, there is no better option than a POSIX shell. (I don’t even have a bash. Problem solved.) I usually recommend Schily’s shells for that, unless you have a POSIX-compatible /bin/sh on your system.

For tasks which would be too complicated (but still absolutely need to be a script, not a binary file), I also use Perl or, rarely, Common Lisp (sbcl --script). Python is a bad language with a weird syntax and a laughable performance (also, dependency hell whenever you update).

I tend to go to python, just because it's more likely to be pre-installed on any system I work on. I enjoy ruby but it's not my goto because it's not likely to be installed for some reason.

subprocess does have a lot of boilerplate you have to use to be able to use it for simple tasks, but the flipside of that is that it's more flexible than any other shell callout syntax. You can easily feed stuff to stdin and read stuff from stdout and stderr all in the same subprocess call, which, in other languages, requires a lot of fiddling and usually some extra libraries.

(24-03-2021, 09:11 AM)mattrose Wrote: I tend to go to python, just because it's more likely to be pre-installed on any system I work on.

I would assume that Perl comes with more systems than Python...? Granted, I haven’t checked that in a while. The only notable Python-focused system I had in the past few years was Gentoo.

I go with Posix shell scripts, or Korn shell when I need some feature not in Posix.

I like Python too, but have never used it enough to become more than a beginner with it. By the time my project has become complex or demanding enough for shell script to be unsuitable, I'm leaving the interpreted languages and going straight to C. Ideally, this would mean only using C to write any missing scriptable components or performance-critical pieces needed to complete the task in shell, but I don't always stop there and often end up writing the whole thing in C.

I tend to go with whatever my system comes with, or sometimes python when it seems more appropriate for the task. I've taken a small powershell class few years ago, but I don't really remember much from it and I'd much rather use bash inside WSL.

I'd really love to master awk one day since it's so powerful and already runs on all of the systems I'm interested in.

The awk book by the language's creators is a lovely piece of work itself.

I do really like awk as well, and I'm using it more and more, but I don't think I could use it as a general purpose scripting language, that's just not what it's meant to be used for.

I am using more and more AWK now. I use AWK, for example, to control my window manager. It process the output from lsc (list clients, a C program I wrote to get the list of clients and their attributes) and pipes into wmctrl (window manager control).
When I cannot use AWK directly, I sometimes write a C program that gets the required data and then pipes into AWK to process it (as I have done with window management).

I really want to learn Perl and maybe replace some C+AWK+Shell combinations with a pure Perl solution.

Other than that, I'm learning Scheme/Racket, but it's more for fun than for using it as a glue language to control and interact with the system.

I use mostly POSIX shell for scripting: using either /bin/dash or busybox's ash.
I use awk each time I need to work with formats which have 2 characteristics: 1) context-dependent informations (like `ip link`'s output, or subsets of YAML, like /var/lib/dpkg/status (ok, it's older than yaml, but a lot nicer, too)) and 2) do not embed strings which support escape characters (like «"hello \"world\"\n"»). Luckily, many shell utilities have such outputs.

When those do not suffice, I usually go with my simplified C++ setup (no exceptions, no RAII, can be statically linked), because I'm too lazy to learn perl.
Python is a no-go for unpaid work: I think a turing-complete language that uses whitespaces as syntax element is a bad idea. Also, I tend to favor things which do not break backward compat and/or have an actual standard, not just a "reference implementation", because I hate restarting from scratch every few years.

Recently, I started playing more seriously with prolog, and I think it might also become a powerful scripting tool, for when I write code which must take different paths depending on pre-conditions. This is a language which feels really powerful to write IAs, sure, but it also feels interesting to check for dependencies, and it came to my mind it might be interesting to use it for things like my runit control files or, why not, generating ninja files to build my C++. Maybe even not bother with ninja files and directly summon compiler, since prolog may be even more suited than ninja or make to do that job.

Nobody mentioned Tcl yet? I saw it being mentioned as “what the shell should have been”. I’ve never really learned it, maybe I should?

Tcl is lovely, but it is unusual.

I usually fallback to sh or sh-flavored bash, which is a limited but very available common denominator. babashka has been creeping into my scripts in places, and of I'll sometimes use emacs lisp in a script form (in a way where I partially load my config, giving me some emacs lisp sugar libraries for doing things)

edit: oil intrigues me the most, the author keeps making incremental improvements over the years and seems to be very vocal with nice ~opinions~ -- mostly I'm interested in using it with existing bash scripts for better feedback/debug abilities