POSIX Shell Programming Challenge

POSIX Shell Programming Challenge - Printable Version
+- nixers (https://nixers.net)
+-- Forum: Development & Graphics (https://nixers.net/Forum-Development-Graphics)
+--- Forum: Programming On Unix (https://nixers.net/Forum-Programming-On-Unix)
+--- Thread: POSIX Shell Programming Challenge (/Thread-POSIX-Shell-Programming-Challenge)

POSIX Shell Programming Challenge - mort - 12-04-2021

Hi,

I thought it would be fun to have a small shell programming challenge on here. The challenge is to iterate over all the files in a directory and print their path, without looking at the internet for guidance. We will then try to come up with ways in which proposed solutions are incorrect.

Here are the criteria:

Iterate over all the files in the directory "test" (assume it exists) and print their names
Include hidden files, but exclude "." and ".."
All legal file names must be supported, so the only characters you may assume are not in a file name are "/" and the 0 byte
Don't be recursive, directories should be treated as files
The body of the loop must be in the same process as the rest of the script
You may only use POSIX shell syntax, no bashisms

Here's an example:

Code:
for f in test/*; do

    echo "$f"

done

And some possible critiques:

The solution only considers visible files, it wouldn't see a file prefixed with "."
The solution doesn't work if there are no (non-hidden) files in the directory; the loop will end up printing "test/*"

Looking forwards to seeing what solutions and non-solutions y'all come up with 🤓

EDIT: The idea behind the challenge is to end up with a shell loop over all the file names in a directory; printing the file names is just an example use case. Imagine replacing "echo" with some shell function or anything else. Also, it's not important whether the file name is prefixed with the path to the directory or not, because adding or removing a path prefix is trivial and not the point of the challenge.

RE: POSIX Shell Programming Challenge - movq - 12-04-2021

How about that:

Code:
for i in test/* test/.*

do

    if ! [ -e "$i" ]

    then

        continue

    fi

    fname=${i##*/}

    if [ "$fname" = . ] || [ "$fname" = .. ]

    then

        continue

    fi

    echo "$i"

done

I very vaguely remember that you posted something similar on IRC a while ago, which showed some very unexpected ways to screw this up. 😁

Is it valid to print `test/foo`? Or should it be just `foo`? The latter opens another can of worms.

RE: POSIX Shell Programming Challenge - s0kx - 12-04-2021

Came up with something really horrible looking before remembering that wsl of course uses bash, which makes me disqualifed. Like mort's example, this also breaks with empty directories. I will certainly be returning to this challenge later..

Code:
~ $ for i in test/{*,.*[^..]}; do echo $i; done

test/directorywithfiles

test/file with spaces.txt

test/textfile.txt

test/.hiddenfile.txt

~ $

RE: POSIX Shell Programming Challenge - s0kx - 12-04-2021

(12-04-2021, 10:49 AM)movq Wrote: Is it valid to print `test/foo`? Or should it be just `foo`? The latter opens another can of worms.

Well since we're only printing files from a single directory, wouldn't that simply be solved by stripping the directory name from the path when echoing it back to the user i.e.

Code:
echo "${i##test/}"

RE: POSIX Shell Programming Challenge - jkl - 12-04-2021

"Using bash" does not disqualify you as long as you stick to the POSIX standard. :P

RE: POSIX Shell Programming Challenge - sth - 12-04-2021

this is pithy and i suppose not quite in line with the challenge but:

ls -1A test/

would do the trick :P

RE: POSIX Shell Programming Challenge - mort - 12-04-2021

I should clarify, the main purpose of the challenge is to loop over files. Printing the file names is just a silly example of a use case for looping over files. Therefore, it doesn't matter whether the file name is prefixed with "./test/" or not; adding or removing the prefix is easy enough to do. Also, relying on "ls" to print all the file names is invalid, because that just outsources the loop to C; we're interested in having a shell script which can iterate over all the files in a directory and do something useful with the file names.

I've updated the original post to clarify.

RE: POSIX Shell Programming Challenge - mort - 12-04-2021

(12-04-2021, 10:49 AM)movq Wrote: How about that:

Code:
for i in test/* test/.* do if ! [ -e "$i" ] then continue fi fname=${i##*/} if [ "$fname" = . ] || [ "$fname" = .. ] then continue fi echo "$i" done

I don't know whether this is technically correct according to POSIX or not, but it doesn't work in ZSH. In ZSH, "test/.*" is an error if there are no hidden files in the directory, and "test/*" is an error if there are no non-hidden files in the directory.

Otherwise, this seems to work really well in the other shells.

RE: POSIX Shell Programming Challenge - seninha - 12-04-2021

(12-04-2021, 11:47 AM)s0kx Wrote: Came up with something really horrible looking before remembering that wsl of course uses bash, which makes me disqualifed. Like mort's example, this also breaks with empty directories. I will certainly be returning to this challenge later..

Code:
~ $ for i in test/{*,.*[^..]}; do echo $i; done test/directorywithfiles test/file with spaces.txt test/textfile.txt test/.hiddenfile.txt ~ $

It also fails when there is no hidden file.
Quoting from The UNIX Programming Environment:

Quote:What happens if no files match the pattern? The shell, rather than complaining (as it did in early versions), passes the string on as though it had been quoted. …

Code:
$ ls x*y x*y not found $ >xyzzy $ ls x*y xyzzy $ ls 'x*y' x*y not found

RE: POSIX Shell Programming Challenge - seninha - 12-04-2021

The idea I had is to print the content of test/ delimited with \0 by find(1) and loop over it, using readlink(1) to get the full path of the file.

Code:
for i in `find test/ -mindepth 1 -maxdepth 1 -name '*' -print0`

do

    readlink -f "$i"

done

But it would require setting $IFS to \0. And setting $IFS to nul means that no splitting occurs rather than splitting on the nul character.

EDIT: My answer is not qualified for the challenge. My find(1) man page says that the -print0 primary is a non-POSIX extension. But it could be easily replaced with -print, I think.

RE: POSIX Shell Programming Challenge - movq - 13-04-2021

(12-04-2021, 02:35 PM)mort Wrote: I don't know whether this is technically correct according to POSIX or not, but it doesn't work in ZSH. In ZSH, "test/.*" is an error if there are no hidden files in the directory, and "test/*" is an error if there are no non-hidden files in the directory.

POSIX says:

Quote:If the pattern does not match any existing filenames or pathnames, the pattern string shall be left unchanged.

So I guess it’s fine. :)

(I wish POSIX would demand Bash’s “nullglob” option. That’s what I want 99.999999% of the time.)

RE: POSIX Shell Programming Challenge - seninha - 15-04-2021

Another challenge, if you don't mind.

What's the best way to replace a $HOME prefix in a path (say, $PWD) with "~"?
For example, replace "/home/phill/tmp" with "~/tmp"

I tried this:

Code:
$ echo "$PWD" | sed "s,^$HOME,~,"

~/tmp

But since we are considering the worst scenarios in this thread, what if $HOME contains a comma?
Is there any POSIX shell failproof solution?

RE: POSIX Shell Programming Challenge - s0kx - 16-04-2021

(15-04-2021, 09:32 PM)phillbush Wrote: Another challenge, if you don't mind.

Yes please!

(15-04-2021, 09:32 PM)phillbush Wrote: what if $HOME contains a comma?

Real men don't use sed! (ಠ_ಠ)

Code:
$ FAKEPWD=/home/user,with,commas/tmp

$ FAKEHOME=/home/user,with,commas

$ echo ~${FAKEPWD##$FAKEHOME}

~/tmp

Just kidding of course. I hope I understood your problem correctly, but according to my simulation this seems to be working as you can see.

Edit: Almost forgot to mention, this just assumes you actually are somewhere inside $HOME, since it just crudly adds '~' to the start and cuts out $HOME....

RE: POSIX Shell Programming Challenge - TheAnachron - 16-04-2021

Not sure if this is stupid or genius:
> printf '/home/user,with,commas/tmp' | awk -F '/home/user,with,commas' '{print $2}'

RE: POSIX Shell Programming Challenge - s0kx - 01-05-2021

phillbush, did you ever find a proper solution to your problem?

Also if anyone can come up with a new challenge, please post it here :)

RE: POSIX Shell Programming Challenge - seninha - 01-05-2021

(01-05-2021, 04:56 AM)s0kx Wrote: phillbush, did you ever find a proper solution to your problem?

Yes, people on #bash at Freenode helped me.

Code:
case "$PWD/" in

"$HOME"/*)

    dir="~${PWD#"$HOME"}"

    ;;

*)

    dir="$PWD"

    ;;

esac

echo $dir