nixers
xprompt: a dmenu rip-off with contextual completion - Printable Version
+- nixers (https://nixers.net)
+-- Forum: Development & Graphics (https://nixers.net/Forum-Development-Graphics)
+--- Forum: Programming On Unix (https://nixers.net/Forum-Programming-On-Unix)
+--- Thread: xprompt: a dmenu rip-off with contextual completion (/Thread-xprompt-a-dmenu-rip-off-with-contextual-completion)

xprompt: a dmenu rip-off with contextual completion - seninha - 15-08-2020

I'm working on xprompt, a project I started recently.

It is like dmenu, but with contextual completion, so you don't have to launch a new dmenu from a previous dmenu instance. The README file shows two use cases some people using dmenu have and that xprompt improves.

You can find more information on its readme and on its manual. And any ideas on how to improve it are welcome.

RE: What are you working on? - movq - 16-08-2020

(15-08-2020, 04:38 PM)phillbush Wrote: I'm working on xprompt

Ohh, that looks interesting.

I’m having a few focus issues (oh god, why is focus handling on X11 such a disaster?) and I wonder: Why did you chose to not show completion options immediately? The user has to press “Tab”. Probably personal preference to make it behave more like a shell?

(This tool has a lot of potential and maybe it should get its own thread.)

RE: What are you working on? - venam - 16-08-2020

(15-08-2020, 04:38 PM)phillbush Wrote: I'm working on xprompt

That's pretty cool!

I was intrigued about how to generate a list that would be compatible with your program based on some shell completion. I wasn't able to extract it out of zsh so I went with bash completion, and here's the result:

I started out with a list of all the programs in the $PATH.
Then I've used the following tool called complete from bash completion system to extract additional programs that have completion:
Code:
# complete | grep -o -P '(\-o (\w+))|(-F _?\w+ \w+)' | sed  's/-F _\?_\?\w\+ //g;s/-o //' | sort | uniq > all_available_completion

I then searched for quite a while to find a way to pass a certain string to a completion system to let it spit back the possible next word. I've found my solution in this blog post. I'll copy the script here:

Code:
#
# Author: Brian Beffa <brbsix@gmail.com>
# Original source: https://brbsix.github.io/2015/11/29/accessing-tab-completion-programmatically-in-bash/
# License: LGPLv3 (http://www.gnu.org/licenses/lgpl-3.0.txt)
#
#https://brbsix.github.io/2015/11/29/accessing-tab-completion-programmatically-in-bash/

get_completions(){
    local completion COMP_CWORD COMP_LINE COMP_POINT COMP_WORDS COMPREPLY=()

    # load bash-completion if necessary
    declare -F _completion_loader &>/dev/null || {
        for f in /usr/share/bash-completion/completions/*; do source $f; done
        source /usr/share/bash-completion/helpers/gst
    source /usr/share/bash-completion/bash_completion
    }

    COMP_LINE=$*
    COMP_POINT=${#COMP_LINE}

    eval set -- "$@"

    COMP_WORDS=("$@")

    # add '' to COMP_WORDS if the last character of the command line is a space
    [[ ${COMP_LINE[@]: -1} = ' ' ]] && COMP_WORDS+=('')

    # index of the last word
    COMP_CWORD=$(( ${#COMP_WORDS[@]} - 1 ))

    # determine completion function
    completion=$(complete -p "$1" 2>/dev/null | awk '{print $(NF-1)}')

    # run _completion_loader only if necessary
    [[ -n $completion ]] || {

        # load completion
        _completion_loader "$1"

        # detect completion
        completion=$(complete -p "$1" 2>/dev/null | awk '{print $(NF-1)}')

    }

    # ensure completion was detected
    [[ -n $completion ]] || return 1

    # execute completion function
    "$completion"

    # print completions to stdout
    printf '%s\n' "${COMPREPLY[@]}" | LC_ALL=C sort
}

The last piece is a script that would loop through all the available programs and generate the input file for xprompt. I wrote it in Perl:

Code:
use strict;
use warnings;

my $filename = $ARGV[0];
open (my $fh, '<', $filename);

while (my $row = <$fh>) {
    chomp $row;
    my $args = `bash -c "source $ARGV[1] && get_completions '$row ' 2> /dev/null"`;
    my @split_args = split /\n/, $args;
    print "$row\n";
    for my $arg (@split_args) {
        print "\t$arg\n";
    }
}

The first argument being the location of the file with all the programs, and the second argument the location of the get_completions function script to get sourced.

Be sure to run this in a directory that will stay empty while the script runs, otherwise you may get a completion that contains the current files in the directory. So output to another directory that isn't the current one.

The result is a file that's quite big, with 540K lines. xprompt still works perfectly though!

EDIT:

And here's a script to add man page information:

Code:
use strict;
use warnings;
use Data::Dumper;

my $filename = 'all_programs_with_args';
open (my $fh, '<', $filename);

sub begins_with
{
    return substr($_[0], 0, length($_[1])) eq $_[1];
}

while (my $row = <$fh>) {
    chomp $row;
    if (begins_with($row, "\t")) {
        print "$row\n";
    } else {
        my $description = `man -f $row | grep -P '(1)|(6)|(7)|(8)' | head -n 1 | sed 's/.*- //' 2> /dev/null`;
        print("$row\t$description\n");
    }
}

You can also remove empty lines with the following, in case the file gets polluted:
Code:
sed '/^\s*$/d' "$@"


RE: What are you working on? - seninha - 16-08-2020

(16-08-2020, 03:54 AM)vain Wrote: Ohh, that looks interesting.

I’m having a few focus issues (oh god, why is focus handling on X11 such a disaster?) and I wonder: Why did you chose to not show completion options immediately? The user has to press “Tab”. Probably personal preference to make it behave more like a shell?
Thanks. I had the idea for xprompt after having to write "mpc" on dmenu for open a new dmenu listing mpc commands that opened a third dmenu listing my songs. Three dmenus for a single command. Now I only click Tab a couple of times and no need to reopen an application multiple times.

I am also having problems with focus. When xprompt closes, the previously focused window loses the focus and the focus goes to whatever window is below the cursor. That's probably not a xprompt problem, but a problem on the wm I'm writing (the one I'd emailed you about). Indeed, focus handling in X11 is a pita.

(16-08-2020, 07:34 AM)venam Wrote: That's pretty cool!

I was intrigued about how to generate a list that would be compatible with your program based on some shell completion. I wasn't able to extract it out of zsh so I went with bash completion, and here's the result:
Thanks. Now that you have done it, I will do something similar with my ksh completions. Too bad it is not as involved as bash's.
Do you think that tab-indented input is a good design choice? Some people finds Tab hard to identify in a file, but it is easy to generate via script and don't need to be escaped like other delimiters such as comma or colon.

(16-08-2020, 07:34 AM)venam Wrote: The result is a file that's quite big, with 540K lines. xprompt still works perfectly though!
My file that feeds xprompt is 3.4MB. It contains all commands, all man pages (listed after xman, a script that opens a pdf of a manpage with zathura), all mpc commands and songs. And it also opens really fast. I had the idea to write a option to cache the stdin into a file that xprompt could read faster (although I'm not quite really willing to do it...).

You can also show the descriptions of options with xprompt.
Just write the description after the option separated by a tab, and it will list the descriptions after the items. I use it to show description of manpages when listing them.

RE: What are you working on? - venam - 16-08-2020

(16-08-2020, 09:48 AM)phillbush Wrote: My file that feeds xprompt is 3.4MB.
The one I generated was 7MB.

(16-08-2020, 09:48 AM)phillbush Wrote: It contains all commands, all man pages (listed after xman, a script that opens a pdf of a manpage with zathura), all mpc commands and songs.
It would be nice to also modify the script to include manpage description, though quite exhaustive.

EDIT: I've added a script to generate the manpages info to my previous post.

RE: xprompt: a dmenu rip-off with contextual completion - seninha - 16-08-2020

Here is my script that feeds xprompt. It is not as involved as venam's because I have to list the completions of each command separately. Also, it was done in a couple of minutes after I have written xprompt, I need to improve it.

It only provides completion for mpc commands, .desktop applications (with my run script, whose -t flag lists .desktop entries) and manpages (with my xman script, whose -t flag lists the manpages).

Here is xprompt.sh:

Code:
#!/bin/sh
# xprompt.sh
# Run `$ xprompt.sh gen` to generate a cache file.
# Run `$ xprompt.sh` to run xprompt.

CACHEDIR="$HOME/usr/local/cache"
CACHEFILE="xprompt.cache"
CACHEPATH="${CACHEDIR}/${CACHEFILE}"

usage() {
    echo "usage: xprompt.sh [gen]" >&2
    exit 1
}

comp_xman() {
    xman -t | sed 's/^/    /'
}

comp_run() {
    run -t | sed 's/^/    /'
}

comp_mpc() {
    mpc help | grep "^  *mpc" | tail +2 |\
    sed 's/\(.......................................................\)./\1    /' |\
    sed 's/  mpc \([^ ]*\) [^    ]*/    \1/'
}

gencompcache() {
    ls $(echo $PATH | tr ':' ' ') | grep -v '/' | grep . | sort | grep -v '^\"' | {
        while read cmd
        do
            echo $cmd
            if [ "$cmd" = xman ]
            then
                comp_xman
            elif [ "$cmd" = run ]
            then
                comp_run
            elif [ "$cmd" = mpc ]
            then
                comp_mpc
            fi
        done
    }
}

case $# in
0)
    <"${CACHEPATH}" xprompt -f -i "exec:" | sh &
    ;;
1)
    case $1 in
    gen) gencompcache >"${CACHEPATH}" ;;
    *) usage ;;
    esac
    ;;
*)
    usage
    ;;
esac


RE: xprompt: a dmenu rip-off with contextual completion - movq - 17-08-2020

(16-08-2020, 03:54 AM)vain Wrote: I’m having a few focus issues

This is really nasty.

xprompt (and dmenu, for that matter) try to throw the WM off its throne. They use override_redirect to try to avoid any interference by the WM in the first place. Then, they grab focus. Not once, but repeatedly until they get it.

My WM on the other hand wants to enforce a certain focus policy: Only that one window, which is currently selected by the WM, shall have focus. This conflicts with xprompt’s/dmenu’s intention (because the WM can never select xprompt/dmenu due to override_redirect). On top of that, I hadn’t even considered that override_redirect clients might steal focus, probably because focus handling is confusing enough with regular clients. All this leads to chaos. Funny that it took several years for me to notice this.

(Honestly, override_redirect is a bad flag, isn’t it? Why have a WM that is supposed to manage all windows, when clients can decide to opt out of this and do all kinds of funny stuff?)

Sorry for the ramblings. This has little to do with xprompt.


– edit: dmenu is fine on their current git master. Commit db6093f says input focus issues have been fixed. dmenu now only grabs focus when running embedded. You already said that on IRC. I dismissed it because it confuses XTerm a little bit (showing solid cursor instead of a hollow one while xprompt/dmenu is running). Other than that, I haven’t found any issues.

From a pragmatic point of view, it’s probably best to not change focus unless you really have to. That is, do what dmenu does and only touch focus when running embedded.

(I might be biased here, because that approach means I can use xprompt without having to add more workarounds to my WM. :))

RE: xprompt: a dmenu rip-off with contextual completion - s0kx - 17-08-2020

(17-08-2020, 10:10 AM)vain Wrote: Honestly, override_redirect is a bad flag, isn’t it?
but smaller/older window managers can only use this to check if your window should be undecorated etc.. Maybe xprompt could enable and disable the use of override_redirect with a command-line flag like lemonbar does??

RE: xprompt: a dmenu rip-off with contextual completion - seninha - 17-08-2020

(17-08-2020, 10:10 AM)vain Wrote: xprompt (and dmenu, for that matter) try to throw the WM off its throne. They use override_redirect to try to avoid any interference by the WM in the first place. Then, they grab focus. Not once, but repeatedly until they get it.
Indeed. But because of the nature of both applications, they cannot be treated as a regular window, they must be treated as a popup menu. EWMH has a property for it: setting _NET_WM_WINDOW_TYPE to _NET_WM_WINDOW_TYPE_MENU makes wms ignore the window or draw it as a menu (above the others). But not all wm support EWMH, while almost all of them respects override_redirect.

(17-08-2020, 10:10 AM)vain Wrote: On top of that, I hadn’t even considered that override_redirect clients might steal focus, probably because focus handling is confusing enough with regular clients.
I also have not considered the case for focus-stealing windows that are override_redirect. But in respect to which windows steal focus, override_redirect-set ones are the most likely to do it.

Exploring a little bit, I came out with this solution for the function that handle FocusIn events in my wm (don't know if it works on katriawm):
Code:
static void
xevent_focusin(XEvent *e)
{
    XFocusChangeEvent *ev = &e->xfocus;

    if (selmon->focused && (selmon->focused->win != ev->window
                        || ev->detail == NotifyPointer
                        || ev->detail == NotifyPointerRoot))
        client_focus(selmon->focused);
}
My client_focus() is your manage_xfocus(), and my 'selmon->focused' is your 'focus'.

Before that I considered handling FocusOut event, as I said to you in IRC, but I have no idea how the events come or their order.

(17-08-2020, 10:10 AM)vain Wrote: From a pragmatic point of view, it’s probably best to not change focus unless you really have to. That is, do what dmenu does and only touch focus when running embedded.

(I might be biased here, because that approach means I can use xprompt without having to add more workarounds to my WM. :))
I really know almost nothing about input methods, but I think it needs the window to be focused, not "fake-focused" like just grabbing key events.
But I can try following this approach; after all, it's just one line to be commented out on xprompt.c! Embedded xprompt keeps stealing focus from its parent anyway.

RE: xprompt: a dmenu rip-off with contextual completion - movq - 20-08-2020

(17-08-2020, 04:00 PM)phillbush Wrote: Exploring a little bit, I came out with this solution for the function that handle FocusIn events in my wm
I think this could be equivalent to my current solution. I had another bug in there, but it looks good now after a few days of testing.

I think that’s sorted out and I can now finally proceed to actually use xprompt. :-)

RE: xprompt: a dmenu rip-off with contextual completion - seninha - 20-08-2020

I've just discovered that there was already an xprompt since 1989 (and it is in FreeBSD ports)!. Its last release before 2019 was in 1991. Then it got revived last year by another developer

Its premise is a little different from mine xprompt: rather than being a text input field for the user to construct a string based on the information on the stdin, it asks the user for one or more strings and print them to stdout, standard input is not used and there is no completion feature.

For example, the following invocation of 1989 xprompt asks the user for a user name and a email:
Code:
$ xprompt -p Name: -p E-mail:

Although it was recently ported to X11R7, for being that old, it uses some obscure and archaic X11 routines and headers.
I'm gonna definitely study its code and see how I can improve my xprompt; and understand how it deals with text input and text edition, etc. I've just got a copy of Nye's Xlib Programming Manual, which is as old as that xprompt (and uses ye old K&R C), hopefully it will help me in understanding that xprompt.

That xprompt is a regular wm-managed window (no override-redirect) that grabs the keyboard but does not steal focus. It currently has no support to unicode/Xft and it uses the Athena Widget library (which is distributed with X11).

Once again, I accidentally named a program with an already existant name.
But now the list of related-but-different programs just got bigger:
- dmenu: select a string from stdin.
- my xprompt: build a string based on items from stdin provided as completions.
- ye old xprompt: create a dialog window for the user to write some strings.

All three are scriptable and pipe-able UNIXy programs.

RE: xprompt: a dmenu rip-off with contextual completion - z3bra - 21-08-2020

I think it's safe to assume that every english noun has been used as x<noun> to name a program running under X11 ☺️

As a response, I propose we now start again, but suffixing with an X instead !

promptx has a nice ring to it !

RE: xprompt: a dmenu rip-off with contextual completion - venam - 21-08-2020

(21-08-2020, 02:58 AM)z3bra Wrote: As a response, I propose we now start again, but suffixing with an X instead !

Quote: It began upon the following occasion.

It is allowed on all hands, that the primitive way of breaking eggs before we eat them, was upon the larger end: but his present Majesty's grandfather, while he was a boy, going to eat an egg, and breaking it according to the ancient practice, happened to cut one of his fingers. Whereupon the Emperor his father published an edict, commanding all his subjects, upon great penalties, to break the smaller end of their eggs.

The people so highly resented this law, that our Histories tell us there have been six rebellions raised on that account, wherein one Emperor lost his life, and another his crown. These civil commotions were constantly formented by the monarchs of Blefuscu, and when they were quelled, the exiles always fled for refuge to that Empire.

It is computed, that eleven thousand persons have, at several times, suffered death, rather than submit to break their eggs at the smaller end. Many hundred large volums have been published upon this controversy: but the books of the Big-Endians have been long forbidden, and the whole party rendered incapable by law of holding employments.

During the course of these troubles, the emperors of Blefuscu did frequently expostulate by their ambassadors, accusing us of making a schism in religion, by offending against a fundamental doctrine of our great prophet Lustrog, in the fifty-fourth chapter of the Brundecral (which is their Alcoran). This, however, is thought to be a mere strain upon the text: for their words are these; That all true believers shall break their eggs at the convenient end: and which is the convenient end, seems, in my humble opinion, to be left to every man's conscience, or at least in the power of the chief magistrate to determine.