Git updater shell script - Programming On Unix

Users browsing this thread: 1 Guest(s)
jkl
Long time nixers
I have mentioned a couple of times that I refuse to use Git for versioning my files. Yet, I sometimes need to compile something which resides in a Git repository, usually from GitHub. I keep those repositories in $HOME/github and (randomly) other folders below $HOME. Keeping them updated is relatively annoying though, so I automatized it.

You might want to modify $BLACKLIST and $GITDIRS, especially if you don’t use the bosh for scripting and/or you keep your repositories elsewhere. (On standard POSIX shells, $GITDIRS won’t work like this. Sorry. I might rewrite that line some day.)

Usage: ./gitup.sh.

Code:
#!/opt/schily/bin/bosh
# Update all Git repositories in/below $HOME.

# ---- VARIABLES

# Contains a list of (substrings of) directories to skip.
# It makes sense to not interfere with Go's own version control,
# and on macOS the ~/Library folder should not be traversed
# either.
BLACKLIST="$HOME/go/ $HOME/Library"

# Contains a list of Git repository directories.
# Currently, this is collected automatically, but it can be a
# manual list as well. Note that "find" on non-Schily shells
# usually does not have a "-call" parameter.
GITDIRS=`find $HOME -maxdepth 3 -name '.?*' -prune -o -type d -call '[ -e "$1/.git" ]' {} \; -prune -print`

# ---- CODE

repocount=0
for DIR in ${GITDIRS}; do
    # Blacklist check:
    allowed=1
    for BAN in ${BLACKLIST}; do
        substrings=`expr "$DIR" : "$BAN"`
        if [ $substrings -ne 0 ] ; then
            # This directory is in/below a blacklisted one.
            allowed=0
            break
        fi
    done

    if [ $allowed -eq 0 ]; then
        continue
    fi

    # This directory is promised to be not blacklisted.
    # Process it.
    repocount=`expr $repocount + 1`
    cd $DIR

    # Check the remote state:
    git fetch -q

    UPSTREAM=${1:-'@{u}'}
    LOCAL=`git rev-parse @`
    REMOTE=`git rev-parse "$UPSTREAM"`
    BASE=`git merge-base @ "$UPSTREAM"`

    if [ $LOCAL = $REMOTE ]; then
        # This repository is up-to-date.
        # Do nothing with it.
        continue
    elif [ $LOCAL = $BASE ]; then
        # The remote version is newer.
        echo "Updating Git repository: $DIR"
        git pull -q
        git log $LOCAL..$REMOTE --date=short --pretty=format:"     - New [%ad] : %s"
    else
        # Something is different.
        echo "The Git repository in $DIR needs to be merged."
        echo "This cannot be done automatically."
    fi
done

if [ $repocount -gt 0 ]; then
    echo "Successfully processed $repocount repositories."
fi

Enjoy. Or modify as you need.
venam
Administrators
That's a very useful script and could easily be adapted to anything other than HOME (well you could just as well use HOME=yourdir ./script too).

Aside: Your shell script writing script are the cleanest I've seen—this is very readable, which is not something I'd say of a lot of shell scripts I've read over the years.
jkl
Long time nixers
HOME=something stops working if you have no single place for your repos. But I did not want to overdo it with configuration parameters. I wrote this for myself and did not intend to share it initially.
And I actually prefer to understand what the hell I was thinking, so writing readable code makes sense to me ... ;-)

Admittedly, sh makes it relatively easy to write a horrible mess of a code. One of the reasons why I resort to other languages for more complicated tasks, e.g. those which require interactive user input.
jkl
Long time nixers
I updated the initial post, adding a log to the git pull branch:

Code:
git log $LOCAL..$REMOTE --date=short --pretty=format:"     - New [%ad] : %s“

I noticed that I, sometimes, want to know what has changed.

Results:

Code:
% ./gitup.sh
Updating Git repository: [REDACTED]/linecook
     - New [2020-10-13] : Merge pull request #3 from mortie/patch-1
     - New [2020-10-13] : Fix the links in the table of contents
Updating Git repository: [REDACTED]/ion
     - New [2020-09-03] : Fix set and status argument handling
Updating Git repository: [REDACTED]/knoxite
     - New [2020-10-12] : Set further default values when executing `knoxite config alias...`
     - New [2020-10-12] : Include some default toml-comments in the struct tags of RepoConfig
     - New [2020-10-12] : Switch to pelletier/go-toml as toml library for the config system
     - New [2020-10-12] : Display the error message too when saving the config fails in tests
Successfully processed 14 repositories.

I have noticed that it takes a while to process 14 repositories. I consider a rewrite in an actual programming language - shell scripts don’t work well with threads. (I tried playing with the parallel command, but it makes the code hard to read.)
mcol
Nixers
Looks useful!

Along a simlar vein is gita which keeps a list of your local git folders and can perform tasks on all of them, much like you do. I assume gita would have good error handling -- for example, if you want to pull all of your repos, but one of them needs merging or something. Although I guess this isn't a problem you have, if you only have the clones around for use rather than development.
jkl
Long time nixers
I use my script only for repositories which I use (= tools in my $PATH).
gita seems useful as well, but I could imagine that, given that it is mostly written in Python, its performance is even worse than mine.
mcol
Nixers
Yeah, very possible.