Git updater shell script

Git updater shell script - Programming On Unix

Users browsing this thread: 6 Guest(s)

	jkl Offline \| 10-09-2020, 02:30 AM \| #1

I have mentioned a couple of times that I refuse to use Git for versioning my files. Yet, I sometimes need to compile something which resides in a Git repository, usually from GitHub. I keep those repositories in $HOME/github and (randomly) other folders below $HOME. Keeping them updated is relatively annoying though, so I automatized it.

You might want to modify $BLACKLIST and $GITDIRS, especially if you don’t use the bosh for scripting and/or you keep your repositories elsewhere. (On standard POSIX shells, $GITDIRS won’t work like this. Sorry. I might rewrite that line some day.)

Usage: ./gitup.sh.

Code:
#!/opt/schily/bin/bosh

# Update all Git repositories in/below $HOME.

# ---- VARIABLES

# Contains a list of (substrings of) directories to skip.

# It makes sense to not interfere with Go's own version control,

# and on macOS the ~/Library folder should not be traversed

# either.

BLACKLIST="$HOME/go/ $HOME/Library"

# Contains a list of Git repository directories.

# Currently, this is collected automatically, but it can be a

# manual list as well. Note that "find" on non-Schily shells

# usually does not have a "-call" parameter.

GITDIRS=`find $HOME -maxdepth 3 -name '.?*' -prune -o -type d -call '[ -e "$1/.git" ]' {} \; -prune -print`

# ---- CODE

repocount=0

for DIR in ${GITDIRS}; do

    # Blacklist check:

    allowed=1

    for BAN in ${BLACKLIST}; do

        substrings=`expr "$DIR" : "$BAN"`

        if [ $substrings -ne 0 ] ; then

            # This directory is in/below a blacklisted one.

            allowed=0

            break

        fi

    done

    if [ $allowed -eq 0 ]; then

        continue

    fi

    # This directory is promised to be not blacklisted.

    # Process it.

    repocount=`expr $repocount + 1`

    cd $DIR

    # Check the remote state:

    git fetch -q

    UPSTREAM=${1:-'@{u}'}

    LOCAL=`git rev-parse @`

    REMOTE=`git rev-parse "$UPSTREAM"`

    BASE=`git merge-base @ "$UPSTREAM"`

    if [ $LOCAL = $REMOTE ]; then

        # This repository is up-to-date.

        # Do nothing with it.

        continue

    elif [ $LOCAL = $BASE ]; then

        # The remote version is newer.

        echo "Updating Git repository: $DIR"

        git pull -q

        git log $LOCAL..$REMOTE --date=short --pretty=format:"     - New [%ad] : %s"

    else

        # Something is different.

        echo "The Git repository in $DIR needs to be merged."

        echo "This cannot be done automatically."

    fi

done

if [ $repocount -gt 0 ]; then

    echo "Successfully processed $repocount repositories."

fi

Enjoy. Or modify as you need.

--
<mort> choosing a terrible license just to be spiteful towards others is possibly the most tux0r thing I've ever seen

	venam Offline \| 10-09-2020, 02:43 AM \| #2

That's a very useful script and could easily be adapted to anything other than HOME (well you could just as well use HOME=yourdir ./script too).

Aside: Your shell script writing script are the cleanest I've seen—this is very readable, which is not something I'd say of a lot of shell scripts I've read over the years.

	jkl Offline \| 10-09-2020, 02:51 AM \| #3

HOME=something stops working if you have no single place for your repos. But I did not want to overdo it with configuration parameters. I wrote this for myself and did not intend to share it initially.
And I actually prefer to understand what the hell I was thinking, so writing readable code makes sense to me ... ;-)

Admittedly, sh makes it relatively easy to write a horrible mess of a code. One of the reasons why I resort to other languages for more complicated tasks, e.g. those which require interactive user input.

--
<mort> choosing a terrible license just to be spiteful towards others is possibly the most tux0r thing I've ever seen

	jkl Offline \| 13-10-2020, 06:16 AM \| #4

I updated the initial post, adding a log to the git pull branch:

Code:
git log $LOCAL..$REMOTE --date=short --pretty=format:"     - New [%ad] : %s“

I noticed that I, sometimes, want to know what has changed.

Results:

Code:
% ./gitup.sh 

Updating Git repository: [REDACTED]/linecook

     - New [2020-10-13] : Merge pull request #3 from mortie/patch-1

     - New [2020-10-13] : Fix the links in the table of contents

Updating Git repository: [REDACTED]/ion

     - New [2020-09-03] : Fix set and status argument handling

Updating Git repository: [REDACTED]/knoxite

     - New [2020-10-12] : Set further default values when executing `knoxite config alias...`

     - New [2020-10-12] : Include some default toml-comments in the struct tags of RepoConfig

     - New [2020-10-12] : Switch to pelletier/go-toml as toml library for the config system

     - New [2020-10-12] : Display the error message too when saving the config fails in tests

Successfully processed 14 repositories.

I have noticed that it takes a while to process 14 repositories. I consider a rewrite in an actual programming language - shell scripts don’t work well with threads. (I tried playing with the parallel command, but it makes the code hard to read.)

--
<mort> choosing a terrible license just to be spiteful towards others is possibly the most tux0r thing I've ever seen

	mcol Offline \| 14-10-2020, 03:19 PM \| #5

Looks useful!

Along a simlar vein is gita which keeps a list of your local git folders and can perform tasks on all of them, much like you do. I assume gita would have good error handling -- for example, if you want to pull all of your repos, but one of them needs merging or something. Although I guess this isn't a problem you have, if you only have the clones around for use rather than development.

	jkl Offline \| 14-10-2020, 07:19 PM \| #6

I use my script only for repositories which I use (= tools in my $PATH).
gita seems useful as well, but I could imagine that, given that it is mostly written in Python, its performance is even worse than mine.

--
<mort> choosing a terrible license just to be spiteful towards others is possibly the most tux0r thing I've ever seen

	mcol Offline \| 15-10-2020, 09:03 AM \| #7

Yeah, very possible.

	Halfwit Offline \| 28-11-2020, 09:00 PM \| #8

I did ~/local for all my xdg dirs, so ~/local/cfg was under git revision, it was entirely painless. Suggestions if you do: gitignore *, and git add -f whatever

View a Printable Version