What do you use as a system monitor

What do you use as a system monitor - Printable Version
+- nixers (https://nixers.net)
+-- Forum: Operating Systems & Administration (https://nixers.net/Forum-Operating-Systems-Administration)
+--- Forum: Servers Administration, Networking, & Virtualization (https://nixers.net/Forum-Servers-Administration-Networking-Virtualization)
+--- Thread: What do you use as a system monitor (/Thread-What-do-you-use-as-a-system-monitor)

What do you use as a system monitor - venam - 10-02-2022

Hello nixers,
Here's a simple thread: What do you use as a system monitor and do you have recommendations.

I normally go for htop and top but I'm looking to broaden my horizons.

Be sure to mention the OS it runs on (if only a particular one), and provide a link.

RE: What do you use as a system monitor - neeasade - 10-02-2022

I use htop. as far as "finding out when shit is whack", I monitor--

- temp and battery in a systemd user service: https://github.com/neeasade/dotfiles/blob/79ee41f404c62630809936c9bedb6a1f729240d1/wm/.wm/services/sensor_alert

- disk space via a section on my panel that pops up: https://github.com/neeasade/dotfiles/blob/79ee41f404c62630809936c9bedb6a1f729240d1/wm/.wm/panel/lemons/misc#L4-L20

I guess it might also be nice to be alerted when there's high IO or cpu usage or something (over some time period/non-bursty)

RE: What do you use as a system monitor - movq - 12-02-2022

htop here, too. It can usually answer all of my questions.

One of the things it can't do is show network usage. For my Linux desktop/laptop, I just have a widget in my status bar (along with CPU usage and such). It's surprisingly rare that I need to show network usage on a server in real time (we have long-term statistics using influx/telegraf). When I do, I usually went for iftop. I just found bmon which has a slightly nicer UI and doesn't need root.

(10-02-2022, 10:08 PM)neeasade Wrote: temp

That's interesting. Why do you need an alert when CPU temp goes up? Cooling issues?

RE: What do you use as a system monitor - neeasade - 12-02-2022

> Why do you need an alert when CPU temp goes up? Cooling issues?

It was for an old thin thinkpad I had that probably needed thermal paste re-applied -- Positioning and a cooler could matter a lot, and so I needed something to nudge me to keep that in mind

edit: I was also QUITE mobile at the time for context. coffee shops, conference rooms, a friends living room, different every day.

RE: What do you use as a system monitor - seninha - 12-02-2022

On my OpenBSD desktop, I configured apmd to log every minute at /var/log. I also configured sensorsd.conf to set a high temperature value that, when passed, a message should be logged.
Then, I run an awk script at X login that reads apmd log (from tail -f) and writes a notification to xnotify's fifo when a interesting line is read.
It currently only greps for lines reporting low battery and high temperature (see the image below, at top right).

For my dock/bar I use WindowMaker-style dockapps, in particular wmbubble, which compiles some CPU and memory info as a duck swimming on a pool of bubbling water. When a mouse is over it, wmbubble shows the load average graph.

[Image: mon.png]

RE: What do you use as a system monitor - venam - 13-02-2022

On that note, I also forgot I have a script for battery status that pops up at certain intervals.
It's running as a systemd user timer (~/.config/systemd/user/timers.target.wants).
The timer unit

Code:
[Unit]

Description=Timer to run battery-notify

[Timer]

OnActiveSec=1min

OnUnitActiveSec=1min

[Install]

WantedBy=timers.target

and service unit:

Code:
[Unit]

Description=battery-notify

[Service]

ExecStart=%h/docu/bin/check-batt

[Install]

WantedBy=default.target

And the actual script that gets called check-batt, which is very similar to what neeasade is doing but with threshold values:

Code:
#!/usr/bin/env bash

BAT_NUM='1'

STATUS_FILE=/tmp/check-bat.tmp

BATTPERC=$(cat /sys/class/power_supply/BAT${BAT_NUM}/capacity)

BATTSTATUS=$(cat /sys/class/power_supply/BAT${BAT_NUM}/status)

LAST_STATUS=''

LAST_NOTIF_LEVEL=0

NOTIF_LEVEL=0

if [ -e $STATUS_FILE ] ; then

    . $STATUS_FILE

fi

save_status() {

    LAST_STATUS=$BATTSTATUS

    echo "LAST_STATUS=${BATTSTATUS}" > $STATUS_FILE

    echo "LAST_NOTIF_LEVEL=${NOTIF_LEVEL}" >> $STATUS_FILE

}

notify() {

    level=$1

    msg=$2

    DISPLAY=:0 /usr/bin/notify-send -t 3500 -u "$level" "POWER" "$msg"

}

# Battery status changed

if [[ "$BATTSTATUS" != "$LAST_STATUS" ]] ; then

    notify "normal" "battery is now $BATTSTATUS, Capacity: $BATTPERC"

    NOTIF_LEVEL=$LAST_NOTIF_LEVEL

    save_status

    exit

fi

if [ "$BATTSTATUS" = Discharging ] && [ "$BATTPERC" -le 10 ]; then

    NOTIF_LEVEL=10

    notify "critical" "battery is less than 10% - Connect charger, going to sleep in 10s"

    sleep 10

    BATTSTATUS=$(cat /sys/class/power_supply/BAT${BAT_NUM}/status)

    if [ "$BATTSTATUS" = Discharging ]; then

        notify "critical" "going to suspend now"

        sleep 3

        logger "Critical battery threshold, suspending"

        save_status

        systemctl suspend

        exit

    else

        notify "normal" "battery is now charging, cancelling suspend"

        save_status

        exit

    fi

fi

if [[ $BATTPERC -le 20 ]] ; then

    NOTIF_LEVEL=20

    notify "critical" "battery $BATTSTATUS is less than 20%"

    save_status

elif [[ $BATTPERC -le 30 ]] ; then

    NOTIF_LEVEL=30

    if [ "$NOTIF_LEVEL" != "$LAST_NOTIF_LEVEL" ]; then

        notify "normal" "battery $BATTSTATUS is at 30%"

        save_status

    fi

elif [[ $BATTPERC -le 50 ]] ; then

    NOTIF_LEVEL=50

    if [ "$NOTIF_LEVEL" != "$LAST_NOTIF_LEVEL" ] ; then

        notify "low" "battery $BATTSTATUS is at 50%"

        save_status

    fi

elif [[ $BATTPERC -le 60 ]] ; then

    NOTIF_LEVEL=60

    if [ "$NOTIF_LEVEL" != "$LAST_NOTIF_LEVEL" ] ; then

        notify "normal" "battery $BATTSTATUS is at 60%"

        save_status

    fi

fi

RE: What do you use as a system monitor - pfr - 14-02-2022

(12-02-2022, 09:36 PM)seninha Wrote: On my OpenBSD desktop

Man, I love your desktop. Everytime I see it I'm intrigued. I wish you would just fork a distro or release a pre-configured live image. It's so polished and consistant. Harmonious even.

Anyway, I keep saying I'm going to use shod once I have a desktop setup (with a mouse). Eventually!

Back to the topic at hand, I quite like nmon as a system monitor.

RE: What do you use as a system monitor - VMS - 15-02-2022

(13-02-2022, 05:49 AM)venam Wrote: On that note, I also forgot I have a script for battery status that pops up at certain intervals.

I do have one too on NetBSD, and the concept is somehow similar:

Code:
#!/bin/sh

    CHARGE=$(/usr/sbin/envstat | grep charge: | sed 's/.*(\(.*\))/\1/')

    BATT=$(echo $CHARGE | awk '{print $1}')

if envstat | grep -Eiq 'charging.*true'; then

  STATUS=charging

else

  STATUS=discharging

fi

notify-send "$(echo -e "battery = $BATT \nstatus  = $STATUS")"

It relies on envsys(4) (Environmental Systems framework), which was written by the creator of Void's XBPS and can print a lot of useful info from monitor devices and sensors in quite a fancy and human-readable fashion.

Usually I bind this to something like Mod4 + b in CTWM.
Similarly, I bind Mod4 + v to:

Code:
#!/bin/sh

VOL=$(mixerctl -n outputs.master | cut -d , -f 1 | \

     awk '{print $1/254*100}' | cut -d . -f 1)

notify-send "volume: $VOL %"

In my ~/.profile I have this little function to monitor power-management related events. I usually have a sticky term running it in foreground.

Code:
pwrd()

        {

             ( tail -f -n0 /var/log/messages & ) | sed -n -e 's/^.*sensor_battery: //p' 

        }

As a pseudo graphical resource monitor, I really love systat; the interface might look a bit ugly by modern standards, as it relies on the native curses library, which is more limited compared to ncurses, but it does its job and does it well. systat can display various statistics at the same time including cpu load, virtual memory, processes, swap usage, disk i/o, open sockets, established connections, network traffic...for example `systat iostat` looks like that:

To quickly display the number of total used buffers in megabytes, I also use this script; useful on server and public access BSD systems like SDF or GREX.

Code:
#!/bin/sh

vmstat -s | awk '

 / bytes per page$/ { bpp = $1 }

 / cached file pages$/ { cfp = $1 }

 / cached executable pages$/ { cep = $1 }

 END { print((cfp + cep) * bpp / 1024 / 1024); }'

As for graphical tools, I really like a tool ported from old versions of SPARC32 Solaris and called xmeter. I was unable to find a description to link here, but, from the man page:

Quote: Xmeter displays a periodically updating histogram of the system
statistics gathered by rstat(3) for the specified hosts. Meters can be
displayed in a vertical, horizontal or rectangular arrangement. As
statistics range between 4 user defineable levels (OK, WARN, ERROR or
FATAL), the background, foreground, highlight, border and internal
border colors, and the background bitmap of each meter can be changed.

rstat is a daemon capable of returning performance statistics obtained from the kernel. These can be read using the rup(1) command. rstat is a Sun RPC protocol, and I suppose that's why I was unable to find a port of xmeter for Linux.

RE: What do you use as a system monitor - pyratebeard - 18-02-2022

I use the same battery script as venam, and I also use htop(1).

Another tool that I quite like is glances.

RE: What do you use as a system monitor - jkl - 21-02-2022

I’m perfectly fine with the built-in tools. I noticed that the additional overhead is just not worth it.

RE: What do you use as a system monitor - prx* - 24-02-2022

Hi,
On OpenBSD, I use this little script to output in tmux status or dwm's bar various info : mpd current song, battery remaining, audio volume, date. There are also functions for disk usage, cpu and ram that I actually don't use (top/vmstat/systat are good for this)

[Image: oKVt.png]

Code:
fs_perc()

{

    df -h $1 | awk 'NR==2 {print $5}'

}

get_vol()

{

    sndioctl output.level | awk -F = '{printf "%d\n",$2*100}'

}

ram() {

    top | awk '/Memory/ { print $3 }'

}

loadavg()

{

    #uptime | awk -F'[a-z]:' '{print $2}' | tr -d ' '

    sysctl -n vm.loadavg

}

bat()

{

    apm | awk 'NR==1 && !/absent/ {printf "%s", $4}'

    #test $(apm -a) -eq 0 && printf "⚡%s%%" $(apm -l)

}

mpdstatus()

{

mpc | awk '

(NR == 1) { 

    status = $0

    if (length(status) > 42) {

        status = substr($0, 1, 42) "..."

    }

}

(NR == 2) && /playing/ {

    isplaying = 1

}

END { if (isplaying) { printf "[%s]", status } }

'

}

temp()

{

    sysctl hw.sensors | awk -F"=" '{print $2; exit}'

}

printf "%s / ⚡ %s / 🔈 %s / %s" \

     "$(mpdstatus)"\

     "$(bat)" \

     "$(get_vol)%" \

     "$(date "+📅 %Y-%m-%d / 🕓 %H:%M")"

RE: What do you use as a system monitor - yakumo.izuru - 17-04-2022

I'm usually using htop and relying on conky for the rest, which is mainly CPU load, RAM usage, assigned IP address, battery charge[*], uptime[*] and date/time[*]

Also handled by tmux via apm(8), get_uptime.sh (a script I wrote), and the default date/time setting

RE: What do you use as a system monitor - maksim - 29-09-2022

I use Btop, it is an awesome TUI utility with a good UI. I also made an Everforest theme for it, which is already inside of Btop.

RE: What do you use as a system monitor - gaak - 31-10-2022

(12-02-2022, 09:36 PM)seninha Wrote: On my OpenBSD desktop, I configured apmd to log every minute at /var/log. I also configured sensorsd.conf to set a high temperature value that, when passed, a message should be logged.

This for everything. When the current state is nominal, nothing should be displayed.

Getting everyone to agree on a message bus is not easy. eg. battery low/high goes to X11 display, while the daily security report goes to email.

Code:
#!/usr/bin/awk -f

BEGIN { max_disk = 0.950 ; min_ifree = 2000; max_load = 1.0; }

/^$/ { is_disk = 0; is_network = 0; }

/^disks:/ { is_disk = 1; next; }

/^network:/ { is_network = 1; next; }

# remove date-time as that breaks duplicate output checking

/^OpenBSD/ {

    for ( i = 4 ; i <= NF; i++ ) {

        $i = "";

    }

}

/load averages:/ { 

    if ( $NF > max_load ) { print; }

    next;

}

! is_network && ! is_disk { print; }

is_network {

  name = $1; colls = int($NF); ofail = int($(NF-1)); ifail = int($(NF-3));

  if ( colls > 0 || ofail > 0 || ifail > 0 ) {

    printf("net %s input errs %s output errs %s collisions %s\n", name, ifail, ofail, colls);

  }

}

is_disk {

  name = $NF; total = int($2); used = int($3); ifree = int($7);

  if ( name == "on" ) { next; }

  if ( used / total > max_disk ) {

    printf("disk %s at %.2f used\n", name, used / total);

  }

  if ( ifree < min_ifree ) {

    printf("disk %s ifree at %d\n", name, ifree);

  }

}