nixers - Servers Administration, Networking, & Virtualization

What do you use as a system monitor

Thu, 10 Feb 2022 16:22:03 +0000

Hello nixers,
Here's a simple thread: What do you use as a system monitor and do you have recommendations.

I normally go for htop and top but I'm looking to broaden my horizons.

Be sure to mention the OS it runs on (if only a particular one), and provide a link.

The Hosting Thread

Tue, 20 Apr 2021 18:15:32 +0000

This is a thread to discuss various hosting topics, including:
* Where do you host? Self-hosting from home? Colo? VPS? "Cloud"?
* What do you host? www/webapps, email, gopher, git, IRC...
* Why do you host your own services, and why did you choose the hosting platform you use?

I can't lie - this is somewhat selfish, because I'm looking at options for moving away from my current hosting platform. But I'm also curious about what everyone is doing - nixers folks are always working on cool projects that need to live online somewhere!

As for me, I run a t3.nano instance in AWS running a webserver. I also connect to that box for IRC. I also have access to my previous employer's webhosting platform because they offer lifetime hosting for anyone who's worked there - so I don't need to run my own mailservers for my domain, thankfully, and I trust them enough for now.

Previous related discussion

$HOME FHS & init for user-oriented systems

Wed, 09 Dec 2020 17:54:37 +0000

Hello.

I am thinking to rework how I manage my $HOME and user sessions, notably, I want to have things there handled with my beloved runsvdir(1).
Examples of daemons/applications that one would maintain alive: MUA (like claws-mail), IRC client, music player, urxvtd, window manager (i3).
This list is not complete, and I think many of my activities could/should be done by real daemons with which I'd interact "normally" (I'm thinking to build a daemon to register songs I like which come from mpd via netradios, for example).

Some of the problems I foresee here:

* management files (daemon status) should be in a tmpfs (persistance here would be annoying at best), yet only root can do that. I could just mkdir in /tmp, but that's quite dirty and fragile imo. /dev/shm would be a better place, but it's not portable. /run is owned by root, so no way here neither.

* starting multiple sessions might launch multiple runsvdir, will make things a mess. Can probably be worked around a way or another... Stopping a session poses the same problems. I would prefer to avoid using elogind, to rely on simpler tools. I see several places on which I could work here: runsv works with "constructors" and "destructors" to reuse the C++ terms, so maybe I could use utmp related tools here?

* make my $HOME a worse mess than it already is.

* processes escape their parent, notably i3's children. That's really annoying... maybe can be solved using namespaces on linux? I'm not really sure what those are, but chpst(1) proposes a way to start the command in a new process group, could that avoid the problem?

So, how do you guys handle those problems? What's your approach on them?

File extensions

Thu, 12 Nov 2020 17:57:24 +0000

For a very long time I thought that .txt or .exe extensions are Windows world heresy and have no real use in GNU/Linux or Unix-like world. But then relativelly recently, I know it's stupid - but I realized that it does provide some value, for example if you want to move all text files from a mixed file directory, you'd just "mv *.txt", while without an extension it would take more actions, piping from file or something like that - more typing, more complexity. What do you think? Do you create all your text files with .txt?
Another one are shell scripts. Most of us are probably using .sh in the name, but you know that in GNU/Linux /bin/sh is a symlink to /bin/bash in 99% of distros. So we are not using Bourne shell, or something strictly POSIX compliant, we are using BASH, I thought, why not using .bash for bash scripts, and .sh for posix compliant ones. What do you think? Am I missing something which would make it dumb, or incompatable somewhere? It should be even better for Unix/Linux cross usage. Why isn't it a standard?
Most interesting are binaries, .exe or .exec could also be useful in some ways, but I don't think that renaming all your system binaries would be very smart idea :)
So names and extensions can provide valuable info, or ease of use, on top of what we already have in form of extensions. Your thoughts?

Android server edition

Tue, 30 Jun 2020 13:55:08 +0000

Hello there fellow nixers,

We are living weird times, and I need to run an android application in "server mode". That is, run all the time, as a service if possible.

I know that I'm asking a lot, and I think the closest I can get for that is an android VM running "headless" , and managed from the cli via `adb`. My servers are running OpenBSD right now, and it seems that vmd can boot android x86 iso. I'm wondering if any of you had tips on how I would go about doing that, or if it's even possible ?

I found the following: https://github.com/tulir/mautrix-whatsap...d-VM-Setup

Which is exactly what I want to do. But it uses the qemu version shipped with the android sdk, and I'm fairly sure I can't use that on OpenBSD.

Time and Time on Unix

Sat, 02 May 2020 10:41:19 +0000

Hello nixers,
I've got myself to put the research about "Time on Unix" together. It's the culmination of the series we've had in the newsletter called "Keeping time and date".

The research has got me down a rabbit hole, so brace yourself for a more or less complete overview of all that you need to know about time on Unix.

I couldn't post it on the forums because of the size of the article, so here it is:

Link to the Time on Unix article.

Remote work solution

Wed, 24 Apr 2019 05:37:34 +0000

Hello nixers,
What's your remote work solution.

What do you use to connect to a machine far away, do your deeds, get things done.
Be it a homemade VPN solution, simply SSH with a terminal multiplexer, or mosh, maybe you have a jump host. Do you use VNC, or X11 forwarding, or thin clients, etc..

Also in which cases do you use those, day job scenario, or home scenario. Does the distance and the connection to the server change your remote workflow.

As for me, I normally use SSH with X11 forwarding for specific applications that are graphical, all within tmux to not accidentally close the terminal. I'm slowly starting to get on the mosh train too.
At work I'm not in the operation team, I'm a developer, and so I only barely use SSH to test things on a test-environment server.
However, I can say that the op team uses a jump-node/firewall where every member has a specific account with ACL to specific credentials that will let them SSH into other remote machines. There are many situations where the connection is extremely slow because of multiple VPN consecutive jump points. And sometimes they are forced to use a graphical environment.

So as far as I'm concerned I'm not very knowledgeable about the topic, I've got a simple solution. I'm really interested in knowing about more complex scenarios.

So what about you nixers, how do you work remotely?

To reboot or not to reboot

Thu, 18 Apr 2019 04:53:41 +0000

Hello fellow nixers,
In this thread we'll discuss an old myth that we've actually never brought as a discussion on the forums yet: To reboot or not to reboot.

It is often said that rebooting to solve an issue is the equivalent of giving up. It might be the norm on some other operating systems but on Unix-like OS with users that should intimately know their machine it's regarded as heresy.

having a high uptime is looked at as an epic tale. A show off of stability and rigidness.

https://hardware.slashdot.org/story/13/0...-of-uptime
https://web.archive.org/web/201806232200...cs/uptimes
https://www.infoworld.com/article/262344...boxes.html

There's even a command uprecords for keeping track of reliability.

However, sometimes, especially when the machine runs as a development one and not a server, it seems mandatory to reboot. Let's say for a kernel upgrade or full system upgrade. Or init system upgrade. Or hardware upgrade. Or even in some cases some kernel modules update (if the system doesn't have a dynamic way to load those).

Still even when updating the kernel some OS, for example Linux, allow to update/patch the kernel without rebooting via features such as kexec.

So here we go:
Nixers, what's your opinion on this topic of rebooting. Do you reboot often, on which type of machines. Do you have any special tricks, any story to share about a time you rebooted and it was worthless, or a time when you actually never found out why the issue was happening in the first place.

My take on the topic. I usually avoid rebooting unless there's a kernel update. I have uptimes going around a 100 days in a row.
On my server I actually almost never reboot.

Code:
00:27:49 up 391 days, 10:18,  1 user,  load average: 0.00, 0.00, 0.00

Code:
00:42:28 up 359 days, 11:43,  1 user,  load average: 0.08, 0.03, 0.05

So on both of my two servers I haven't rebooted in around a year.

When the service is running fine I think there's not much to worry about unless there's an important security update to do on the kernel.

I reboot most often, on my home machine and work dev machine, when I run into the kind of issues that make me pull my hair off and after trying everything else and that didn't work.

My latest story was about a card reader that wasn't being recognized by the official proprietary driver. For those who have no clue about card readers on Unix, they work through a daemon called pcscd that will match the appropriate driver (/usr/lib/pcsc/drivers/ for instance) depending on the device vendor ID and product ID. There are some drivers that are open that can come from the package manager or along with other tools such as opensc.
In all cases, the reader I wanted to try wasn't recognized by pcscd even after following all I thought was right. So I gave up, thought something might have been messed up in udev or something that I didn't caught.

But still after rebooting there was nothing.
Oh no the uptime went away!

Digging further in the issue. Checking dmesg for the venderID and productID that was being reported and trying to match it with the Info.plist that should contain what points to the proprietary driver, I found out that the IDs aren't even listed. I tried adding them but in vain, the driver doesn't support the device. Apparently they forgot to update their Linux driver.

So I ended up using another card reader (acr38) that's widely supported.

Reboot lesson learned.

So what do you think about rebooting and how frequently it should be done.

Network Managers

Tue, 16 Apr 2019 15:18:56 +0000

Hello nixers,
Which network manager do you use and is there any reasons for it.

There are a lot of options out there to configure wifi, ethernet, dns, ip, vpn, proxies, etc.. all in one place. We call the softwares handling this network managers.

You can do it manually by using wpa_supplicant, or ip, or whatnot.
You can use a terminal cli or curses such ass wifi-menu, nmcli, or nmtui.
You can use a graphical network manager such as the one that comes with the gnome desktop environment or the one that comes with KDE plasma.

There's more info on the Arch wiki however I'm looking for the reasons you use what you use.

On my side, I use wifi-menu which is a simple curses wrapper over /etc/netctl (netctl) and wpa_supplicant. This only manages wifi and doesn't allow much configuration. As for ethernet I do it manually.

So nixers, what network managers do you use?

Surfing the IPv6 turbo wave

Thu, 28 Mar 2019 20:01:08 +0000

Salutions amis nixers!

I finally decided to take my online experience to the next level and setup this very old new-internet stuff called "Internet Protocol version 6" !

Awesome right?

This is all new to me though, and it seems like a huge part of what I know about ipv4 will be totally useless, and I'm looking for advices and best practices for a good ipv6 setup.

My current server provider is handing a generous /48 range, which I can split in as many /56 as I have servers (4 actually).
Each server can then be delegated a prefix via DHCPv6, and I assign an IP in this delegated prefix to my server so it is then reachable over the new internet.

My question here is: what the hell should I do with all these IP addresses?!

Am I supposed to keep the /48, delegate it to one server and then use this server to assign IPv6 addresses to the other servers? This is unlikely because it creates a single PoF...

Should I just accept the fact I'm creating one sub-prefix per server, each prefix having only one address in it?

Is that how ipv6 works? You just waste millions of IP because nobody cares and we'll never run out of it?

How do you guys go with that new tech?

Scripts to be "quick and efficient"

Sun, 02 Sep 2018 17:35:37 +0000

Hello fellow nixers,
In this thread let's share our opinions on those brand new tools for "efficiency" that pop-up so often on tech-news websites.

I'm referring to scripts aliasing, prettifiers, curses wrappers, helpers of all sorts, fancier interfaces over old tools, etc..
Especially tools that are made to make other tools more usable or prettier.

Tools and scripts such as:

autojump
bashmarks
lazygit
bat
prettyping
colordiff
httpie
and much much more, you get the idea

Here's my stand on this topic:

I don't usually install any of those fancy wrappers for efficiency purposes because I keep switching between machines while working and I can't be bothered to install one in a place and not the other. I'd rather stay vanilla.
However, I still have maybe two or three color wrappers such as colordiff that I use quite a lot before sending patches or pushing online my changes.
Tools are tools in the end, if you can extend your environment to fit what you need then it's for the best. CLI are also easy to write so it let's anyone contribute with new tools.

Share what you think of tools wrappers and prettifiers, and don't forget to mention some of them too.

Sharing Media

Sat, 25 Aug 2018 07:42:01 +0000

Hello fellow nixers,
In this thread let's share how we share files and media with others.

In relation with the entry of last week newsletter "Dropbox is dropping support" in issue 89. I thought of opening the topic of how everyone handles file sharing, especially media files. It's not uncommon to find ourselves in a situation where we have to sync or at least give to someone else a file that resides on our machine, so what do you do in that scenario.

from Dropbox, to rsync and ftp, to SAMBA shares, to NAS (network attached storage), to putting files on your http web server index and display them prettily with h5ai, to centralized home media center on raspberry pi like Kodi or Xbian or others, to any other media center solution, to a mini uploader like paste.xinu.at , to sharing physical media like USB and DVDs, to your own solution.

In my case I use a simple uploader paster for quick small files and when it comes to big files I use a physical medium, a harddisk or USB usually, as my internet speed isn't able to send enormous files. The disadvantage is that I can't share files I would like to share with people that are not next to me.

What machine or device do you use to share media and what is your solution, how do you handle this, what are the downsides to your solution?

Data Storage On Unix

Sun, 15 Oct 2017 15:50:15 +0000

My very own mails

Thu, 20 Jul 2017 22:00:21 +0000

I've been trying to setup my very own mail server for quite some time now. This has many benefits: cool domain name, 100% control, 100% privacy, etc...

But I've been struggling with that for way longer than my sysadmin title would let me admit.
I'm asking for help here, in case someone ever did it successfully in the past. My case is pretty common I think, so here is the context:

One server, with one postfix instance
One "main" domain: z3bra.org
One virtual domain: domain.alt
Ability to send mails from both domains

So far, sending mails from the main domain works great. My real problem comes from the virtual domain, as I can see that the mails leaving my server, but they either get bounced, spammed or even dropped without notice.
Google flags me as SPAM without giving a reason, and I managed to get the following error message from mail.com:

Quote:host mx01.mail.com[74.208.5.22] said: 550-Requested
action not taken: mailbox unavailable 550 invalid DNS MX or A/AAAA resource
record (in reply to MAIL FROM command)

All MX records for these domains are correct, and point to mail., which itself points to the IP address of my server.
The PTR record for this IP points to mail.z3bra.org. I tried adding another PTR record for the virtual domain, but it's still propagating and I'm not sure an IP can have multiple PTR records.

For what matters, all SPF records are ok for both domains (even goggle acknoledged it).

I'm now running out of ideas to get my mails delivered. This looks like an incorrect DNS setup, but I'm not sure what should be done at this point....

Please someone tell me you have an idea. I'm fairly sure the solution is simple!

Processes On Unix

Sun, 16 Jul 2017 09:50:47 +0000