+- nixers (https://nixers.net)
+-- Forum: Operating Systems & Administration (https://nixers.net/Forum-Operating-Systems-Administration)
+--- Forum: GNU/Linux (https://nixers.net/Forum-GNU-Linux)
+--- Thread: Musl Distros (/Thread-Musl-Distros)
Pages:
1
2
Musl Distros - xikuuky - 27-12-2015
Hey guys! I really like Musl but I'm having trouble finding good distros that support it. I really liked Alpine but I can't get it to boot. Any ideas? I would prefer stable distros but if a experimental distro is stable enough that's okay too. Can Crux be a Musl system?
Thanks! - Xikuuky
RE: Musl Distros - xikuuky - 27-12-2015
UPDATE: I think I am going to make a distro that fulfills my needs. It will probably run inside another system using Busybox/Toybox and Musl. Maybe with something like (B,C,H)LFS?
RE: Musl Distros - Houseoftea - 27-12-2015
(27-12-2015, 06:02 AM)xikuuky Wrote: make a distro
Keep us updated also please see the thread: https://nixers.net/showthread.php?tid=1741
There are others doing similar things to you, I know z3bra posted in the other 'smallest distro' thread but he was messing around with musl and the like, you should talk to him about this.
Thought I should add this:
http://wiki.musl-libc.org/wiki/Projects_using_musl
List of all the projects using musl, lots of distros on there you can pull inspiration from
RE: Musl Distros - sth - 27-12-2015
void! stable, rolling release, actively developed, musl-flavor.
https://voidlinux.eu
RE: Musl Distros - z3bra - 28-12-2015
Alpine is not experimental, it's definitely stable (My personnal server runs alpine, and I never ran into a problem.
Voidlinux has a musl flavor as sth said.
For crux, musl isn't supported, and if you go down this route, you will certainely have to patch a bunch of softwares.
RE: Musl Distros - xikuuky - 28-12-2015
(27-12-2015, 01:41 PM)sth Wrote: void! stable, rolling release, actively developed, musl-flavor.I already use Void. A bunch of the packages on it can be jokes for Musl, I build everything by hand.
https://voidlinux.eu
(28-12-2015, 04:22 AM)z3bra Wrote: Alpine is not experimental, it's definitely stable (My personnal server runs alpine, and I never ran into a problem.Alpine does not like install medium (a very old 16GB drive and I have no CD drive or CD's.
Voidlinux has a musl flavor as sth said.
For crux, musl isn't supported, and if you go down this route, you will certainely have to patch a bunch of softwares.
(27-12-2015, 12:55 PM)Houseoftea Wrote: Keep us updated also please see the thread: https://nixers.net/showthread.php?tid=1741I can do that! :) I'll look at that.
There are others doing similar things to you, I know z3bra posted in the other 'smallest distro' thread but he was messing around with musl and the like, you should talk to him about this.
Thought I should add this:
http://wiki.musl-libc.org/wiki/Projects_using_musl
List of all the projects using musl, lots of distros on there you can pull inspiration from
Here, an update.
==================
Looked at many proejects that use or will use musl plus a lot of lightweight distros and I have a bit of a idea on how I will accomplish this. I'm going to try LFS then BLFS, and eventually ALFS (unless people can just install from what I give them [doubt it but possible]).
For packages I will supply a package manager I wrote myself that builds packages from git using shell scripts. It is a less flexible system but it works for what I'll do. We are going to probably be 64bit exclusive (sorry ARM, x86, and MIPS users). If it isn't hard to cross compile, I may add x86. I'll have the base system preinstalled hopefully and I'll hopefully be able to update it with ease. Make + autotools will install, uninstall, and configure software. The package manager runs on Git and Make so it works perfectly. Users will have a .software or .packages or ~/Code/Soft(ware) area that holds the source of the packages. I'd like feedback. Thank you!
Also, I'll include Toy/Busybox hopefully and have bash and zsh installed by default and scripts already on the machine to build things like Python, Python2, Vim, and the such. :)
I looked at Solyste and I like the idea of mksh as the default shell. In my experience that is very fast. Also, how hard is it to compile X?
RE: Musl Distros - z3bra - 28-12-2015
xikuuky Wrote:For packages I will supply a package manager I wrote myself that builds packages from git using shell scripts. It is a less flexible system but it works for what I'll do.
"Wrote"? Did you write anything yet? 'cause I'd be interrested in reading what you've done so far. I'm currently working on a package manager myself I already have a "working" solution using shell script, but I'm not satisfied with it, so I'm rewriting everything from scratch in C, focusing on different concepts.
xikuuky Wrote:We are going to probably be 64bit exclusive (sorry ARM, x86, and MIPS users). If it isn't hard to cross compile, I may add x86.
x86 is a CPU architecture, which uses either 32 or 64bits. So you're probably opposing x86_64 to x86 there. Beware the wording is such a pedantic forum ;)
xikuuky Wrote:I'll have the base system preinstalled hopefully and I'll hopefully be able to update it with ease. Make + autotools will install, uninstall, and configure software. The package manager runs on Git and Make so it works perfectly. Users will have a .software or .packages or ~/Code/Soft(ware) area that holds the source of the packages. I'd like feedback. Thank you!
It's the simplest way to go indeed, and if softwares were done properly, we wouldn't need package managers. The problem is that not all software use a makefile, and even if they do, not many propose the "uninstall" target.
This is why package manager were created in the first place, they track files intalled by each package, and allow the user to easily remove/update them. It later handled package dependency, which became an issue when dynamic linking started to kick off.
You should take a look at how other people solved those problems. here are a few links to get you started:
- http://git.z3bra.org/cgit.cgi/pkit/tree/README
- https://github.com/dimkr/packlim
- https://github.com/radare/slpm
- http://www.linuxfromscratch.org/lfs/view/development/chapter06/pkgmgt.html
xikuuky Wrote:Also, I'll include Toy/Busybox hopefully and have bash and zsh installed by default and scripts already on the machine to build things like Python, Python2, Vim, and the such. :)
Choosing the default set of packages is always a hard task ;)
I'd personally go with the bare minimum: init, device manager, shell, coreutils, netutils and that's all. I'll certainely have to add the package manager and curl to the live usb though, to make bootstraping easier.
xikuuky Wrote:how hard is it to compile X?
really hard if you need to package everything yourself.
RE: Musl Distros - xikuuky - 28-12-2015
(28-12-2015, 08:46 AM)z3bra Wrote: "Wrote"? Did you write anything yet? 'cause I'd be interrested in reading what you've done so far. I'm currently working on a package manager myself I already have a "working" solution using shell script, but I'm not satisfied with it, so I'm rewriting everything from scratch in C, focusing on different concepts.Sadly the code is gone but it just downloaded a script and executed it line for line. I found a way to easily uninstall stuff with make. Just type make -n install and it tells you what stuff will be modified. Maybe I'll go without a package manager or I'll just get something like emerge on here(go ports! :) ). I really like just compiling everything.
(28-12-2015, 08:46 AM)z3bra Wrote: x86 is a CPU architecture, which uses either 32 or 64bits. So you're probably opposing x86_64 to x86 there. Beware the wording is such a pedantic forum ;)Thank you so much! Now I won't look like an idiot. o/ :)
(28-12-2015, 08:46 AM)z3bra Wrote: It's the simplest way to go indeed, and if softwares were done properly, we wouldn't need package managers. The problem is that not all software use a makefile, and even if they do, not many propose the "uninstall" target.I'll definitely go through this list. Thank you so much buddy! :) I'm new to C[++,go] and the such (I can read it just fine) so it would be a big deal if you would read my source code for me. I want to use a compiled language for speed (Python is fast but C can be faster).
This is why package manager were created in the first place, they track files intalled by each package, and allow the user to easily remove/update them. It later handled package dependency, which became an issue when dynamic linking started to kick off.
You should take a look at how other people solved those problems. here are a few links to get you started:
(28-12-2015, 08:46 AM)z3bra Wrote: Choosing the default set of packages is always a hard task ;)I think I'll do bare minimum plus a little extra. :) Extra being a few shells and make, autotools, and musl. What is typically in netutils? What if I made the package manager the user's choice to install? I've seen installs like that (I once made a Ubuntu install use Emerge XD). I might include curl but then again, wget is a core package in both busybox/toybox. I use Curl personally but Wget is very popular.
I'd personally go with the bare minimum: init, device manager, shell, coreutils, netutils and that's all. I'll certainely have to add the package manager and curl to the live usb though, to make bootstraping easier.
As for init, I see the big options as: SystemD (not happening), OpenRC (I'm open to it), or Runit (my favorite). This is my distro so if I want to add Runit I can right? This is meant to be for my needs (and anyone with needs similar to mine).
(28-12-2015, 08:46 AM)z3bra Wrote: really hard if you need to package everything yourself.I am not packaging X. Packages, or even a package manager will (possibly) be left to the package manager the user installs.
UPDATE:
========
Elder Scrolls Online is still downloading. It is almost done so I'll hopefully get to boot into Linux tomorrow/tonight GMT+7 (Jakarta).
Decisions:
Init (Runit or OpenRC)
Minimal Core Utils (Busybox or Toybox)
?Package Manager? (user choice or I write one or I include one)
That's all.
RE: Musl Distros - Houseoftea - 29-12-2015
Init (Runit or OpenRC) :: runit
Minimal Core Utils (Busybox or Toybox) :: toybox
?Package Manager? :: packlim
RE: Musl Distros - z3bra - 29-12-2015
OpenRC is indeed a shitty service manager. I really can't see the hype behind it. The fact it's used in gentoo doesn't make it a good system, more people should understand this. Runit on the other hand implements way more good practices, even though some people reported issues with it when dealing with systems with a lot of services (especially with service restarting/failure detection).
For the core utils, you neither busybox nor toybox are POSIX compliant, making them poor alternatives to coreutils which is POSIX-compliant (but adds a shiton of useless/bloated features on top of their utils, making them unreliable and bug-prone).
There is the suckless base which is fairly good and POSIX-compliant. They also provide a way to build a sbase-box binary if that's what you really want.
Regarding the package manager, packlim is indeed fairly interresting, but I don't know how one would integrate it in another project than rlsd2.
I like the way it handle packages though. This result in a really simple format for packages, and it even has package signing.
I don't understand much why the author added a TCL interface though.
There is also no documentation for the build system, which seems interresting, but makes use of many different technologies, resulting in a big mess to me (makefile, shell scripts, TCL, json, ...).
RE: Musl Distros - Houseoftea - 29-12-2015
Packlad is another package manager that the guy made, but that ones written in all c
And spuk was interesting ;)
RE: Musl Distros - xikuuky - 29-12-2015
(29-12-2015, 10:55 AM)Houseoftea Wrote: Init (Runit or OpenRC) :: runit
Minimal Core Utils (Busybox or Toybox) :: toybox
?Package Manager? :: packlim
(29-12-2015, 01:48 PM)z3bra Wrote: For the core utils, you neither busybox nor toybox are POSIX compliant, making them poor alternatives to coreutils which is POSIX-compliant (but adds a shiton of useless/bloated features on top of their utils, making them unreliable and bug-prone).Runit for sure then. I have installed sbase on my own system and I like it. sbase plus ubase seems to be more complete of a full coreutils though. I couldn't get packlim to install. So update this is now:
There is the suckless base which is fairly good and POSIX-compliant. They also provide a way to build a sbase-box binary if that's what you really want.
Init: Runit
Minimal Core Utils: sbase
?Package Manager?: Still none. Maybe Butch
Could Pacman be a good package manager? Or would Crux or Gentoo's be better? :)
Also, checkout Sabotage Linux. Similair to what we/I are making. I'm pretty sure we could change it to fit our needs. It is a very small system. :)
RE: Musl Distros - z3bra - 29-12-2015
As I just discussed on IRC, I'm not making a distro for a community, or to fit people's needs. It's mostly for fun, and have something I built with my own hands.
Alone, I can't write a better package manager than pacman, apt or yum, I can't make an init system that will put systemd to shame, or a daemon manager as good as daemontools...
But I can make these softs. Make mine, and put them together to build a distro I can call my own. That's my goal. And I'll certainly fail at some point, just so I'll be able to do it again later :P
RE: Musl Distros - xikuuky - 30-12-2015
(29-12-2015, 03:03 PM)Houseoftea Wrote: Packlad is another package manager that the guy made, but that ones written in all cI will look into those. I didn't see those last night.
And spuk was interesting ;)
(29-12-2015, 05:11 PM)z3bra Wrote: As I just discussed on IRC, I'm not making a distro for a community, or to fit people's needs. It's mostly for fun, and have something I built with my own hands.I think I'll make my distro just for me too then (anyone can look at it, install it, fork it, whatever) that way I can make the perfect system I want to use! :)
Alone, I can't write a better package manager than pacman, apt or yum, I can't make an init system that will put systemd to shame, or a daemon manager as good as daemontools...
But I can make these softs. Make mine, and put them together to build a distro I can call my own. That's my goal. And I'll certainly fail at some point, just so I'll be able to do it again later :P
UPDATE
=======
I think the base for everything will be Sabotage Linux. I can easily write a package that would install my system into theirs. That makes life easy. :)
RE: Musl Distros - xikuuky - 30-12-2015
UPDATE
======
Stali is very similair and fits my needs. I may continue this as a hobby but for a more working distro please use Stali.
RE: Musl Distros - z3bra - 30-12-2015
For what its worth, I finally have a working package manager! It's dead simple, not finished and certainly has bugs, but it can install, list and remove packages!
http://git.z3bra.org/cgit.cgi/pm
RE: Musl Distros - Houseoftea - 30-12-2015
(30-12-2015, 09:13 PM)z3bra Wrote: For what its worth, I finally have a working package manager! It's dead simple, not finished and certainly has bugs, but it can install, list and remove packages!
http://git.z3bra.org/cgit.cgi/pm
hey z3bra I was playing around a little with pm and I couldnt get it working exactly.
I installed it, and pm -i (pack) works fine but pm -a pack:(pack) does not work at all.
am I doing something dumb?
RE: Musl Distros - z3bra - 31-12-2015
You can only install packages built locally. Said otherwise, tarballs. For example, run the following:
Code:
make
make DESTDIR=rootfs install
(cd rootfs; tar cz * > ../pm:0.1.tar.bz2)
pm -i pm\:0.1.tar.bz2
pm -a pm\:0.1.tar.bz2
pm -iThat will install the pack "pm" to the PACKAGE_ROOT specified in the source (this directory should already exist)
RE: Musl Distros - z3bra - 07-01-2016
I made some progress on my distro!
Here is everything I can do:
* build packs statically against musl (including pcc and binutils)
* sync packs from a remote repo into a local directory
* install packs in a chroot
* compile a "hello world!" with pcc from within the chroot
Here is a proof: http://raw.z3bra.org/dev/random/distro.webm
For the upcoming steps:
* fix bugs in the pack manager
* boot AGAIN an LXC container / on real hardware
* build packs from within the distro
* enjoy!
RE: Musl Distros - apk - 07-01-2016
(07-01-2016, 08:28 PM)z3bra Wrote: Here is a proof: http://raw.z3bra.org/dev/random/distro.webm
highly impressed, z3bra. im under the impression pm has no way of showing package updates, though?
Of course, a package manager that works with tarballs, thats extremely difficult if not impossible to do.
RE: Musl Distros - z3bra - 08-01-2016
For now, I decided to focus on add/delete/list only, as they are the most important actions.
Updates will definitely be handled though, and in a simple way. Here is what I have in mind:
if package version is different in the tarball name compared to what's installed, then:
0. remove file that are not in the tarball anymore
1. unpack files from the tarball, overwriting old file
2. update the "version" file in the metadata directory
For the two first step, the only condition would be that creation time == last modification time.
If the user modified it by hand, it means we shouldn't remove it, or replace. If it's the case, then two actions will be taken:
0. warn the user that the file shouldn't exist anymore and will not be removed in the future
1. write the new file as ${filename}.new and report it to the user
I can't think of why this process would not cover all cases :)
RE: Musl Distros - apk - 08-01-2016
Very cool indeed z3bra. Like I said, I'm really intrigued by your package manager, but I have to ask, would that process only be possible if the tarball(s) in question were already on the filesystem? Or does this mechanism work with the download location of the tarball? I've noticed that projects TEND to have predictable tarball locations (IE: most GNU software uses the same FTP/HTTP hierarchy), but this isn't true for a lot of software.
Honestly I'm really only honing in on this specific feature because a few years ago I attempted the same thing you have and I failed miserably.
RE: Musl Distros - z3bra - 15-01-2016
pm(1) only takes local files as arguments, it is not meant to deal with external repos of some kind.
I have written two small shell script, namely repo(1) and repogen(1) to deal with remote tarball fetching or whatever. I've listed some examples in pm's README, but for all the lazy asses in there (I know there's a lot of you!), here is an idea of how it works.
Let's say you have two machines, "server" and "client". On the server, you'll have the build system of your choice, having all tarballs created under /var/www/dl.z3bra.org/releases. You'd use repogen(1) to turn this directory into a repository (it simply generate a .list file, with the name, version and checksum of all tarballs):
Code:
[root@server ~]# repogen /var/www/dl.z3bra.org/releases
bzip2 1.0.6 2df1fe882e46a1e614d0fa2c72590c2c0354923d
curl 7.42.1 be76b8b93528156fe36049ee92dd64001607b263
dash 0.5.8 3a4833d32c1c633c11bd048d6efd89cc0b7620d0
fs 0.1 3ffeef271370b629ed6a2b93023c0372fd176680
gzip 1.3.3 b82268b6ad4d86db2d11aa28352b8c20cd40a994
iproute2 3.18.0 c47b14dfda5617e3a79d491ad9e9e109c2a81ebc
mk 0.1 dd7bbbe7d7cd0f69fc44d606b94d3ef574cf3693
mksh R50e 01119230f24f488777cff2cc46cc5aae1aecaa6a
musl 1.1.12 eaa54d777dd5d341378c1d3742b30f532e4ac653
pcc-libs 1.1.0 ada38ca8ee75e734663a0be75927df1303954c81
pcc 1.1.0 9a1b3f7fd8cad795026cf98415165dc8756d296c
sbase 0.0 c7a78f26f9d3ee92c1d43d3078e12f54c9cda6e7
ubase 0.0 73b4e3ee83b78b0ddcf14672713f3e6d11311eaeNow that you have your remote repo setup, you can sync it locally using the repo(1) utility:
Code:
[z3bra@client ~]$ export REPO=$HOME/repo
[z3bra@client ~]$ repo -s
bzip2 1.0.6 2df1fe882e46a1e614d0fa2c72590c2c0354923d
curl 7.42.1 be76b8b93528156fe36049ee92dd64001607b263
dash 0.5.8 3a4833d32c1c633c11bd048d6efd89cc0b7620d0
fs 0.1 3ffeef271370b629ed6a2b93023c0372fd176680
gzip 1.3.3 b82268b6ad4d86db2d11aa28352b8c20cd40a994
iproute2 3.18.0 c47b14dfda5617e3a79d491ad9e9e109c2a81ebc
mk 0.1 dd7bbbe7d7cd0f69fc44d606b94d3ef574cf3693
mksh R50e 01119230f24f488777cff2cc46cc5aae1aecaa6a
musl 1.1.12 eaa54d777dd5d341378c1d3742b30f532e4ac653
pcc-libs 1.1.0 ada38ca8ee75e734663a0be75927df1303954c81
pcc 1.1.0 9a1b3f7fd8cad795026cf98415165dc8756d296c
sbase 0.0 c7a78f26f9d3ee92c1d43d3078e12f54c9cda6e7
ubase 0.0 73b4e3ee83b78b0ddcf14672713f3e6d11311eaeSyncing the repository will NOT download all the tarballs. That would be a huge waste of time. You can list the content of your local repo with "repo -l". This will give the same output as when you sync the repo, minus the checksums.
To actually download a tarball, simply run repo(1) with the name of the pack you want. Once done, the full path to the tarball will be printed to stdout:
Code:
[z3bra@client ~]$ repo dash
/home/z3bra/repo/dash:0.5.8.tar.bz2
[z3bra@client ~] tree $REPO
/home/z3bra/repo
├── dash:0.5.8.tar.bz2
└── .listIf you call the command again, repo will check the tarball's checksum against its local .list file, and if they match, will output the full path again to stdout, without redownloading the file (this makes sense right?)
You can then use repo's output to feed pm(1) without having to know where the tarballs are located:
Code:
[z3bra@client ~]$ pm -a $(repo dash)is equivalent to:
Code:
[z3bra@client ~]$ repo dash
/home/z3bra/repo/dash:0.5.8.tar.bz2
[z3bra@client ~]$ pm -a /home/z3bra/repo/dash:0.5.8.tar.bz2At some point, I'll try to make pm read path from stdin, leading to a simpler way to install tarballs ;)
Repo can already read pack names from stdin, leading to some neat one liners! For example, to download all packs except the libraries, you can run the following:
Code:
[z3bra@client ~]$ repo -s | grep -v 'lib' | cut -f1 | repo
/home/z3bra/repo/bzip2:1.0.6.tar.bz2
/home/z3bra/repo/curl:7.42.1.tar.bz2
/home/z3bra/repo/dash:0.5.8.tar.bz2
/home/z3bra/repo/fs:0.1.tar.bz2
/home/z3bra/repo/gzip:1.3.3.tar.bz2
/home/z3bra/repo/iproute2:3.18.0.tar.bz2
/home/z3bra/repo/mk:0.1.tar.bz2
/home/z3bra/repo/mksh:R50e.tar.bz2
/home/z3bra/repo/musl:1.1.12.tar.bz2
/home/z3bra/repo/pcc:1.1.0.tar.bz2
/home/z3bra/repo/sbase:0.0.tar.bz2
/home/z3bra/repo/ubase:0.0.tar.bz2Note that this is still a work in progress, which is why I didn't commit neither repo(1) nor repogen(1). I'm still trying to figure out how they should work, and what the workflow should be.
If you're interrested in giving it a shot, and reporting some points that might be awkward or incoherent, just ask and I'll paste the scripts ;)
RE: Musl Distros - venam - 16-01-2016
(15-01-2016, 08:51 AM)z3bra Wrote: pm(1) only takes local files as arguments, it is not meant to deal with external repos of some kind.Seems like a great program to do something like packages sharing between test machines or work machines, for example on a local network box.
I have written two small shell script, namely repo(1) and repogen(1) to deal with remote tarball fetching or whatever.
It's easier to setup than any other package manager syncing repo and offers flexibility to choose what is shared.
RE: Musl Distros - XcelQ - 24-01-2016
I love adam distro already if he get it working. that my dream musl based os.
RE: Musl Distros - z3bra - 09-02-2016
A little update for this thread. My pack manager seems to be working now, in a sane, reliable and repeatable way. I am now focusing on making the distro a standalone environment.
I want to be able to bootstrap my distro from within it. The real missing part here is gcc. That bitch is a hell to get right.
I've dropped my "in-house" cross-compiler for something more professional and reliable: crosstools-ng. It provides a toolchain that is more solid than my hacky solution, but also less flexible (my toolchain could ONLY compile static binaries, and didn't make use of any directory having a /usr prefix).
So I'll need to investigate it a bit further. Still, I've been able to use it with most of my ports, so it's a nice step forward.
If anyone is interrested in giving it a try, I can help!
RE: Musl Distros - venam - 09-02-2016
(09-02-2016, 07:56 AM)z3bra Wrote: If anyone is interrested in giving it a try, I can help!I did give it a try!
The whole setup process was quite tedious.
pm is amazingly simple and fits well the static compilation.
I didn't dig too much into how mk works to help with gcc compilation, I'll try to focus on that next time.
It's all more complex than I thought.
RE: Musl Distros - z3bra - 09-02-2016
It is indeed complex; but it's also a huge mine of knowledge! This is what kept me hooked from the start.
RE: Musl Distros - z3bra - 15-06-2017
Quick update! I'm finally running my distro in a chroot, with all packs compiled from within this chroot (including gcc and binutils!). A few packs are still resisting the compilation process (m4 and diffutils for example) but overall, it works quite well!
I will hopefully boot it from real hardware soon, so I can start working on a better init and service management programs!
RE: Musl Distros - jkl - 15-06-2017
2017 update:
Note that midipix, the native "musl for Windows", is already shaping up nicely:
http://midipix.org