Hey guys! I really like Musl but I'm having trouble finding good distros that support it. I really liked Alpine but I can't get it to boot. Any ideas? I would prefer stable distros but if a experimental distro is stable enough that's okay too. Can Crux be a Musl system?

Thanks! - Xikuuky

UPDATE: I think I am going to make a distro that fulfills my needs. It will probably run inside another system using Busybox/Toybox and Musl. Maybe with something like (B,C,H)LFS?

(27-12-2015, 06:02 AM)xikuuky Wrote: make a distro

Keep us updated also please see the thread: https://nixers.net/showthread.php?tid=1741
There are others doing similar things to you, I know z3bra posted in the other 'smallest distro' thread but he was messing around with musl and the like, you should talk to him about this.

---

Thought I should add this:

http://wiki.musl-libc.org/wiki/Projects_using_musl

List of all the projects using musl, lots of distros on there you can pull inspiration from

void! stable, rolling release, actively developed, musl-flavor.

https://voidlinux.eu

Alpine is not experimental, it's definitely stable (My personnal server runs alpine, and I never ran into a problem.
Voidlinux has a musl flavor as sth said.

For crux, musl isn't supported, and if you go down this route, you will certainely have to patch a bunch of softwares.

(27-12-2015, 01:41 PM)sth Wrote: void! stable, rolling release, actively developed, musl-flavor.

https://voidlinux.eu

I already use Void. A bunch of the packages on it can be jokes for Musl, I build everything by hand.

(28-12-2015, 04:22 AM)z3bra Wrote: Alpine is not experimental, it's definitely stable (My personnal server runs alpine, and I never ran into a problem.
Voidlinux has a musl flavor as sth said.

For crux, musl isn't supported, and if you go down this route, you will certainely have to patch a bunch of softwares.

Alpine does not like install medium (a very old 16GB drive and I have no CD drive or CD's.

(27-12-2015, 12:55 PM)Houseoftea Wrote: Keep us updated also please see the thread: https://nixers.net/showthread.php?tid=1741
There are others doing similar things to you, I know z3bra posted in the other 'smallest distro' thread but he was messing around with musl and the like, you should talk to him about this.
Thought I should add this:

http://wiki.musl-libc.org/wiki/Projects_using_musl

List of all the projects using musl, lots of distros on there you can pull inspiration from

I can do that! :) I'll look at that.

Here, an update.
==================

Looked at many proejects that use or will use musl plus a lot of lightweight distros and I have a bit of a idea on how I will accomplish this. I'm going to try LFS then BLFS, and eventually ALFS (unless people can just install from what I give them [doubt it but possible]).

For packages I will supply a package manager I wrote myself that builds packages from git using shell scripts. It is a less flexible system but it works for what I'll do. We are going to probably be 64bit exclusive (sorry ARM, x86, and MIPS users). If it isn't hard to cross compile, I may add x86. I'll have the base system preinstalled hopefully and I'll hopefully be able to update it with ease. Make + autotools will install, uninstall, and configure software. The package manager runs on Git and Make so it works perfectly. Users will have a .software or .packages or ~/Code/Soft(ware) area that holds the source of the packages. I'd like feedback. Thank you!

---

Also, I'll include Toy/Busybox hopefully and have bash and zsh installed by default and scripts already on the machine to build things like Python, Python2, Vim, and the such. :)

I looked at Solyste and I like the idea of mksh as the default shell. In my experience that is very fast. Also, how hard is it to compile X?

xikuuky Wrote:For packages I will supply a package manager I wrote myself that builds packages from git using shell scripts. It is a less flexible system but it works for what I'll do.

"Wrote"? Did you write anything yet? 'cause I'd be interrested in reading what you've done so far. I'm currently working on a package manager myself I already have a "working" solution using shell script, but I'm not satisfied with it, so I'm rewriting everything from scratch in C, focusing on different concepts.

xikuuky Wrote:We are going to probably be 64bit exclusive (sorry ARM, x86, and MIPS users). If it isn't hard to cross compile, I may add x86.

x86 is a CPU architecture, which uses either 32 or 64bits. So you're probably opposing x86_64 to x86 there. Beware the wording is such a pedantic forum ;)

xikuuky Wrote:I'll have the base system preinstalled hopefully and I'll hopefully be able to update it with ease. Make + autotools will install, uninstall, and configure software. The package manager runs on Git and Make so it works perfectly. Users will have a .software or .packages or ~/Code/Soft(ware) area that holds the source of the packages. I'd like feedback. Thank you!

It's the simplest way to go indeed, and if softwares were done properly, we wouldn't need package managers. The problem is that not all software use a makefile, and even if they do, not many propose the "uninstall" target.
This is why package manager were created in the first place, they track files intalled by each package, and allow the user to easily remove/update them. It later handled package dependency, which became an issue when dynamic linking started to kick off.

You should take a look at how other people solved those problems. here are a few links to get you started:

• http://git.z3bra.org/cgit.cgi/pkit/tree/README
  
• https://github.com/dimkr/packlim
  
• https://github.com/radare/slpm
  
• http://www.linuxfromscratch.org/lfs/view...kgmgt.html
  

xikuuky Wrote:Also, I'll include Toy/Busybox hopefully and have bash and zsh installed by default and scripts already on the machine to build things like Python, Python2, Vim, and the such. :)

Choosing the default set of packages is always a hard task ;)
I'd personally go with the bare minimum: init, device manager, shell, coreutils, netutils and that's all. I'll certainely have to add the package manager and curl to the live usb though, to make bootstraping easier.

xikuuky Wrote:how hard is it to compile X?

really hard if you need to package everything yourself.

(28-12-2015, 08:46 AM)z3bra Wrote: "Wrote"? Did you write anything yet? 'cause I'd be interrested in reading what you've done so far. I'm currently working on a package manager myself I already have a "working" solution using shell script, but I'm not satisfied with it, so I'm rewriting everything from scratch in C, focusing on different concepts.

Sadly the code is gone but it just downloaded a script and executed it line for line. I found a way to easily uninstall stuff with make. Just type make -n install and it tells you what stuff will be modified. Maybe I'll go without a package manager or I'll just get something like emerge on here(go ports! :) ). I really like just compiling everything.

(28-12-2015, 08:46 AM)z3bra Wrote: x86 is a CPU architecture, which uses either 32 or 64bits. So you're probably opposing x86_64 to x86 there. Beware the wording is such a pedantic forum ;)

Thank you so much! Now I won't look like an idiot. o/ :)

(28-12-2015, 08:46 AM)z3bra Wrote: It's the simplest way to go indeed, and if softwares were done properly, we wouldn't need package managers. The problem is that not all software use a makefile, and even if they do, not many propose the "uninstall" target.
This is why package manager were created in the first place, they track files intalled by each package, and allow the user to easily remove/update them. It later handled package dependency, which became an issue when dynamic linking started to kick off.

You should take a look at how other people solved those problems. here are a few links to get you started:

• http://git.z3bra.org/cgit.cgi/pkit/tree/README
  
• https://github.com/dimkr/packlim
  
• https://github.com/radare/slpm
  
• http://www.linuxfromscratch.org/lfs/view...kgmgt.html
  

I'll definitely go through this list. Thank you so much buddy! :) I'm new to C[++,go] and the such (I can read it just fine) so it would be a big deal if you would read my source code for me. I want to use a compiled language for speed (Python is fast but C can be faster).

(28-12-2015, 08:46 AM)z3bra Wrote: Choosing the default set of packages is always a hard task ;)
I'd personally go with the bare minimum: init, device manager, shell, coreutils, netutils and that's all. I'll certainely have to add the package manager and curl to the live usb though, to make bootstraping easier.

I think I'll do bare minimum plus a little extra. :) Extra being a few shells and make, autotools, and musl. What is typically in netutils? What if I made the package manager the user's choice to install? I've seen installs like that (I once made a Ubuntu install use Emerge XD). I might include curl but then again, wget is a core package in both busybox/toybox. I use Curl personally but Wget is very popular.
As for init, I see the big options as: SystemD (not happening), OpenRC (I'm open to it), or Runit (my favorite). This is my distro so if I want to add Runit I can right? This is meant to be for my needs (and anyone with needs similar to mine).

(28-12-2015, 08:46 AM)z3bra Wrote: really hard if you need to package everything yourself.

I am not packaging X. Packages, or even a package manager will (possibly) be left to the package manager the user installs.

UPDATE:
========
Elder Scrolls Online is still downloading. It is almost done so I'll hopefully get to boot into Linux tomorrow/tonight GMT+7 (Jakarta).
Decisions:
Init (Runit or OpenRC)
Minimal Core Utils (Busybox or Toybox)
?Package Manager? (user choice or I write one or I include one)

That's all.

Init (Runit or OpenRC) :: runit
Minimal Core Utils (Busybox or Toybox) :: toybox
?Package Manager? :: packlim

OpenRC is indeed a shitty service manager. I really can't see the hype behind it. The fact it's used in gentoo doesn't make it a good system, more people should understand this. Runit on the other hand implements way more good practices, even though some people reported issues with it when dealing with systems with a lot of services (especially with service restarting/failure detection).

For the core utils, you neither busybox nor toybox are POSIX compliant, making them poor alternatives to coreutils which is POSIX-compliant (but adds a shiton of useless/bloated features on top of their utils, making them unreliable and bug-prone).
There is the suckless base which is fairly good and POSIX-compliant. They also provide a way to build a sbase-box binary if that's what you really want.

Regarding the package manager, packlim is indeed fairly interresting, but I don't know how one would integrate it in another project than rlsd2.
I like the way it handle packages though. This result in a really simple format for packages, and it even has package signing.
I don't understand much why the author added a TCL interface though.
There is also no documentation for the build system, which seems interresting, but makes use of many different technologies, resulting in a big mess to me (makefile, shell scripts, TCL, json, ...).

Packlad is another package manager that the guy made, but that ones written in all c
And spuk was interesting ;)

(29-12-2015, 10:55 AM)Houseoftea Wrote: Init (Runit or OpenRC) :: runit
Minimal Core Utils (Busybox or Toybox) :: toybox
?Package Manager? :: packlim

(29-12-2015, 01:48 PM)z3bra Wrote: For the core utils, you neither busybox nor toybox are POSIX compliant, making them poor alternatives to coreutils which is POSIX-compliant (but adds a shiton of useless/bloated features on top of their utils, making them unreliable and bug-prone).
There is the suckless base which is fairly good and POSIX-compliant. They also provide a way to build a sbase-box binary if that's what you really want.

Runit for sure then. I have installed sbase on my own system and I like it. sbase plus ubase seems to be more complete of a full coreutils though. I couldn't get packlim to install. So update this is now:

Init: Runit
Minimal Core Utils: sbase
?Package Manager?: Still none. Maybe Butch

Could Pacman be a good package manager? Or would Crux or Gentoo's be better? :)
Also, checkout Sabotage Linux. Similair to what we/I are making. I'm pretty sure we could change it to fit our needs. It is a very small system. :)

As I just discussed on IRC, I'm not making a distro for a community, or to fit people's needs. It's mostly for fun, and have something I built with my own hands.
Alone, I can't write a better package manager than pacman, apt or yum, I can't make an init system that will put systemd to shame, or a daemon manager as good as daemontools...

But I can make these softs. Make mine, and put them together to build a distro I can call my own. That's my goal. And I'll certainly fail at some point, just so I'll be able to do it again later :P

(29-12-2015, 03:03 PM)Houseoftea Wrote: Packlad is another package manager that the guy made, but that ones written in all c
And spuk was interesting ;)

I will look into those. I didn't see those last night.

(29-12-2015, 05:11 PM)z3bra Wrote: As I just discussed on IRC, I'm not making a distro for a community, or to fit people's needs. It's mostly for fun, and have something I built with my own hands.
Alone, I can't write a better package manager than pacman, apt or yum, I can't make an init system that will put systemd to shame, or a daemon manager as good as daemontools...

But I can make these softs. Make mine, and put them together to build a distro I can call my own. That's my goal. And I'll certainly fail at some point, just so I'll be able to do it again later :P

I think I'll make my distro just for me too then (anyone can look at it, install it, fork it, whatever) that way I can make the perfect system I want to use! :)

---

UPDATE
=======
I think the base for everything will be Sabotage Linux. I can easily write a package that would install my system into theirs. That makes life easy. :)

UPDATE
======
Stali is very similair and fits my needs. I may continue this as a hobby but for a more working distro please use Stali.

For what its worth, I finally have a working package manager! It's dead simple, not finished and certainly has bugs, but it can install, list and remove packages!

http://git.z3bra.org/cgit.cgi/pm

(30-12-2015, 09:13 PM)z3bra Wrote: For what its worth, I finally have a working package manager! It's dead simple, not finished and certainly has bugs, but it can install, list and remove packages!

http://git.z3bra.org/cgit.cgi/pm

hey z3bra I was playing around a little with pm and I couldnt get it working exactly.
I installed it, and pm -i (pack) works fine but pm -a pack:(pack) does not work at all.
am I doing something dumb?

You can only install packages built locally. Said otherwise, tarballs. For example, run the following:


That will install the pack "pm" to the PACKAGE_ROOT specified in the source (this directory should already exist)

I made some progress on my distro!
Here is everything I can do:

* build packs statically against musl (including pcc and binutils)
* sync packs from a remote repo into a local directory
* install packs in a chroot
* compile a "hello world!" with pcc from within the chroot

Here is a proof: http://raw.z3bra.org/dev/random/distro.webm

For the upcoming steps:

* fix bugs in the pack manager
* boot AGAIN an LXC container / on real hardware
* build packs from within the distro
* enjoy!

(07-01-2016, 08:28 PM)z3bra Wrote: Here is a proof: http://raw.z3bra.org/dev/random/distro.webm

highly impressed, z3bra. im under the impression pm has no way of showing package updates, though?
Of course, a package manager that works with tarballs, thats extremely difficult if not impossible to do.

For now, I decided to focus on add/delete/list only, as they are the most important actions.
Updates will definitely be handled though, and in a simple way. Here is what I have in mind:

if package version is different in the tarball name compared to what's installed, then:

0. remove file that are not in the tarball anymore
1. unpack files from the tarball, overwriting old file
2. update the "version" file in the metadata directory

For the two first step, the only condition would be that creation time == last modification time.
If the user modified it by hand, it means we shouldn't remove it, or replace. If it's the case, then two actions will be taken:

0. warn the user that the file shouldn't exist anymore and will not be removed in the future
1. write the new file as ${filename}.new and report it to the user

I can't think of why this process would not cover all cases :)

Very cool indeed z3bra. Like I said, I'm really intrigued by your package manager, but I have to ask, would that process only be possible if the tarball(s) in question were already on the filesystem? Or does this mechanism work with the download location of the tarball? I've noticed that projects TEND to have predictable tarball locations (IE: most GNU software uses the same FTP/HTTP hierarchy), but this isn't true for a lot of software.

Honestly I'm really only honing in on this specific feature because a few years ago I attempted the same thing you have and I failed miserably.

pm(1) only takes local files as arguments, it is not meant to deal with external repos of some kind.
I have written two small shell script, namely repo(1) and repogen(1) to deal with remote tarball fetching or whatever. I've listed some examples in pm's README, but for all the lazy asses in there (I know there's a lot of you!), here is an idea of how it works.

Let's say you have two machines, "server" and "client". On the server, you'll have the build system of your choice, having all tarballs created under /var/www/dl.z3bra.org/releases. You'd use repogen(1) to turn this directory into a repository (it simply generate a .list file, with the name, version and checksum of all tarballs):


Now that you have your remote repo setup, you can sync it locally using the repo(1) utility:


Syncing the repository will NOT download all the tarballs. That would be a huge waste of time. You can list the content of your local repo with "repo -l". This will give the same output as when you sync the repo, minus the checksums.
To actually download a tarball, simply run repo(1) with the name of the pack you want. Once done, the full path to the tarball will be printed to stdout:


If you call the command again, repo will check the tarball's checksum against its local .list file, and if they match, will output the full path again to stdout, without redownloading the file (this makes sense right?)
You can then use repo's output to feed pm(1) without having to know where the tarballs are located:


is equivalent to:


At some point, I'll try to make pm read path from stdin, leading to a simpler way to install tarballs ;)
Repo can already read pack names from stdin, leading to some neat one liners! For example, to download all packs except the libraries, you can run the following:


Note that this is still a work in progress, which is why I didn't commit neither repo(1) nor repogen(1). I'm still trying to figure out how they should work, and what the workflow should be.
If you're interrested in giving it a shot, and reporting some points that might be awkward or incoherent, just ask and I'll paste the scripts ;)

(15-01-2016, 08:51 AM)z3bra Wrote: pm(1) only takes local files as arguments, it is not meant to deal with external repos of some kind.
I have written two small shell script, namely repo(1) and repogen(1) to deal with remote tarball fetching or whatever.

Seems like a great program to do something like packages sharing between test machines or work machines, for example on a local network box.
It's easier to setup than any other package manager syncing repo and offers flexibility to choose what is shared.

I love adam distro already if he get it working. that my dream musl based os.

A little update for this thread. My pack manager seems to be working now, in a sane, reliable and repeatable way. I am now focusing on making the distro a standalone environment.
I want to be able to bootstrap my distro from within it. The real missing part here is gcc. That bitch is a hell to get right.
I've dropped my "in-house" cross-compiler for something more professional and reliable: crosstools-ng. It provides a toolchain that is more solid than my hacky solution, but also less flexible (my toolchain could ONLY compile static binaries, and didn't make use of any directory having a /usr prefix).
So I'll need to investigate it a bit further. Still, I've been able to use it with most of my ports, so it's a nice step forward.

If anyone is interrested in giving it a try, I can help!

(09-02-2016, 07:56 AM)z3bra Wrote: If anyone is interrested in giving it a try, I can help!

I did give it a try!
The whole setup process was quite tedious.
pm is amazingly simple and fits well the static compilation.
I didn't dig too much into how mk works to help with gcc compilation, I'll try to focus on that next time.
It's all more complex than I thought.

It is indeed complex; but it's also a huge mine of knowledge! This is what kept me hooked from the start.

Quick update! I'm finally running my distro in a chroot, with all packs compiled from within this chroot (including gcc and binutils!). A few packs are still resisting the compilation process (m4 and diffutils for example) but overall, it works quite well!

I will hopefully boot it from real hardware soon, so I can start working on a better init and service management programs!

2017 update:
Note that midipix, the native "musl for Windows", is already shaping up nicely:
http://midipix.org