One thing I've noticed after I got interested in all things *nix is the different file formats out there. Before, I didn't really think about other formats than jpeg and png for images and mp3 for music, etc.

As an example, I've been programming using the farbfeld image format recently and using its associated tools for converting to other formats, such as pngs.

What's your opinion on new formats for media? Are they useful or do they divide media more and force developers to create tools for these new formats? What interesting media formats do you use?

I just love file and protocol formats, it's one of the thing I'm fascinated by, maybe because it's a thing I deal with a lot professionally.

Recently I've been spending a lot of time with the JPEG format, I'll be posting an article about this on my blog soon (here now. Especially when it concerns thinking about formats like we think about physical matter, its composition, and fluidity.
I'm interested in how resilient the format is, if it has ECC or error detection mechanism, if small nudges can break it, if it has a compression mechanism, if it contains metadata, and more.

Personally, I think the novelty lies in how to represent the same content but more efficiently. For video, audio, and images, it's all about removing or joining information that us human can't always discern. I'm impressed by the likes of webp, avif, and others. All stemming from websites wanting to fight the web obesity while still keeping ton of pictures and videos.

I use farbfeld for all my image viewing, because compressing with gzip/bzip2 is good enough, and the "lel" image viewer works well for my use cases.

For videos, I try to put everything as webm, as from my experience it has a good quality/size ratio, when encoded properly (especially screencasts). I know that blind uses a raw format not suitable for storage, but some videos compressed with bzip2 might be "good enough". It can read farbfeld images natively too.

When it comes to sound though... I'm a complete noob, so I don't know much about existing formats. I think ogg vorbis is nice, so I use that, or mp3.

As for my opinion on these formats, well, simple formats make it easy to deal with. I've worked woth farbfeld too (got a few commits in xscreenshots IIRC), and blind obviously uses a raw format to make dealing with pixels easier. There is a really gokd reason for this formats to exist, however some format being maybe more complex are needed to achieve better/faster compression.

I just wished there was a format to concatenate music or videos with cat(1).

venam Wrote:All stemming from websites wanting to fight the web obesity while still keeping ton of pictures and videos.

Marking the <script> tag obsolete would cure web obesity better than saving a few bytes in images IMO.

(03-10-2020, 02:44 PM)venam Wrote: I'm impressed by the likes of webp, avif, and others.

I completely agree. What I've seen so far of webp is amazingly good, and I've not really seen any negative sides with it apart from it being lossy. Much better quality to size ratio than jpeg as far as I know.

(03-10-2020, 02:50 PM)z3bra Wrote: When it comes to sound though... I'm a complete noob, so I don't know much about existing formats. I think ogg vorbis is nice, so I use that, or mp3.

Personally, I get all my music in flac just to be sure I don't lose any information, but it takes a lot more space than mp3s.

(03-10-2020, 02:50 PM)z3bra Wrote: I just wished there was a format to concatenate music or videos with cat(1).

That would actually be so cool. Probably possible with purely raw, no metadata formats.

Speaking about formats and encoding, do anyone know why, for example, farbfeld stores pixels with each channel on 16 bits? I think I've seen 16 bits wide channels on other tools, too, but I just can't see the why... I mean, the old (usual?) 8-bit channels already implies images with 2^24 colors, and AFAIK human eyes just can't distinguish, like, #0000FF and #0000FE, so, why double the sizes? To me it really feels like wasting memory and (motherboard) bandwidth, same for those 64bits pointers, except for specialized uses?

(04-10-2020, 06:24 AM)freem Wrote: Speaking about formats and encoding, do anyone know why, for example, farbfeld stores pixels with each channel on 16 bits?

While I haven’t found an “official” statement, I believe the reason is something along these lines:

Quote:For some domains it's useful to preserve more detail than the eye
can differentiate, so that different image enhancement or
modification processes can work better. Like with audio; 44kHz is
about the limit of human differentiation, but 96 / 128kHz is better
for working with it, say for remixing.

http://lists.suckless.org/dev/1407/22960.html

If you want to do more searching, you should probably look at the discussions about “imagefile”, which was the predecessor of farbfeld. A lot of the discussions happened back then.

Also, there are a couple of talks by Laslo Hunhold on farbfeld and color spaces, maybe he explains it in one of them.

(04-10-2020, 07:54 AM)vain Wrote: While I haven’t found an “official” statement, I believe the reason is something along these lines:

Quote: Quote:
For some domains it's useful to preserve more detail than the eye
can differentiate, so that different image enhancement or
modification processes can work better. Like with audio; 44kHz is
about the limit of human differentiation, but 96 / 128kHz is better
for working with it, say for remixing.

That's actually a very interesting point. So farbfeld could be thought as a cold storage format, a higher lossless resolution, while some current known compressed format could be used to serve to users.

This reminds me of what Netflix does with movies, it stores the master file, and then generates some specific output using the current best video encoding they can find. If in the future there's a new one then they can use the original and convert it to it.

To quote the farbfeld FAQ :

Quote:Why use 16-Bits-per-channel all the time? Isn't this a total waste?

Not when you take compression into account. To make this clearer, assume a paletted image with 5 colors and no transparency. So the data is only a set of regular chunks (color1, ..., color5) in a certain order. Compression algorithms have been designed to recognize those chunks and can even look at how these chunks interact.Local tests have shown that farbfeld easily beats paletted PNG-images. Try for yourself and look at the bzipped results! There is no need for special grayscale, palette, RGB, 1-, 2-, 4-, 8-, 16-Bit subformats. Just use 16-Bit RGBA all the time and let compression take care of the rest.

(04-10-2020, 07:54 AM)vain Wrote:

Quote:For some domains it's useful to preserve more detail than the eye
can differentiate, so that different image enhancement or
modification processes can work better. Like with audio; 44kHz is
about the limit of human differentiation, but 96 / 128kHz is better
for working with it, say for remixing.

No, that quote works perfectly, and makes perfect sense. I did guessed that specialized uses would be a good reason for that after all.

(04-10-2020, 09:19 AM)z3bra Wrote: To quote the farbfeld FAQ :

Quote:Compression algorithms have been designed to recognize those chunks and can even look at how these chunks interact.

Yes, I had not thought about this, using wider pixel sizes could result in helping compression algorithms finding patterns. And a better compression will also reduce loading time at least for very big file (which is why linux kernel is compressed, even if I'm not certain that this is noticeable with SSDs).

I wonder how that kind of thinking would apply on text. Could result in compressed full codepoint strings being smaller than compressed UTF-8...

[edit]
hurg... /quote missing (breaking all formating)

I had a go at torturing image formats recently, maybe this will give a glance into what image formats are about. here's the article.

Let's add some part of a discussion we had on IRC. We were discussing how QR codes are solid and flexible format. You can create art with them, extract them from grainy and blurry videos and whatnot.

(05-10-2020, 07:39 AM)venam Wrote: I had a go at torturing image formats recently, maybe this will give a glance into what image formats are about. here's the article.

Had a look at your post, it's a very interesting read. Gonna try to make a C program or something to modify my bits a bit more efficiently, looking forwards to the results.
Concepts were very well explained, I feel I know a bit more now, thanks!

I recently had a go with kaitai format parser and thought it would be interesting to bump this thread. A similar tool called GNU Poke also comes to mind.
So anyway, let's discuss alternative format of common files, and other formatting tools.

(03-10-2020, 04:03 PM)opFez Wrote:

(03-10-2020, 02:50 PM)z3bra Wrote: I just wished there was a format to concatenate music or videos with cat(1).

That would actually be so cool. Probably possible with purely raw, no metadata formats.

ffmpeg's concat filter doesn't work for you?

It does, but is much more complex than cat(1). If videos where just streams of data, you could simply concatenate them by appending one after the other.