The mkosi OS generation tool
mkosi is definitely a tool with a focus on developer's needs for building OS images, for testing and debugging, but also for generating production images with cryptographic protection. A typical use-case would be to add a mkosi.default file to an existing project (for example, one written in C or Python), and thus making it easy to generate an OS image for it. mkosi will put together the image with development headers and tools, compile your code in it, run your test suite, then throw away the image again, and build a new one, this time without development headers and tools, and install your build artifacts in it. This final image is then "production-ready", and only contains your built program and the minimal set of packages you configured otherwise. Such an image could then be deployed with casync (or any other tool of course) to be delivered to your set of servers, or IoT devices or whatever you are building."
Posted Jun 28, 2017 18:57 UTC (Wed)
by jgg (subscriber, #55211)
[Link] (7 responses)
https://github.com/linux-rdma/rdma-core/blob/master/build...
Which was inspired by other tools and customised to run smoothly with the source tree it is embedded in.
mkosi is basically the same idea but for systemd. It doesn't really do anything particularly special, expect that it exports the container into a unique disk image format. Other tools like rkt can already do this, but their images are for running in kvm, not bare metal.
The last time I tried to use mkosi (232) it just failed. The main trouble seems to be that it does a lot of work in the host, so it is quiet sensitive to the host configuration, which to my mind, largely defeats the entire purpose.. It was also bonkers slow, but maybe I didn't get far enough to have the caching turn on, or that version didn't have sane caching yet.. Plus it wanted to run as root on the host and do all sorts of strange things - again not the kind of safety I want from a containerised build tool.
Posted Jun 28, 2017 19:24 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link] (6 responses)
"unique disk image format"? Do you mean the raw GPT disk images mkosi can generate with that? I don't find that particularly "unique", it's probably one of the most universally understood disk format these days, as any Windows, any ChromeOS and pretty much everything recent uses GPT.
And to clarify, after the initial packages have been extracted (i.e. right after the initial debootstrap or dnf --rootinstall), everything relevant we run runs inside an nspawn container.
mkosi is currently at version number 3, hence I don't think you can have tested mkosi 232 ;-). "232" sounds more like a systemd release version.
Lennart
Posted Jun 28, 2017 21:05 UTC (Wed)
by jgg (subscriber, #55211)
[Link] (5 responses)
In the mainstream container world the popular way to run a container under kvm is with 9pfs (eg in rkt), this is what I mean by 'unique'. This is actually where I ended up after trying mkosi for systemd. Instead, I built systemd using a building container (with a script similar to the one I linked), then 'make installed' it into a temporary running container and directly ran kvm and 9pfs exporting the containers / as rootfs to run the build. Much faster than building an image, and I get more trivial points to customise things I need, like using my own kernel build or modifying the FS before booting to test the stuff I needed..
For mkosi it was the initial package extracting/install that failed for me, and it didn't cache the download step before that, so the entire thing was a PITA to debug. It also seemed like it wanted to run various loop devices and other things on the host when it was building the image, which I'd honestly rather it do inside a privileged container.. I also gave up trying to build a Fedora boot image from my Ubuntu box (again, needs too many host dependencies).
IMHO, the thing about docker, is it has one really good idea - dockerfile, with the overlayfs based caching layers and downloadable seed - and a bunch of not so good, out dated execution around that good idea. systemd has good execution in systemd-nspawnd, rkt provides most of the orchestration piece on top of that, but that entire world lacks a refined equivalent to dockerfile to make the images in the first place..
You are right, I used something between v1-v2 (dec 16 github TOT) in an attempt to do some work with systemd 232.
Posted Jun 29, 2017 13:16 UTC (Thu)
by gdamjan (subscriber, #33634)
[Link] (2 responses)
The docker file itself, is just awful. It's like a worse shell script
Posted Jun 29, 2017 17:05 UTC (Thu)
by zlynx (guest, #2285)
[Link] (1 responses)
Posted Jun 30, 2017 8:30 UTC (Fri)
by gdamjan (subscriber, #33634)
[Link]
Posted Jul 2, 2017 1:33 UTC (Sun)
by sbaugh (guest, #103291)
[Link] (1 responses)
Posted Jul 5, 2017 23:23 UTC (Wed)
by nix (subscriber, #2304)
[Link]
Posted Jun 29, 2017 1:29 UTC (Thu)
by pabs (subscriber, #43278)
[Link] (1 responses)
Posted Jun 29, 2017 2:11 UTC (Thu)
by rahulsundaram (subscriber, #21946)
[Link]
Posted Jun 29, 2017 17:37 UTC (Thu)
by iainn (guest, #64312)
[Link] (2 responses)
I wanted an easy way of spinning up another distro. My first thought was to check if LXD were suitable, but to use LXD on Fedora you have to disable SELinux. My second thought was Docker, but it wasn't suitable because I wanted to persist state changes.
I found mkosi + systemd-nspawn slightly easier, and hopefully provide better isolation, than setting up a chroot.
Posted Jun 30, 2017 16:57 UTC (Fri)
by stgraber (subscriber, #57367)
[Link] (1 responses)
sudo dnf install snapd
Posted Jul 3, 2017 9:41 UTC (Mon)
by iainn (guest, #64312)
[Link]
But thanks, that's good to know. I hadn't considered a Snap package. Nor had I seen the recent news that snaps work on Fedora with SELinux.
Posted Jul 2, 2017 3:39 UTC (Sun)
by mrdocs (guest, #21409)
[Link] (4 responses)
Seems to me another NIH here by RH.
Posted Jul 2, 2017 15:02 UTC (Sun)
by rahulsundaram (subscriber, #21946)
[Link] (2 responses)
What makes you think Red Hat is deciding this?
Posted Jul 2, 2017 21:15 UTC (Sun)
by jospoortvliet (guest, #33164)
[Link] (1 responses)
Posted Jul 2, 2017 21:46 UTC (Sun)
by rahulsundaram (subscriber, #21946)
[Link]
That is a really weak rationale to think that this is done under some organizational direction. systemd itself was written by Lennart on his own time originally.
Posted Oct 20, 2019 15:16 UTC (Sun)
by dexit (guest, #133513)
[Link]
Posted Oct 20, 2019 14:45 UTC (Sun)
by dexit (guest, #133513)
[Link]
On Manjaro in Oct. 2019, only the Debian image gets created successfully.
Arch, Fedora, Centos, ClearLinux, Ubuntu, Mageia, OpenSUSE all fail for one reason or another. However sometimes its just that the packager tools such as zypper for SUSE do not build from AUR or anyhow.
In other cases the python error suggests some mkosi problem. btw: Arch pacstrap is here: https://projects.archlinux.org/arch-install-scripts.git/t...
Posted Aug 17, 2021 0:13 UTC (Tue)
by GNUtoo (guest, #61279)
[Link]
I've been looking for some software with the abstraction of mkosi for a long time.
The first thing that came to my mind are things like vagrant[1] and packer[2] to
So to build an image with vagrand and/or packer you could either:
- Use one of the images made by hashicorp (the company behind packer and
- Use official VM images or cloud images from distributions as a base.
- Build base images from official distributions installers. The issue here
- Like Tails, you could also write yourself the code to generate that base
Guix[7] (and probably Nix as well) already solved this issue as you can package
The issue is that not everyone wants to use Guix, so having some of these
And here mkosi has the ability to build base images using the bootstrap tools
The big limitation in the case of mkosi seems to be that it took the decision
Unfortunately the "legacy free" seems to limit a lot the use cases:
I'll also try to look at alternatives of mkosi that have the same abstraction level as that abstraction level really seem to fit many of the use cases I have.
References:
Denis.
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
sudo snap install lxd
sudo lxd init
lxd on Fedora
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
The mkosi OS generation tool
well I for one did look at KIWI but it supports a rather limited set of hosts and target OS-images. Then again, mkosi fails for all targets save Debian, of course. A clear lose-lose situation.
The mkosi OS generation tool
The mkosi OS generation tool
Use cases
create system images in a variety of formats, but they seem better adapted at
modifying existing base images than creating them from scratch.
vagrant), or the community of its users[3][4] as a base. The issue here
is that it might not be possible to trust these images in the same way
than images that are part of official distribution releases. This is
because the vagrant repositories and the distributions have different
policies.
The issue here is that not all the distributions have such images, and
you might also want to build types of images that aren't released by your
distribution (like add integrity checking, or use the latest packages
instead of the ones from the last release).
is that, even if it can work, sending keystrokes once the installer is
booted doesn't seem like the right abstraction to me[5].
image[6], but the limitation here is that it might not work for all the
images types, and it would probably be better to have some common project
to do something like that instead.
the same software in several way (in a VM[8], in a disk image[8], in a debian
package[9], in a docker container[9], etc). And in some cases where it produces
a booting system (like with the VM, disk image, or docker container) there is
also an API that takes care of the configuration of some applications like Nginx.
features available for other distributions is a good idea I think.
like debootstrap, pacstrap, and so on to build images. So that looks useful
even if it's not going to enable users to install any distribution from any
distribution, but work inside distributions and bootstrap tools could be done
to imrpove the situation.
to be legacy free (its manual states "mkosi - Build Legacy-Free OS Images").
- Some distributions are still supporting computers with i686 processors, so
using mkosi to create official installer or VM images won't work here.
- Many ARM computers have system on a chip that can boot on block devices and
that load the bootloader from either an offset on the block device, or in
a FAT MBR partition. The later cannot be supported with mkosi.
- Some modern computers running Libreboot or Coreboot, and some VMs use
SeaBIOS for simplicity. Here again it can't produce images with MBR.
-----------------
[1]https://www.vagrantup.com
[2]https://www.packer.io
[3]https://www.vagrantup.com/vagrant-cloud/boxes#vagrant-box...
[4]https://www.vagrantup.com/vagrant-cloud/boxes/catalog
[5]https://www.packer.io/docs/builders/virtualbox/ovf#boot-c...
[6]https://gitlab.tails.boum.org/tails/tails/-/raw/master/va...
[7]https://guix.gnu.org/
[8]with guix system image
[9]with guix pack