Goa - sticking together a little Unix (part 1)
Whereas the first article about Goa was concerned with building software using commodity build systems, this article takes you on a ride of creating a small Unix OS out of Genode's ready-to-use building blocks, publishing the result, and deploying it on top of Sculpt OS.
In the Goa introduction linked above, I already hinted at Goa's functionality beyond the porting and building of software. This article shows the fun and productive way of crafting component compositions out of Genode's readily available building blocks. What could be a better example than building an old-school operating system - Unix - that we all know and love?
Preparations
Before continuing, please make sure to have installed the Goa tool, which is available at https://github.com/genodelabs/goa. If you have it installed already, please make sure the tool is up to date. You can issue the following command to update Goa to the latest version:
$ goa update-goa
Hello bash
As the first step, we want to get a life sign of the bash shell. We start with a new Goa project appropriately named unix that hosts a runtime package but no source code.
$ mkdir unix
$ cd unix
unix$ mkdir -p pkg/unix
Let's pretend we don't know what we are doing and create an archives file with only bash listed, and an almost empty runtime file. The runtime starts the binary init, which is supposed to be a ROM module.
The pkg/unix/archives file:
genodelabs/src/bash
The pkg/unix/runtime file:
<runtime ram="100M" caps="5000" binary="init">
<content>
<rom label="init"/>
</content>
</runtime>
Let's see what happens when issuing the run command we know from the previous article:
unix$ goa run
download genodelabs/bin/x86_64/bash/2023-10-24.tar.xz download genodelabs/bin/x86_64/bash/2023-10-24.tar.xz.sig download genodelabs/src/bash/2023-10-24.tar.xz download genodelabs/src/bash/2023-10-24.tar.xz.sig download genodelabs/api/libc/2023-10-03.tar.xz download genodelabs/api/libc/2023-10-03.tar.xz.sig download genodelabs/api/noux/2023-06-15.tar.xz download genodelabs/api/noux/2023-06-15.tar.xz.sig download genodelabs/api/posix/2020-05-17.tar.xz download genodelabs/api/posix/2020-05-17.tar.xz.sig [unix] Error: runtime lacks a configuration You may declare a 'config' attribute in the <runtime> node, or define a <config> node inside the <runtime> node.
Let's follow the advice by adding an empty <config> node to our pkg/unix/runtime file:
<runtime ram="100M" caps="5000" binary="init">
<config/>
<content>
<rom label="init"/>
</content>
</runtime>
Besides the error message, we could see that Goa automatically downloaded bash along with its dependencies such as the libc. Besides the binaries, it also fetches all source codes. You can find all the downloads at var/depot/. One particularly interesting directory is the binary archive for bash:
unix$ ls var/depot/genodelabs/bin/x86_64/bash/2023-10-24/
bash.tar
It contains a single tar archive, which in turn, contains all installation files of bash. Let's take a look inside:
unix$ tar tf var/depot/genodelabs/bin/x86_64/bash/2023-10-24/bash.tar
./ ./share ./share/doc ... ./bin/bashbug ./bin/bash
Of course, the most interesting bit is the bash executable at bin/bash. When using the binary archive, the whole bash.tar is supplemented to Genode as a single ROM module. Let's add it to the <content> of the pkg/unix/runtime:
<runtime ram="100M" caps="5000" binary="init">
<config/>
<content>
<rom label="init"/>
<rom label="bash.tar"/>
</content>
</runtime>
After issuing goa run again, Goa downloads the additional packages needed to run our pkg/unix on Linux, integrates the scenario, and starts it.
download genodelabs/bin/x86_64/base-linux/2023-10-24.tar.xz download genodelabs/bin/x86_64/base-linux/2023-10-24.tar.xz.sig download genodelabs/bin/x86_64/init/2023-10-24.tar.xz download genodelabs/bin/x86_64/init/2023-10-24.tar.xz.sig download genodelabs/src/base-linux/2023-10-24.tar.xz download genodelabs/src/base-linux/2023-10-24.tar.xz.sig download genodelabs/src/init/2023-10-24.tar.xz download genodelabs/src/init/2023-10-24.tar.xz.sig download genodelabs/api/base/2023-10-24.tar.xz download genodelabs/api/base/2023-10-24.tar.xz.sig download genodelabs/api/os/2023-08-21.tar.xz download genodelabs/api/os/2023-08-21.tar.xz.sig download genodelabs/api/report_session/2023-05-26.tar.xz download genodelabs/api/report_session/2023-05-26.tar.xz.sig download genodelabs/api/sandbox/2023-10-03.tar.xz download genodelabs/api/sandbox/2023-10-03.tar.xz.sig download genodelabs/api/timer_session/2023-10-03.tar.xz download genodelabs/api/timer_session/2023-10-03.tar.xz.sig Genode sculpt-23.10 17592186044415 MiB RAM and 18997 caps assigned to init
You can find bash.tar added to the var/run/ directory, which comprises all the ROM modules of our Genode system.
Of course, we cannot start a TAR archive. It is not an executable after all. We rather need to access the content of the archive. Here, the combination of three Genode components namely VFS, fs_rom, and init comes to the rescue.
-
The VFS server is able mount a TAR archive locally as a virtual file system and offer its content as a file-system service.
-
The fs_rom component provides a ROM service by fetching the content of ROM modules from a file system. By connecting the fs_rom with the VFS component, the files of the bash.tar archives become available as ROM modules. With the bash executable binary accessible, we can execute it.
-
The init component allows us to stick components together and let the result appear to the surrounding system as a single component. We can use it to host the composition of the VFS, fs_rom, and bash.
Note that our pkg/unix/runtime refers to Genode's init component in the attribute binary="init". So as a whole, our subsystem will be an instance of init. Internally, init will host several child components and manage their resources and relationships according to its configuration. Let's start with a fresh init that hosts only the VFS server by replacing our empty <config/> in our pkg/unix/runtime file by the following configuration.
<config>
<parent-provides>
<service name="ROM"/>
<service name="LOG"/>
<service name="RM"/>
<service name="CPU"/>
<service name="PD"/>
<service name="Timer"/>
</parent-provides>
<start name="vfs" caps="100">
<resource name="RAM" quantum="10M"/>
<provides> <service name="File_system"/> </provides>
<config>
<vfs> <tar name="bash.tar"/> </vfs>
<default-policy root="/" />
</config>
<route> <any-service> <parent/> </any-service> </route>
</start>
</config>
The <default-policy> expresses that any client should be able to access the root of the virtual file system in a read-only fashion.
When trying to run the scenario now, we see a bunch of messages:
unix$ goa run
[unix] config <parent-provides> mentions a timer service;
consider adding <timer/> as a required runtime service
Genode sculpt-23.10
17592186044415 MiB RAM and 18997 caps assigned to init
[init -> unix] Error: vfs: environment ROM session denied ...
The first message points out that init's <parent-provides> declaration refers to a service that should better also be announced as a requirement in the runtime file. This can be done by adding the following <requires> node inside the <runtime> node.
<runtime>
...
<requires>
<timer/>
</requires>
...
</runtime>
The subsequent "Error:" messages tell us that init requested the ROM module vfs that is not available to the scenario, yet. To make this ingredient available to our scenario, we have to declare it in the archives and as <content> in the runtime file. While we are at it, lets also capture the need for init because our entire scenario is based on this component. Let's add the following lines to pkg/unix/archive:
genodelabs/src/vfs genodelabs/src/init
Also make sure to have the ROM modules listed as <content> in the pkg/unix/runtime so that it looks as follows:
<content> <rom label="init"/> <rom label="bash.tar"/> <rom label="vfs"/> </content>
When issuing goa run again, we can see Goa downloading the additional components. On the attempt to start the scenario, we are confronted with another error message:
[init -> unix -> vfs] Error: Could not open ROM session for "vfs.lib.so"
This message tells us that the VFS server requests another ROM module, which is a shared library. The vfs.lib.so contains the actual implementation of the virtual file system. It comes in the form of a library to enable its use either locally by an individual application or via the VFS server. The library is part of the genodelabs/src/vfs archive that is already listed in our archives file. So we can resolve this error by adding a corresponding <rom> entry to the runtime file. The <content> should now look as follows:
<content> <rom label="init"/> <rom label="bash.tar"/> <rom label="vfs"/> <rom label="vfs.lib.so"/> </content>
When running the scenario again, we see a sign of hope:
unix$ goa run
Genode sculpt-23.10 17592186044415 MiB RAM and 18997 caps assigned to init
No further errors! That means that the VFS server is running and has presumably mounted the bash.tar archive. On a second terminal, you can indeed observe the VFS server showing up.
$ ps u
... [Genode] init ... [Genode] init -> timer ... [Genode] init -> unix ... [Genode] init -> unix -> vfs
The second piece of the puzzle is the fs_rom server, which can be added to the <config> node of pkg/unix/runtime with the following snippet:
<start name="vfs_rom" caps="100">
<resource name="RAM" quantum="10M"/>
<binary name="fs_rom"/>
<provides> <service name="ROM"/> </provides>
<config/>
<route>
<service name="File_system"> <child name="vfs"/> </service>
<any-service> <parent/> </any-service>
</route>
</start>
By using the <binary> node, we can label the component in a meaningful way, calling it "vfs_rom". The first entry of the <route> node defines that the request for a file-system session should be routed to the "vfs" component.
On the next attempt to issue goa run, we face an error message:
[init -> unix] Error: vfs_rom: environment ROM session denied
By now, I'm sure you know how to resolve this one. Corresponding entries to your archives file and the runtime file's <content> are added swiftly. The fs_rom component gives us no life sign, which is normal. If you want to get a little bit more action on screen, you may add the verbose="yes" attribute to init's <config> node. Another try of goa run reveals the following output.
unix$ goa run
Genode sculpt-23.10 17592186044415 MiB RAM and 18997 caps assigned to init [init -> unix] parent provides [init -> unix] service "ROM" [init -> unix] service "LOG" [init -> unix] service "RM" [init -> unix] service "CPU" [init -> unix] service "PD" [init -> unix] service "Timer" [init -> unix] child "vfs" [init -> unix] RAM quota: 9992K [init -> unix] cap quota: 66 [init -> unix] ELF binary: vfs [init -> unix] priority: 0 [init -> unix] provides service File_system [init -> unix] child "vfs_rom" [init -> unix] RAM quota: 9992K [init -> unix] cap quota: 66 [init -> unix] ELF binary: fs_rom [init -> unix] priority: 0 [init -> unix] provides service ROM [init -> unix] child "vfs" announces service "File_system" [init -> unix] child "vfs_rom" announces service "ROM"
That looks promising. Now with the bash executable available as ROM module, let's give the bash shell a spin:
<start name="/bin/bash" caps="1000">
<resource name="RAM" quantum="10M" />
<config>
<libc stdin="/dev/null" stdout="/dev/log" stderr="/dev/log"
rtc="/dev/null"/>
<vfs>
<dir name="dev"> <null/> <log/> </dir>
</vfs>
<arg value="bash"/>
<arg value="-c"/>
<arg value="echo files at /dev: /dev/*"/>
</config>
<route>
<service name="ROM" label_last="/bin/bash">
<child name="vfs_rom"/> </service>
<any-service> <parent/> </any-service>
</route>
</start>
The following parts are worth highlighting:
-
The bash has its own VFS! This has nothing to do with the VFS server we started above. In fact, bash's VFS - as configured by the <vfs> node - merely contains the two pseudo files /dev/null and /dev/log. The latter one is a LOG connection that enables the bash to write messages to the outside world.
-
The <libc> node contains the configuration of the C runtime used by bash. Here we say how the standard output should go, or that the C runtime should obtain its "real-time-clock" information from /dev/null. No time for you this time!
-
Via the sequence of <arg> nodes, we execute the command
echo files at /dev: /dev/*
It uses the shell's file globbing mechanism to obtain the list of files matching the pattern "/dev/*" and prints it via the echo built-in command.
-
The <route> rules explicitly tell init that the binary of the component should be obtained from the "vfs_rom" component.
When trying to goa run the scenario now, we have to add a few more entries to our archives and <content>, specifically because bash uses the C runtime (libc and libm) as well as the posix library. The full list of archives now looks as follows:
genodelabs/src/bash genodelabs/src/vfs genodelabs/src/init genodelabs/src/fs_rom genodelabs/src/libc genodelabs/src/posix
For reference, the <rom> modules listed in the runtime file's <content> node:
<content> <rom label="init"/> <rom label="bash.tar"/> <rom label="vfs"/> <rom label="vfs.lib.so"/> <rom label="fs_rom"/> <rom label="libc.lib.so"/> <rom label="libm.lib.so"/> <rom label="posix.lib.so"/> </content>
Once these stumbling blocks are out of the way, goa run greets us with the following output:
... [init -> unix] child "vfs" announces service "File_system" [init -> unix] child "vfs_rom" announces service "ROM" [init -> unix -> /bin/bash] files at /dev: /dev/log /dev/null [init -> unix] child "/bin/bash" exited with exit value 0
The message "files at /dev: /dev/log /dev/null" is the output of the bash command we have hoped for!
Granted, our scenario is still a far cry from being a Unix system. In the next episode, we will add an interactive terminal. Read on...
Edit (2021-03-02): updated to Genode 21.02
Edit (2023-05-02): updated to Sculpt OS 23.04
Edit (2023-11-15): updated to Sculpt OS 23.10