Goa - streamlining the development of Genode applications
The development of applications for Genode used to require a lot of learning about Genode's way of organizing source code, our custom build system, and the use of run scripts. With Goa, I introduce a new tool that aims at largely removing these burdens from application developers.
In contrast to the tools that come with Genode, which were designed for developing complete systems, Goa is focused on the development of individual applications. In a nutshell, it streamlines the following tasks:
-
The porting of 3rd-party software to Genode, which typically involves
-
Downloading 3rd-party source code via Git, Subversion, or in the form of archives,
-
Applying patches to the downloaded source code, and
-
Keeping track of changes locally made to the downloaded source code.
-
-
Building software using standard build tools like CMake, alleviating the need to deal with Genode's custom build system. Goa takes care of automatically installing of the required Genode APIs and supplying the right parameters to the build system so that Genode executables are produced.
-
Rapidly testing the software directly on the developer's Linux host system. Goa automatically downloads Genode components needed for the test scenario.
-
Since Genode executables are binary-compatible between Linux and microkernels, the same software as just tested on Linux can be deployed on top of the other kernels. Goa takes care of exporting the software in the format expected by Genode's package management.
-
Publishing (archiving and cryptographically signing) the software so that it becomes available to other Genode users, in particular users of Sculpt OS.
A first example, using a plain old Makefile
Let's say, we want to build a hello-world application that uses the raw Genode API with no libc whatsoever.
First, create a project directory, let's call it "hello":
$ mkdir hello hello$ cd hello
By convention, the project name corresponds to the name of the directory. Source codes are stored in a src/ sub directory. Let's create a file at src/hello.cc with the following content:
#include <base/log.h>
#include <base/component.h>
void Component::construct(Genode::Env &)
{
Genode::log("Hello");
}
Besides the hello.cc file, let's create a Makefile at src/Makefile with the following content:
hello: hello.cc
Now, let's give goa a first try:
hello$ goa build
Goa responds with the following message:
[hello] Error: hello has a 'src' directory but lacks an 'artifacts' file.
You may start with an empty file.
The so-called artifacts file tells Goa about the expected end result of the build process. Even though we already know from our Makefile that our only build artifact will be the executable binary called "hello", let's follow Goa's advise of starting with an empty artifacts file.
hello$ touch artifacts
As a notable side effect of the goa build command, Goa has created a new directory called var/ within the project directory. The var/ directory is the designated place for generated files such as the build directory.
Upon the next attempt of issuing the goa build command, now with an artifacts file in place, Goa attempts to compile our program but with pretty limited success:
hello.cc:1:10: fatal error: base/log.h: No such file or directory
#include <base/log.h>
^~~~~~~~~~~~
compilation terminated.
make: *** [hello] Error 1
[hello:make] <builtin>: recipe for target 'hello' failed
Error: build via make failed
Our program tries to include a header file that is nowhere to be found. To resolve this problem, we can tell Goa that our project needs to use the Genode base API, by placing a file named used_apis with the following content into the project directory.
genodelabs/api/base
This line tells Goa that our project depends on Genode's base API that features the base/log.h and base/component.h headers. When issuing the command goa build again, we see the following message:
download genodelabs/api/base/2023-10-24.tar.xz download genodelabs/api/base/2023-10-24.tar.xz.sig
Goa automatically downloaded the base API for us and installed it into a fresh depot at var/depot/genodelabs/api/base/2023-10-24/. But not only that, it also re-attempted the build of the program. If we take a look at var/build/x86_64/, we see the hello executable. If the output was too unspectacular for your taste, you may append the --verbose argument to the goa build command to see more details about the steps taken.
To run the program, we need to tell Goa, which part of the build artifacts are relevant to us. In our case, it's the hello executable binary. We can declare this information in our artifacts file by adding the following line. It refers to the respective file relative to the var/build/x86_64/ directory.
hello
If we issue the goa build command again, we can see that this file appears at var/bin/x86_64/hello. The content of the bin directory is meant for the integration into a Genode scenario.
Speaking of a Genode scenario, to run the program within a Genode system, we have to define the "contract" between the program and the surrounding system. This contract has the form of a runtime package. Let's create one with the name "hello":
hello$ mkdir -p pkg/hello
A runtime package needs at least two files, a README file and a runtime file. The README file should give brief information about the purpose of the Genode subsystem for human readers. The runtime file contains the contractual information. Create a file pkg/hello/runtime with the following content:
<runtime ram="1M" caps="100" binary="hello">
<config/>
<content>
<rom label="hello"/>
</content>
</runtime>
Here we declare the binary we want to start, how much RAM and capabilities the subsystem expects, configuration information passed to the subsystem, and the content of the package. In this case, we only have a single ROM module called "hello", which is our binary.
For running the scenario, we can use the goa run command:
hello$ goa run
download genodelabs/bin/x86_64/base-linux/2023-10-24.tar.xz download genodelabs/bin/x86_64/base-linux/2023-10-24.tar.xz.sig download genodelabs/bin/x86_64/init/2023-10-24.tar.xz download genodelabs/bin/x86_64/init/2023-10-24.tar.xz.sig download genodelabs/src/base-linux/2023-10-24.tar.xz download genodelabs/src/base-linux/2023-10-24.tar.xz.sig download genodelabs/src/init/2023-10-24.tar.xz download genodelabs/src/init/2023-10-24.tar.xz.sig download genodelabs/api/os/2023-08-21.tar.xz download genodelabs/api/os/2023-08-21.tar.xz.sig download genodelabs/api/report_session/2023-05-26.tar.xz download genodelabs/api/report_session/2023-05-26.tar.xz.sig download genodelabs/api/sandbox/2023-10-03.tar.xz download genodelabs/api/sandbox/2023-10-03.tar.xz.sig download genodelabs/api/timer_session/2023-10-03.tar.xz download genodelabs/api/timer_session/2023-10-03.tar.xz.sig Genode sculpt-23.10 17592186044415 MiB RAM and 18997 caps assigned to init [init -> hello] Hello
We can see that Goa automatically installed the dependencies needed to execute the runtime package, integrates a Genode scenario, and executes it directly on Linux. If you switch to another terminal, you can see the Genode processes:
$ ps a | grep Genode 8646 pts/3 Sl+ 0:00 [Genode] init 8649 pts/3 Sl+ 0:00 [Genode] init -> hello 8650 pts/3 Sl+ 0:02 [Genode] init -> timer
You can cancel the execution of the Genode scenario via Control-C.
A second example, using CMake
As another step, let us create a new project that executes the 2nd step of the excellent CMake tutorial. Let's call the project "cmake_step2". Instead of copying the code into the cmake_step2/src/ directory, let us better tell Goa to download the code from the original tutorial. This can be done by creating an import file in the project directory. Create the file cmake_step2/import with the following content:
LICENSE := BSD VERSION := master DOWNLOADS := cmake_step2.sparse-git URL(cmake_step2) := https://github.com/Kitware/CMake REV(cmake_step2) := HEAD DIR(cmake_step2) := src SPARSE_PATH(cmake_step2) := Help/guide/tutorial/Step2
This import file describes the download of only the specified sub directory of the CMake project from GitHub. Let's give it a try:
cmake_step2$ goa import import download https://github.com/Kitware/CMake/trunk/Help/guide/tutorial/Step2 import generate import.hash
After the command finished, we find the source code sitting nicely in a new src/ directory. Let's try to build it just after creating an empty artifacts file.
cmake_step2$ touch artifacts
cmake_step2$ goa build
[cmake_step2:cmake] -- The C compiler identification is GNU 12.3.0
[cmake_step2:cmake] -- The CXX compiler identification is GNU 12.3.0
[cmake_step2:cmake] -- Detecting C compiler ABI info
[cmake_step2:cmake] -- Detecting C compiler ABI info - done
[cmake_step2:cmake] -- Check for working C compiler: /usr/local/genode/tool/23.05/bin/genode-x86-gcc - skipped
[cmake_step2:cmake] -- Detecting C compile features
[cmake_step2:cmake] -- Detecting C compile features - done
[cmake_step2:cmake] -- Detecting CXX compiler ABI info
[cmake_step2:cmake] -- Detecting CXX compiler ABI info - done
[cmake_step2:cmake] -- Check for working CXX compiler: /usr/local/genode/tool/23.05/bin/genode-x86-g++ - skipped
[cmake_step2:cmake] -- Detecting CXX compile features
[cmake_step2:cmake] -- Detecting CXX compile features - done
[cmake_step2:cmake] -- Configuring done
[cmake_step2:cmake] -- Generating done
[cmake_step2:cmake] -- Build files have been written to: .../cmake_step2/var/build/x86_64
[cmake_step2:cmake] Scanning dependencies of target Tutorial
[cmake_step2:cmake] [ 50%] Building CXX object CMakeFiles/Tutorial.dir/tutorial.cxx.obj
.../cmake_step2/src/tutorial.cxx:2:10: fatal error: cmath: No such file or directory
2 | #include <cmath>
| ^~~~~~~
compilation terminated.
Apparently, the example requires the standard C++ library. We can supply this API to the project by creating a used_apis file with the following content:
genodelabs/api/posix genodelabs/api/libc genodelabs/api/stdcxx
The posix API is needed because - unlike a raw Genode component - the program starts at a main function. The libc is needed as a dependency of the Standard C++ library.
When issuing the goa build command again, we see that Goa downloads the required APIs and successfully builds the example program:
cmake_step2$ goa build
download genodelabs/api/libc/2023-10-03.tar.xz download genodelabs/api/libc/2023-10-03.tar.xz.sig download genodelabs/api/posix/2020-05-17.tar.xz download genodelabs/api/posix/2020-05-17.tar.xz.sig download genodelabs/api/stdcxx/2023-10-24.tar.xz download genodelabs/api/stdcxx/2023-10-24.tar.xz.sig [cmake_step2:cmake] -- Configuring done [cmake_step2:cmake] -- Generating done [cmake_step2:cmake] -- Build files have been written to: .../cmake_step2/var/build/x86_64 [cmake_step2:cmake] [ 50%] Building CXX object CMakeFiles/Tutorial.dir/tutorial.cxx.obj [cmake_step2:cmake] [100%] Linking CXX executable Tutorial [cmake_step2:cmake] [100%] Built target Tutorial
The resulting executable binary can be found at var/build/x86_64/Tutorial. Let's declare it a build artifact by mentioning it our artifacts by adding the following line.
Tutorial
To run the program, we need a runtime package that is slightly more advanced than the first hello example. This time, we need declare that our runtime requires content from other depot archives in addition to our program by creating a file pkg/cmake_step2/archives with the following content:
genodelabs/src/posix genodelabs/src/libc genodelabs/src/vfs genodelabs/src/stdcxx
This way, our sub system incorporates the shared libraries found in those depot archives. A suitable pkg/cmake_step2/runtime for running the program within a Genode scenario looks like this:
<runtime ram="10M" caps="1000" binary="Tutorial">
<config>
<libc stdout="/dev/log" stderr="/dev/log"/>
<vfs>
<dir name="dev">
<log/>
</dir>
</vfs>
<arg value="Tutorial"/>
<arg value="24"/>
</config>
<content>
<rom label="Tutorial"/>
<rom label="posix.lib.so"/>
<rom label="libc.lib.so"/>
<rom label="libm.lib.so"/>
<rom label="stdcxx.lib.so"/>
<rom label="vfs.lib.so"/>
</content>
</runtime>
Since the tutorial uses the C runtime, we have to supply a configuration that defines how the virtual file system of the component looks like, and where the program's standard output should go. We also specify the first and second arguments of the POSIX program as "Tutorial" (name of the program) and "24" as its actual argument. The <content> lists all ROM modules required.
With this runtime package in place, let's give the Tutorial a run:
cmake_step2/$ goa run
[cmake_step2:cmake] -- Configuring done [cmake_step2:cmake] -- Generating done [cmake_step2:cmake] -- Build files have been written to: .../cmake_step2/var/build/x86_64 [cmake_step2:cmake] [100%] Built target Tutorial download genodelabs/bin/x86_64/libc/2023-10-24.tar.xz download genodelabs/bin/x86_64/libc/2023-10-24.tar.xz.sig download genodelabs/bin/x86_64/posix/2023-10-24.tar.xz download genodelabs/bin/x86_64/posix/2023-10-24.tar.xz.sig download genodelabs/bin/x86_64/stdcxx/2023-10-24.tar.xz download genodelabs/bin/x86_64/stdcxx/2023-10-24.tar.xz.sig download genodelabs/bin/x86_64/vfs/2023-10-24.tar.xz download genodelabs/bin/x86_64/vfs/2023-10-24.tar.xz.sig download genodelabs/src/libc/2023-10-24.tar.xz download genodelabs/src/libc/2023-10-24.tar.xz.sig download genodelabs/src/posix/2023-10-24.tar.xz download genodelabs/src/posix/2023-10-24.tar.xz.sig download genodelabs/src/stdcxx/2023-10-24.tar.xz download genodelabs/src/stdcxx/2023-10-24.tar.xz.sig download genodelabs/src/vfs/2023-10-24.tar.xz download genodelabs/src/vfs/2023-10-24.tar.xz.sig download genodelabs/api/base/2023-10-24.tar.xz download genodelabs/api/base/2023-10-24.tar.xz.sig download genodelabs/api/block_session/2023-05-26.tar.xz download genodelabs/api/block_session/2023-05-26.tar.xz.sig download genodelabs/api/file_system_session/2023-05-26.tar.xz download genodelabs/api/file_system_session/2023-05-26.tar.xz.sig download genodelabs/api/net/2023-07-13.tar.xz download genodelabs/api/net/2023-07-13.tar.xz.sig download genodelabs/api/os/2023-08-21.tar.xz download genodelabs/api/os/2023-08-21.tar.xz.sig download genodelabs/api/report_session/2023-05-26.tar.xz download genodelabs/api/report_session/2023-05-26.tar.xz.sig download genodelabs/api/rtc_session/2023-05-26.tar.xz download genodelabs/api/rtc_session/2023-05-26.tar.xz.sig download genodelabs/api/so/2020-05-17.tar.xz download genodelabs/api/so/2020-05-17.tar.xz.sig download genodelabs/api/terminal_session/2023-05-26.tar.xz download genodelabs/api/terminal_session/2023-05-26.tar.xz.sig download genodelabs/api/timer_session/2023-10-03.tar.xz download genodelabs/api/timer_session/2023-10-03.tar.xz.sig download genodelabs/api/vfs/2023-05-26.tar.xz download genodelabs/api/vfs/2023-05-26.tar.xz.sig download genodelabs/bin/x86_64/base-linux/2023-10-24.tar.xz download genodelabs/bin/x86_64/base-linux/2023-10-24.tar.xz.sig download genodelabs/bin/x86_64/init/2023-10-24.tar.xz download genodelabs/bin/x86_64/init/2023-10-24.tar.xz.sig download genodelabs/src/base-linux/2023-10-24.tar.xz download genodelabs/src/base-linux/2023-10-24.tar.xz.sig download genodelabs/src/init/2023-10-24.tar.xz download genodelabs/src/init/2023-10-24.tar.xz.sig download genodelabs/api/sandbox/2023-10-03.tar.xz download genodelabs/api/sandbox/2023-10-03.tar.xz.sig Genode sculpt-23.10 17592186044415 MiB RAM and 18997 caps assigned to init [init -> cmake_step2] The square root of 24 is 4.89898 [init] child "cmake_step2" exited with exit value 0
We see that Goa takes care of downloading all dependencies needed to host the subsystem and subsequently executes the scenario. The program built by the tutorial prints the result "The square root of 24 is 4.89898".
The next step
The intro above covers just a tiny part of the scope of Goa.
For further exploration, read the next episode about using Goa to create a little Unix...
Please also feel welcome to explore Goa on your own. A good starting point would be the built-in help command:
goa help
If you have feedback, please don't hesitate to reach out!
Edit (2021-03-02): updated to Genode 21.02
Edit (2023-05-02): updated to Sculpt OS 23.04
Edit (2023-11-15): updated to Sculpt OS 23.10