Some tweaks to be able to use ocaml-solo5 with miou

[dinosaure]

The first commit is just about a STUB_IGNORE. Miou allocates a Stdlib.Condition.t but it does not use it (it does not use pthread_cond_wait). However, OCaml try to delete the value (because it is unused) with pthread_cond_destroy.

The second commit is probably more controversial and difficult to explain if we take into account the history of how to build an unikernel with dune but my experiments show that it's probably the best to actually be able to compile a really simple unikernel without opam monorepo. We must ensure that this commit does not break mirage-skeleton but it should not. A nice try will be the crypto unikernel to see if we bring correctly the archive of mirage-crypto compiled by dune (and not the archive available via opam).

[hannesm]

We discussed this PR at the MirageOS bi-weekly meeting.

The first commit is really fine, please open a separate PR and we can merge that immediately.

The second commit, we like the idea very much. But we are unsure. Especially about safety: how can we assure that C stubs compiled for the host system are not linked into the unikernel? How does this work in respect to cross-compilation?

Focusing on the first question, we could look into installed META files whether they link archives, or have a flag in opam files when they don't include C stubs (rather hard, since that'd need to be done across the opam-repository).

[dinosaure]

Especially about safety: how can we assure that C stubs compiled for the host system are not linked into the unikernel? How does this work in respect to cross-compilation?

If we stay within the environment of the mirage tool and opam-monorepo. The problem doesn't arise, as dune prefers to use the available sources rather than what may be available via OPAM. So, as far as C stubs are concerned, dune will always prefer to compile those available from the duniverse/ folder rather than use those that have been compiled (via the host toolchain) and are available via OPAM.

From a strictly Mirage user point of view, this shouldn't change anything.

However, if you want to compile a unikernel outside the framework offered by the mirage tool (and dune). The question arises. However, this depends on the tool you use to build the unikernel (dune without the mirage tool, ocamlfind or ocamlopt) and the dependencies you need (whether they have C stubs or not).

From my point of view, being able to compile a unikernel using the libraries available in OPAM is a case that remains, in view of the design decisions that have been made since MirageOS 4, at the margin. It's also a case that's under-documented (if at all). The risk of linking archives that have been compiled with the host toolchain is therefore essentially for expert use.

[hannesm]

From a strictly Mirage user point of view, this shouldn't change anything.

That sounds good. Thanks for your explanation. I'd suggest to test (or someone with ocamlfind/dune expertise jumps in) that this is really the case, and if the answer is yes - let's merge and release! :)

[shym]

To elaborate a bit on the question of safety, even if I now understand that this is meant to be “Cool if that works for simple cases”:

• have you an idea whether there is some risky packages that discover their environment we would care about for that application? (I’m thinking about something similar to what lwt.unix does, where its build system is discovered dynamically)
• I’m curious whether you’ve hooked this up with your reproducible builds, to see whether we end up with the same result on both cases, with the already built packages in the OPAM switch and with the packages built locally with the specific Solo5 target.

[shym]

Extra note for the context I have in mind: I’ve heard vaguely about some preliminary work to teach the OCaml compiler to detect incompatibilities between .cmxa when they were built with inconsistent (configure-time? I’m not sure) parameters of the compiler. So this sort of issue must have appeared in use cases, even without cross compilation being involved.

[dinosaure]

have you an idea whether there is some risky packages that discover their environment we would care about for that application? (I’m thinking about something similar to what lwt.unix does, where its build system is discovered dynamically)

This type of package, in this case, could not be used because unix.cmxa is unavailable via the Solo5 toolchain. Once again, I'd like to stress that the possibility of using the *.cmx{,a} available from OPAM is marginal and is in no way presented as a solution for building unikernels in MirageOS.

It is obvious that libraries such as gmp, mirage-crypto, lwt.unix will cause problems if we try to use them from OPAM (and therefore compile from the host toolchain) in a unikernel.

However, if we consider what is specific and what is available, it is clear that, prior to this patch, we are forbidding ourselves from using the vast majority of libraries because some of them (and they can be counted on the fingers of one hand) require to be fully compiled with the Solo5 toolchain.

We also have to admit that these issues are more related to the question of cross-compiling with OCaml than with the ecosystem as it is evolving. It's also an issue that's broader than MirageOS (we can take the example of Melange which, as far as dune is concerned, runs into the same problems as us).

What is certain is that compiling libraries such as fmt, for example, with the host toolchain or the Solo5 toolchain produces the same artefacts.

I’m curious whether you’ve hooked this up with your reproducible builds, to see whether we end up with the same result on both cases, with the already built packages in the OPAM switch and with the packages built locally with the specific Solo5 target.

In this case, I just specified, when compiling fmt, to use the Solo5 toolchain and I did a diff on the *.cmxa obtained with the one available on OPAM. The example is small and there are certainly cases where this reproducibility is no longer true. Once again, the aim is not to initiate a new possibility of building a unikernel with OPAM packages but to leave this possibility to the experts so that they can iterate quickly.

Extra note for the context I have in mind: I’ve heard vaguely about some preliminary work to teach the OCaml compiler to detect incompatibilities between .cmxa when they were built with inconsistent (configure-time? I’m not sure) parameters of the compiler. So this sort of issue must have appeared in use cases, even without cross compilation being involved.

Sorry, but I'm not aware of any such proposal, but if it's in the pipeline, it could indeed be interesting.

[hannesm]

Extra note for the context I have in mind: I’ve heard vaguely about some preliminary work to teach the OCaml compiler to detect incompatibilities between .cmxa when they were built with inconsistent (configure-time? I’m not sure) parameters of the compiler. So this sort of issue must have appeared in use cases, even without cross compilation being involved.
Sorry, but I'm not aware of any such proposal, but if it's in the pipeline, it could indeed be interesting.

I found the following discussion: ocaml/ocaml#13423 (comment) -- and I think for us this is not really relevant, since we build the compiler with the same configure options as the one in the switch (that's why we depend on ocaml-option-no-flat-float-array etc.). So, my assumption is that the OCaml compilation results in the exact same artifact using ocaml-solo5's compiler or the switch compiler.

There may be differences for e.g. fmt and other buenzli libraries -- since with the current mirage approach using duniverse/ocaml-monorepo, we use some overlays https://github.com/dune-universe/opam-overlays/ that have (a) eventually a different version than the main opam repository packages and (b) arbitrary patches influencing the build system (e.g. "use dune" -- where using dune may have an effect on which compilation flags are used, and thus the OCaml compiler may behave differently).

TL;DR: I appreciate the possibility to use the switch-installed opam packages, and at the same time understand safety concerns (link C code compiled for host instead of unikernel). As far as I understand, within this repository we can't fully solve the safety issues. We have to rely on users of this repository to avoid that (as it is done nowadays using mirage & opam-monorepo). Any other tooling on top of ocaml-solo5 will need to take these safety considerations as well.

[hannesm]

@shym (or anyone else): I'll merge and cut a release unless you've concerns.

[shym]

@shym (or anyone else): I'll merge and cut a release unless you've concerns.

No longer from me, thanks!