Skip to content

Commit

Permalink
[larry-robotics#3] Finalize announcement
Browse files Browse the repository at this point in the history
  • Loading branch information
elfenpiff committed Dec 11, 2023
1 parent 675a940 commit 60ca367
Showing 1 changed file with 150 additions and 31 deletions.
181 changes: 150 additions & 31 deletions ANNOUNCEMENT.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,51 @@
# Brace Yourself Rust Is Coming - Announcing iceoryx2

Over a year ago, I embarked on a journey to learn Rust—a language heralded for its
I'm thrilled to unveil a significant milestone in the development of iceoryx —
I am proud to announce the release of iceoryx generation 2, crafted entirely in
Rust. As one of the main developers of iceoryx, this journey commenced over a
year ago as a personal side project.

Iceoryx2, or iceoryx generation 2, is poised to inherit all the familiar
features you've come to rely on from its predecessor. In its initial release,
expect support for publish-subscribe, service discovery, and dedicated event
messaging — all without the need for a central broker (RouDi).
Furthermore, iceoryx2 is designed to seamlessly run on Linux, FreeBSD, and Windows.

Our ongoing efforts extend beyond this initial release. We are actively
developing request-response messaging, refining the waitset, and expanding
platform support to include macOS and Android. While these exciting features
are in the pipeline, their official integration is slated for the coming year
due to time constraints.

One of the most notable enhancements with iceoryx2 lies in its performance over
iceoryx, demonstrating a significant increase across various benchmarks.
Explore the detailed [benchmarks]() to witness the tangible improvements.

However, the true gems of this upgrade manifest through the adoption of Rust.
Switching to Rust has unlocked several advantages that I find particularly
compelling:

- **Comprehensive Documentation:** Thanks to `cargo doc`, iceoryx2 boasts a
consistent and detailed documentation, enriched with numerous examples.

- **Enhanced Safety:** The Rust language inherently provides a safer API,
eliminating common pitfalls such as lifetime and hidden concurrency issues
— a challenge often encountered in C++.

- **Simplified Certification:** From my perspective, certifying the code base
becomes more straightforward with Rust. The language effectively addresses
numerous challenges inherent in C++, contributing to a more reliable and
secure version of iceoryx.

As we step into this new era with iceoryx generation 2, powered by Rust, we
invite you to explore the enhanced capabilities and witness firsthand the
positive impact on both performance and reliability. Your feedback is
invaluable as we continue to refine and expand iceoryx2 to meet the evolving
needs of our user community.

## Why Rust

Over a year ago, I embarked on a journey to learn Rust — a language heralded for its
emphasis on memory safety, concurrency, and reliability. The catalyst for this
exploration was the growing industry consensus favoring memory-safe languages, as
highlighted by the NSA's endorsement of
Expand All @@ -14,9 +59,11 @@ and the industry witnesses a paradigm shift away from C/C++ to Rust, it is worth
noting that Rust's commitment to safety is not merely a claim; its safety features can
be [rigorously proven](https://research.ralfj.de/phd/thesis-screen.pdf).
This substantiates Rust as a reliable and secure choice and raises the
question: Why does the automotive domain lag behind?
question:

Enter iceoryx2—the fruition of my efforts to reimagine iceoryx from the ground up,
Why does the automotive domain lag behind?

Enter iceoryx2 — the fruition of my efforts to reimagine iceoryx from the ground up,
exclusively in Rust. This venture aimed not only to comprehend the language but also
to evaluate whether Rust truly lived up to its promises. The resounding answer is a
definitive "yes," and more.
Expand All @@ -41,9 +88,12 @@ Rust takes the lead in inter-process zero-copy communication within our speciali
domain. Iceoryx2 is not just an upgrade—it's a leap forward into a safer, more
efficient era.

## Example: Receiving Data
## Example: Sending Data

In both iterations of iceoryx, the communication revolves around ports as endpoints. In this context, the `Publisher` assumes the role of the sender in a publish-subscribe messaging pattern. When users intend to transmit data, they typically follow a set of common steps across both versions.
In both iterations of iceoryx, the communication revolves around ports as endpoints.
In this context, the `Publisher` assumes the role of the sender in a publish-subscribe
messaging pattern. When users intend to transmit data, they typically follow a set
of common steps across both versions.

1. Invoke `Publisher::loan()` to obtain a `Sample` for storing the data to be sent.
2. Utilize `Publisher::send(sample)` to dispatch the data to all receiving endpoints (`Subscriber`).
Expand Down Expand Up @@ -130,36 +180,105 @@ but also strengthens the overall integrity of the codebase.
## Lifetimes And Accidental Concurrency
The `Sample` that is returned by the `Publisher` represents a memory
resource that could be leaked when not handled correctly. So we can use in both
languages the
[RAII](https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization)
idiom. In essence, we define the class/struct as the owner of the resource and
as soon as the object goes out of scope the resource is released. In our case,
the memory is deallocated.
When a user calls `Publisher::loan()` and decides that they no longer want
to send the `Sample`, all they have to do is destroy the acquired `Sample`. The
destructor of the `Sample` ensures that the memory is safely returned to the
`Publisher`.
So no problem?
Examining our earlier example, we find that the `Sample` returned by the
`Publisher` represents a memory resource susceptible to leaks if mishandled.
To address this, both C++ and Rust leverage the
[RAII](https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization)
idiom, where the class/struct serves as the resource owner, ensuring that the
memory is released when the object goes out of scope.
Let me ask you:
However, subtle issues arise:
1. What happens when the `Publisher` goes out of scope before the `Sample`?
2. What happens when the `Sample` is moved into another thread and goes
out of scope? Then we accidentally access the `Publisher` concurrently
without even realizing it.
1. What happens when the `Publisher` goes out of scope before the `Sample`?
2. What if the `Sample` is moved into another thread and goes out of scope,
leading to accidental concurrent access of the `Publisher`?
### C++
### Rust
## Concurrency
In C++, the lifetime issue can be tackled by using a `std::shared_ptr`,
where the `Publisher` owns an inner construct, and the `Sample` holds a copy of
that shared pointer. This way, the inner construct is removed when there are no
more owners. In our domain, constraints on using the heap or most STL constructs
add complexity, but solutions involving reference counting and external memory
locations are viable, albeit with drawbacks.
Moving a `Sample` to another thread, causing concurrent use of the `Publisher`,
demands more careful consideration. Options include making the `Publisher`
universally thread-safe (with potential performance impact) or a runtime
detection in the `Sample` destructor using a thread-local variable.
Unfortunately, both solutions introduce performance overhead and necessitate
thorough concurrent stress testing.
The least robust option, which I would advise against in a safety-critical
library, is to rely solely on documenting the restrictions. However, this
approach raises pertinent questions:
* Will every developer diligently read and adhere to the documentation before
utilizing a function?
* Can we trust reviewers to thoroughly examine the documentation of all
functions they review?
* In the dynamic environment of code evolution, will developers be cognizant
of all affected code when contracts change?
In my view, the answer to all three questions tends to be "no." Unless
addressed by automated tools, relying solely on documentation risks inevitable
mistakes.
Consequently, all available solutions present trade-offs: they may introduce a
performance hit, entail implementation overhead, or result in an API that is
susceptible to misuse when documentation is not meticulously reviewed.
## TODO
### Rust
* loan_uninit,
* one argument was no certified compiler, now its there - so whats your excuse
to not use rust in a new project of a mission critical system
Rust, with its strong focus on safety, resolves these challenges at the compiler
level. By explicitly adding the `'publisher` lifetime in the function declaration,
```rust
fn loan(&'publisher self) -> Sample<'publisher> {}
```
we inform the compiler that the `Sample` can exist at most as long as the
`Publisher`. If a user violates this contract, attempting to destroy the
`Publisher` while holding a `Sample`, the compiler prevents the program from
compiling — providing an optimal solution without added overhead.

Accidental concurrency issues, such as moving a `Sample` to another thread, are
also impossible in Rust. Objects intended for thread movement must implement
the `Send` trait, which is not implemented by default. Any attempt to violate
this contract is again caught by the compiler during compilation, demonstrating
Rust's ability to handle such problems at compile time without introducing
performance or implementation overhead.

## Future

This leads me back to the initial question: Why does the automotive domain lag
behind?

A mere year ago, when I embarked on the development of iceoryx2, the absence of
a certified Rust compiler posed a critical barrier. Without this fundamental
component, certifying code in the automotive domain seemed nearly insurmountable.
Additionally, projects within the automotive sector often span years and
frequently involve interactions with legacy C and C++ code bases—essential
factors that cannot be overlooked.

Today, however, the landscape has evolved significantly. We now have access to
a [certified Rust compiler](https://ferrous-systems.com/ferrocene/). Rust
facilitates seamless invocation of functions from C code bases, thanks to
[bindgen](https://github.com/rust-lang/rust-bindgen), and although incorporating
C++ code poses some challenges, it is achievable. Moreover, the reverse is also
true—integrating a Rust library into an existing C/C++ code base and invoking
[Rust functions within C](https://doc.rust-lang.org/nomicon/ffi.html) is feasible.

In my estimation, the opportune moment has arrived to take the next stride and
emerge as pioneers in successfully certifying code for ISO 26262 (ASIL D), with
iceoryx2 as the ideal project for this endeavor. While this undertaking will
undoubtedly require time, particularly given the intricacies of inter-process
zero-copy communication, I approach it with confidence. Drawing on my experience
certifying C++ code and the insights gained from rewriting iceoryx in Rust, I
believe we can navigate these challenges more efficiently.

Embracing Rust not only accelerates the development of new software components
but also reduces costs, courtesy of the modern safety and developer features it
offers. This, in turn, empowers our customers to write superior code swiftly,
leading to substantial cost savings.

I invite you to follow our journey, staying tuned until the day I pen another
blog article proudly announcing the safety certification of iceoryx2.

0 comments on commit 60ca367

Please sign in to comment.