Interesting, didn't hear from this system so far. Seems to be funded by the EU. Apparently it is written in pure Rust since 2020, and Andrew "bunnie" Huang seems to be involved.
From the talk linked above, they went to considerable effort to design a system with a cheap processor which nevertheless contains an mmu, and so most other embedded kernels, which assume the lack of one, are not applicable. So the point of writing in rust is that they can ensure that some of the guarantees of rust are enforced by the hardware. (It's been a while since I watched that talk, so I don't recall exactly which ones). And this is a microkernel, not a monolithic kernel, so they will be using hardware guarantees even between kernel components.
Use of "unsafe" is unavoidable. Various pieces of hardware are directly writing into the address space. Concepts of "ownership" and "mutability" go beyond code semantics.
It's not really about infrastructure but yes kernels and firmwares have to do a lot of stuff the compiler can't verify as safe, eg writing to a magic memory address you obtained from the datasheet that enables some feature of the chip. And that will need to happen in unsafe code blocks. I wouldn't call that a problem but it is a reality.
Are you one of the authors? Concerning the "infrastructure": Rust assumes a runtime, the standard library assumes a stack exists, a heap exists, and that main() is called by an OS; in a kernel, none of this is true. And the borrow checker cannot reason about things like e.g. DMA controllers mutating memory the CPU believes it owns, Memory-mapped I/O where a "read" has side effects (violating functional purity), context switches that require saving register state to arbitrary memory locations, or interrupt handlers that violate the call stack model. That's what I mean by "infrastructure". It's essentially the same issue with every programming language to some degree, but for Rust it is relevant to understand that the "safety guarantees" don't apply to all parts of an operating system, even if written in Rust.
Rust's safety model only applies to code you write in your program, and there's a lot that's unsafe (cannot be verified by the compiler) about writing a kernel or a firmware, agreed. You could have similar problems when doing FFI as well.
Key aspects from the talk iirc (I was in the audience :)):
* Real time embedded CPUs are usually without an MMU -> kernels such as FreeRTOS lack secure memory due to the lack of MMUs in those CPUs
* A kernel targeting embedded CPUs with MMUs that supports secure memory management
* Secure memory communication a there called server/client method to communicate leveraging Rust borrow checker build time for later having "user-land processes" to communicate via pages.
These things combined allow a very small kernel, with user-space implementation of usually kernel-level functionality, such as the system clock timer (presented in the talk).
All of this is meant to provide a complete trustworthy processing chain, from CPU dies that can be inspected through infrared microscopy through the CPU epoxy package/cover to the entire build/software tool chain.
The Xous OS project both takes care of the Kernel, but also the CPU/RISC-V runtime with an MMU, something that is usually quite difficult to obtain - but due to synergy effects with another chip consumer/organization they managed getting their custom processor manufactured.
> Every Xous Server contains a central loop that receives a Message, matches the Message Opcode, and runs the corresponding rust code
Rust? Only Rust?
An OS has no business dictating implementation language. Inside my isolated microservice, I should be able to run anything I damn well please.
I'm a fan of microkernels and microservice models in general, but not if they sacrifice one of their core advantages: arms-length decoupling of implementation strategies through having isolated services communicate only through stable, versioned interfaces.
> A thread connects to a Server using a 128-bit ID. This ID may be textual if the server uses a well-known name that is exactly 16-bytes wide such as b"ticktimer-server" or b"xous-name-server", or it may be a random number generated by a TRNG.
What? This mechanism seems ripe for squatting attacks. How do I know I'm talking to the service I want to contact instead of somebody squatting the name? Using the name namespace for randomly generated IDs (binary!) or an ASCII name stuffed into the same bytes.
Better to give every object on the system its own unique unforgeable, unguessable ID and treat mapping from human-legible names to these strong IDs as its own service, one that can have namespace and authentication policies tailored to a given environment.
> since sending pages of memory is extremely cheap.
Depending on architecture, doing virtual address tricks ranges from expensive to exorbitant. Real-world systems doing bulk transfers over shared memory either rotate among pre-mapped buffers (Wayland, Surface/BufferQueue) or just have the kernel do one efficient scatter/gather memcpy into address space controlled by the recipient (Binder).
I'm not excited by this "lend" IPC primitive Xous has. Seems like more trouble than it's worth. You can add a queue of pre-mapped buffers on top as a separate service if you need it.
> Processes can allocate interrupts by calling the ClaimInterrupt call.
Good! It's about time more people write drivers as regular programs that treat IRQs like any other input event and less as magical things that for some ghastly reason must run with ultimate privileges just to do a DMA once in a while.
That said, just as a matter of elegance, I'd treat an interrupt literally like an regular input source and make it a device node on the FS, not some special kind of resource managed with its own system call.
In Linux terms, I should be able to open /dev/irq/5 and expect it to work like an eventfd. Isn't that elegant?
> ...memory will not be backed by a real page, and will only be allocated by the kernel once you access the page
Ugh. Contractual overcommit. Linux does overcommit too. It's an unfixable mistake. I'm disappointed to see a greenfield OS adopt the same strategy. Doubly so for an embedded system that might want precise control over commit charge.
See, in more mature virtual memory setups, we distinguish between reserving address space (which you do with mmap and such) and reserving allocated capacity (which we call "commit"). If you turn overcommit off on Linux (or use Windows at all) you get an elegant model where you can mmap(..., PROT_NONE) and not have your process "billed" for the memory in your allocated region -- but once you protect(..., PROT_WRITE), you can "charged" for that memory because after the mprotect returns, you're contractually permitted to write to that memory with the expectation you don't segfault or get some kind of "Opps. Just kidding. Don't have the memory after all!" signal.
> IncreaseHeap(usize, MemoryFlags) will increase a program's heap by the given amount
What?? No! sbrk() is a terrible interface. Why get locked into having one region of address space called "the heap"? Modern systems (OpenBSD does especially well here among POSIX systems) don't have a "heap" like a damn PDP-11. Instead, malloc allocates out of memory pools that it internally manages using general purpose mmap. The set of anonymous memory regions so managed is what constitutes the heap. No magic. Kernel doesn't even need to know what a heap is. It speaks only the sweet, soothing language of mmap.
> There are different memory regions in virtual
Wait. There are two dozen hard coded virtual addresses that form an ABI? There goes ASLR. What is this, MS-DOS? Should I load an XMS driver? Maybe shadow video RAM?
> The kernel supports enabling the gdb-stub feature which will provide a gdb-compatible server on the 3rd serial port
Good. Maybe they have build IDs and a symbol server too?
> The loader uses a miniature version of the ELF file format.
Good, but...
> A problem with the ELF format is that it contains a lot of overhead
Bad call. I'm not a big fan of ELF (e.g. relative to PE) but it's not that bad and any conceivable savings in things like dynamic section segment descriptions isn't going to be worth a lifetime of compatibility headaches.
Just use standard ELF. It was designed for computers shittier than the ones in disposable vapes today.
> ELF supports multiple flags. For example, it is possible to mark a section as Executable, Read-Only, or Read-Write.
Yes, but...
> Unfortunately these flags don't work well in practice, and issues can arise from various permissions problems.
Eh, they work fine. (Plus, the section flags aren't relevant. Dead metadata. You don't need a section table at all, technically. An ELF loader cares about the segment table, and segments often span more than one section.)
> The Xous build system uses the xtask concept
You want Yocto. You'll lost a huge chunk of your audience once they learn your OS doesn't build with Yocto. Is it fair? No. Yocto sucks. But it's what the embedded world uses, and if you're already asking them to make a leap of faith using your new OS, you don't ask them to wear a blindfold and learn a new build system at the same time.
("Isn't Yocto for Linux?" You can use Yocto to build whatever OS you want if you don't use Poky, the default Linux distribution the Yocto build system produces. Mechanism vs. policy separation.)
> Push notifications are used when we want to be alerted of a truly unpredictable, asynchronous event that can happen at any time.
IMHO, the more useful distinction is between two-way and one-way messages. For the latter, you don't expect a response, but otherwise every single part of the protocol stack is the same. I wouldn't have made a separate "push notification" facility.
> The Plausibly Deniable DataBase (PDDB) is Xous' filesystem ...features "plausible deniability", which aims to make it difficult to prove "beyond a reasonable doubt" that additional secrets exist on the disk
I'm super happy to see a feature like this integrated into an OS.
> The core of a Mutex is a single AtomicUsize. This value is 0 when the Mutex is unlocked, and nonzero when it is locked.
No priority inheritance? Shame. I'm a huge fan of robust PI in small systems as a way to bound critical operation latencies dynamically.
> An OS has no business dictating implementation language.
As opposed to all those OSes that only publish headers in one language right that require everyone to go through heroic effort to interoperate with it?
> This mechanism seems ripe for squatting attacks. How do I know I'm talking to the service I want to contact instead of somebody squatting the name?
Nobody is going to stop you from typosquatting yourself, no
> As opposed to all those OSes that only publish headers in one language right that require everyone to go through heroic effort to interoperate with it?
Everyone can speak C ABI. It's just a matter of making memory and registers spell out the right things. Only Rust can speak to a Rust API that needs monomorphization before it can be used.
> Nobody is going to stop you from typosquatting yourself, no
True, but if it's about the same amount of work either way, you might as well design the system such that it could conceivably run services at different trust levels.
And if you don't have different trust levels? Why not just stick everything in one address space, like Midori?
I can definitely see a case for something simpler and more streamlined than your typical Unix system, but I imagine that thing being something more this Zephyr or FreeRTOS and not require an MMU. This project does require an MMU and virtual memory.
Is there a PDF version of the book (https://betrusted.io/xous-book/)?
I assume the "kernel" makes heavy use of "unsafe", because all the infrastructure assumed by Rust is not available. Or how was this solved?
The standard library requires a heap and such, but you can enable the no_std attribute to work in environments where they don't exist. https://docs.rust-embedded.org/book/intro/no-std.html
Rust's safety model only applies to code you write in your program, and there's a lot that's unsafe (cannot be verified by the compiler) about writing a kernel or a firmware, agreed. You could have similar problems when doing FFI as well.
Source: I'm writing Rust without a runtime without a heap and without a main function. You can too.
* Real time embedded CPUs are usually without an MMU -> kernels such as FreeRTOS lack secure memory due to the lack of MMUs in those CPUs
* A kernel targeting embedded CPUs with MMUs that supports secure memory management
* Secure memory communication a there called server/client method to communicate leveraging Rust borrow checker build time for later having "user-land processes" to communicate via pages.
These things combined allow a very small kernel, with user-space implementation of usually kernel-level functionality, such as the system clock timer (presented in the talk).
All of this is meant to provide a complete trustworthy processing chain, from CPU dies that can be inspected through infrared microscopy through the CPU epoxy package/cover to the entire build/software tool chain.
The Xous OS project both takes care of the Kernel, but also the CPU/RISC-V runtime with an MMU, something that is usually quite difficult to obtain - but due to synergy effects with another chip consumer/organization they managed getting their custom processor manufactured.
The former is proprietary. The latter kernel is GPL2, similar to Linux.
And seL4 is a kernel, not an OS. And it pretty hard to work with specially if you want any kind of dynamic system.
> Every Xous Server contains a central loop that receives a Message, matches the Message Opcode, and runs the corresponding rust code
Rust? Only Rust?
An OS has no business dictating implementation language. Inside my isolated microservice, I should be able to run anything I damn well please.
I'm a fan of microkernels and microservice models in general, but not if they sacrifice one of their core advantages: arms-length decoupling of implementation strategies through having isolated services communicate only through stable, versioned interfaces.
> A thread connects to a Server using a 128-bit ID. This ID may be textual if the server uses a well-known name that is exactly 16-bytes wide such as b"ticktimer-server" or b"xous-name-server", or it may be a random number generated by a TRNG.
What? This mechanism seems ripe for squatting attacks. How do I know I'm talking to the service I want to contact instead of somebody squatting the name? Using the name namespace for randomly generated IDs (binary!) or an ASCII name stuffed into the same bytes.
Better to give every object on the system its own unique unforgeable, unguessable ID and treat mapping from human-legible names to these strong IDs as its own service, one that can have namespace and authentication policies tailored to a given environment.
> since sending pages of memory is extremely cheap.
Depending on architecture, doing virtual address tricks ranges from expensive to exorbitant. Real-world systems doing bulk transfers over shared memory either rotate among pre-mapped buffers (Wayland, Surface/BufferQueue) or just have the kernel do one efficient scatter/gather memcpy into address space controlled by the recipient (Binder).
I'm not excited by this "lend" IPC primitive Xous has. Seems like more trouble than it's worth. You can add a queue of pre-mapped buffers on top as a separate service if you need it.
> Processes can allocate interrupts by calling the ClaimInterrupt call.
Good! It's about time more people write drivers as regular programs that treat IRQs like any other input event and less as magical things that for some ghastly reason must run with ultimate privileges just to do a DMA once in a while.
That said, just as a matter of elegance, I'd treat an interrupt literally like an regular input source and make it a device node on the FS, not some special kind of resource managed with its own system call.
In Linux terms, I should be able to open /dev/irq/5 and expect it to work like an eventfd. Isn't that elegant?
> ...memory will not be backed by a real page, and will only be allocated by the kernel once you access the page
Ugh. Contractual overcommit. Linux does overcommit too. It's an unfixable mistake. I'm disappointed to see a greenfield OS adopt the same strategy. Doubly so for an embedded system that might want precise control over commit charge.
See, in more mature virtual memory setups, we distinguish between reserving address space (which you do with mmap and such) and reserving allocated capacity (which we call "commit"). If you turn overcommit off on Linux (or use Windows at all) you get an elegant model where you can mmap(..., PROT_NONE) and not have your process "billed" for the memory in your allocated region -- but once you protect(..., PROT_WRITE), you can "charged" for that memory because after the mprotect returns, you're contractually permitted to write to that memory with the expectation you don't segfault or get some kind of "Opps. Just kidding. Don't have the memory after all!" signal.
> IncreaseHeap(usize, MemoryFlags) will increase a program's heap by the given amount
What?? No! sbrk() is a terrible interface. Why get locked into having one region of address space called "the heap"? Modern systems (OpenBSD does especially well here among POSIX systems) don't have a "heap" like a damn PDP-11. Instead, malloc allocates out of memory pools that it internally manages using general purpose mmap. The set of anonymous memory regions so managed is what constitutes the heap. No magic. Kernel doesn't even need to know what a heap is. It speaks only the sweet, soothing language of mmap.
> There are different memory regions in virtual
Wait. There are two dozen hard coded virtual addresses that form an ABI? There goes ASLR. What is this, MS-DOS? Should I load an XMS driver? Maybe shadow video RAM?
> The kernel supports enabling the gdb-stub feature which will provide a gdb-compatible server on the 3rd serial port
Good. Maybe they have build IDs and a symbol server too?
> The loader uses a miniature version of the ELF file format.
Good, but...
> A problem with the ELF format is that it contains a lot of overhead
Bad call. I'm not a big fan of ELF (e.g. relative to PE) but it's not that bad and any conceivable savings in things like dynamic section segment descriptions isn't going to be worth a lifetime of compatibility headaches.
Just use standard ELF. It was designed for computers shittier than the ones in disposable vapes today.
> ELF supports multiple flags. For example, it is possible to mark a section as Executable, Read-Only, or Read-Write.
Yes, but...
> Unfortunately these flags don't work well in practice, and issues can arise from various permissions problems.
Eh, they work fine. (Plus, the section flags aren't relevant. Dead metadata. You don't need a section table at all, technically. An ELF loader cares about the segment table, and segments often span more than one section.)
> The Xous build system uses the xtask concept
You want Yocto. You'll lost a huge chunk of your audience once they learn your OS doesn't build with Yocto. Is it fair? No. Yocto sucks. But it's what the embedded world uses, and if you're already asking them to make a leap of faith using your new OS, you don't ask them to wear a blindfold and learn a new build system at the same time.
("Isn't Yocto for Linux?" You can use Yocto to build whatever OS you want if you don't use Poky, the default Linux distribution the Yocto build system produces. Mechanism vs. policy separation.)
> Push notifications are used when we want to be alerted of a truly unpredictable, asynchronous event that can happen at any time.
IMHO, the more useful distinction is between two-way and one-way messages. For the latter, you don't expect a response, but otherwise every single part of the protocol stack is the same. I wouldn't have made a separate "push notification" facility.
> The Plausibly Deniable DataBase (PDDB) is Xous' filesystem ...features "plausible deniability", which aims to make it difficult to prove "beyond a reasonable doubt" that additional secrets exist on the disk
I'm super happy to see a feature like this integrated into an OS.
> The core of a Mutex is a single AtomicUsize. This value is 0 when the Mutex is unlocked, and nonzero when it is locked.
No priority inheritance? Shame. I'm a huge fan of robust PI in small systems as a way to bound critical operation latencies dynamically.
As opposed to all those OSes that only publish headers in one language right that require everyone to go through heroic effort to interoperate with it?
> This mechanism seems ripe for squatting attacks. How do I know I'm talking to the service I want to contact instead of somebody squatting the name?
Nobody is going to stop you from typosquatting yourself, no
Everyone can speak C ABI. It's just a matter of making memory and registers spell out the right things. Only Rust can speak to a Rust API that needs monomorphization before it can be used.
> Nobody is going to stop you from typosquatting yourself, no
True, but if it's about the same amount of work either way, you might as well design the system such that it could conceivably run services at different trust levels.
And if you don't have different trust levels? Why not just stick everything in one address space, like Midori?
I can definitely see a case for something simpler and more streamlined than your typical Unix system, but I imagine that thing being something more this Zephyr or FreeRTOS and not require an MMU. This project does require an MMU and virtual memory.