The goal is to have a basic "framebuffer driver" for mnemos.

I think this would work something like:

• use the embedded graphics simulator, or similar sdl2/web tech to build a gui window
• use embedded-graphics (and maybe embedded-text) as graphics primitives
• provide some kind of interface to draw to the frame buffer, such as:
  • providing an X/Y start coord, and a rectangle to be painted on to the frame buffer
  • provide a way to provide a "full frame update" (this might just be the other option starting at (0, 0) and fully sized).

I think mostly, you'll want a "whole framebuffer", say 800x600, and you should take smaller rectangle chunks of arbitrary size, and draw them to the frame buffer. This way your "demo task" can draw something like a 320x240 chunk using embedded-graphics, and just paint that on to the large, "dumb" framebuffer.

To do this, you will likely need to do the following (short summary):

1. Add a new driver to the kernel that owns the frame buffer and whatever window context that needs to exist.
2. Define a message type for sending update commands.
3. Process incoming commands, update the framebuffer, redraw the framebuffer whenever updated
4. Provide some kind of demo that draws something, or does an animation, or takes data from a virtual port or something and draws it

Longer summary:

make a driver

misc note: The simulator is called "melpomene". You can find it in source/melpomene. You can run it with MELPOMENE_TRACE=trace cargo run. You'll see a lot of tracing events there. If you want to see a lot more, check out this comment which will show you how to run the virtual serial ports that are already working in the simulator.

Drivers in mnemos are basically just async tasks. "real drivers" should be self contained, but for the simulator, you can use whatever. You can spawn threads if you want, and we also have a tokio runtime going that provides sleep and uart stuff.

Check out the sim drivers folder for simulator specific drivers, and kernel drivers for real drivers. You'll probably want to use the Heap types to allocate your frame buffer.

Don't worry about getting any of this perfect yet.

Once you have a driver, you'll probably need to set it up in the initialization function, and spawn it into the kernel's executor. Check out [kernel_entry()](https://github.com/tosc-rs/mnemos/blob/09ef1595527a388fd01ea3bfdc6741561f4a8a04/source/melpomene/src/main.rs#L68) in melpo for how other tasks do this. You might also need to modify how much memory is given to the kernel at initialization. Go ahead and bump it to like 16MiB or whatever you need.

Define message types

mnemos has two main communication channels you'll see. One is stream based, and that's all the ones you see called bbq. I don't think you'll need those for this. The other one is basically an MPSC async queue, called KChannel. You can basically create a kchannel that contains your type (and optionally a "reply" channel producer), and define whatever message types you want. Since it'll probably be relatively "big" buffers, your message should probably also use heap types for this.

You can see an example of a message type in the serial_mux driver, and how a kchannel is used in the current main task.

You might want to have something like a FrameChunk, which holds an appropriately sized heap allocation, and implements that DrawTarget trait from e-g.

This way you could do something like:

// this makes a 320 x 240 x 2 allocation to act as a 16bpp frame chunk
let mut new_frame = FrameChunk::new(320, 240).await;
// This uses e-g methods to paint this sub-frame
Circle::new(...)
    .whatever(...)
    .eg(...)
    .methods(...)
    .draw(&mut new_frame).unwrap();
// send the sub-frame over the kchannel
driver_producer.send(Command::Draw { x: 0, y: 100, chunk: new_frame }).await.unwrap();

Process incoming commands

Again, the serial mux driver is a good example of accepting messages and doing stuff with them. Here you will do whatever your framebuffer logic stuff is.

Provide some kind of demo

Right now, userspace isn't up and running. So just define some async demo task (in melpo's main, or somewhere else), that holds the new driver's kchannel producer, and draw something every 1 second or so.

This branch rewrites the TCP serial port simulated driver to run in Melpomene's Tokio runtime, rather than spawning its own thread. This should make Melpomene a bit more efficient, since the runtime thread can run other simulated drivers while the serial driver task is waiting for IO, instead of doing a bunch of little sleeps.

I think this shouldn't result in any real behavioral change, although I haven't tested it super extensively. If I open crowtty and then kill it after a while, Melpomene still sees the TCP stream disconnection...but I wasn't sure how I was supposed to actually start the simulated serial port after launching crowtty? Any suggestions on further testing would be appreciated! :)

Hey it's james, back with another terrible thread solution to a simulation problem.

This time, I use an OS thread to take ownership of the bidi buffer and directly feed bytes into and out of a TCP port, mapped to port 9999.

This can be opened locally with ncat to see the loopback:

stty -icanon -echo && ncat 127.0.0.1 9999
hello world!

Have tracing report the address of the underlying storage (rather than the address of the Arc itself) so that it is easier to see related comms in traces.

Before:

  10.670354843s TRACE Kernel:Driver B:BBQueue::read_grant{queue=0x7f7758014548 side=BSide}: kernel::bbq: awaiting bbqueue read grant
  11.168783665s  WARN Kernel:Driver A: melpomene: Driver A: Reading...
  11.170023311s TRACE Kernel:Driver A:BBQueue::read_grant{queue=0x7f77580139a0 side=ASide}: kernel::bbq: Got bbqueue read grant size=4
  11.170525095s  WARN Kernel:Driver A: melpomene: Driver A: Got data buf=[245, 245, 245, 245]

After

  2.079232514s TRACE Kernel:Driver A:BBQueue::send_grant_exact{size=8 queue=0x7f08a8014700 side=ASide}: kernel::bbq: Got bbqueue exact write grant
  2.079650683s  INFO Kernel:Driver A: melpomene: Driver A: Sleeping...
  2.080172776s  WARN Kernel:Driver B: melpomene: Driver B: Reading...
  2.080742401s TRACE Kernel:Driver B:BBQueue::read_grant{queue=0x7f08a8014700 side=BSide}: kernel::bbq: Got bbqueue read grant size=8
  2.081164016s  WARN Kernel:Driver B: melpomene: Driver B: Got data buf=[1, 1, 1, 1, 1, 1, 1, 1]
  2.081413875s  INFO Kernel:Driver B: melpomene: Driver B: Writing...

This builds on PR #5, but instead of spawning a separate OS thread for every Sleepy future, we instead change the simulator to run in a single-threaded Tokio runtime, with each sleep driven by a task spawned on that runtime. This way, we only need a single thread to drive all of the simulator services.

In the future, we can also use the same runtime to drive any simulator services that use I/O (e.g. implementing a simulated serial port using a host TCP socket). We may also want to use Tokio synchronization primitives to block the simulator rt on kernel initialization, instead of the current spin loop, but I'll address that in a separate PR.

This adds a number of u32 ID fields to each message in order to make tracing interactions between services easier.

u32 might be too much (I suspect u16 is probably enough, at least for service/client IDs), but I think this is a fine place to start. Most of the metadata insertion is "hidden" from end-consumers, happening in the built-in client and service handles, which means it should be hard to get wrong or forget to do.

Prior to this, both the console addr and modality addr were getting the same default constructor, meaning both the console and modality were trying to bind/connect to port 6699, which seems like a weird and fun behavior of flatten. I've made the names more verbose now to avoid the issue.

Provides an implementation of #25.

There aren't any major conceptual changes here, but some implementation tweaks were made.

Additionally, I've gone ahead and "fully broken" userspace comms (the existing structures don't totally fit, and fixing them would require fixing some other underlying details, which I will do as a follow-up in a later RFC to restore userspace comms). The largest pre-req for this fix is deciding how to do async/await notifications across the ABI barrier.

Currently, this repo contains three separate Cargo workspaces: one for source/, which contains the actual mnemOS crates, and then a separate root workspace for each crate in tools/.

This is a bit unfortunate, as it means that it's not possible to run cargo run --bin WHATEVER from the root directory of the repo. Instead, in order to run melpomene, it's necessary to cd into source. Similarly, if I want to run tools like crowtty, I need to cd into tools/crowtty. This is very mildly annoying. The lack of a global workspace also means that it's not currently possible to run cargo commands like cargo build --workspace and build everything in one go.

This branch rectifies this by moving the workspace's Cargo.toml to the root of the repo, and including all crates in the workspace (both source/ and tools/). The potential downside of this is that it means there's a lockfile, Cargo.toml, and .cargo/ directory in the root of the workspace, which previously was nice and clean, and didn't have a bunch of random config files and stuff in it. But, I think this is probably not that bad?

If you'd really prefer not to make this change, that's fine --- having to change dirs was really only a minor inconvenience.

This PR adds the "serial mux" which allows for virtual serial ports over a single physical serial link (or in the current case, a single simulated serial port over a TCP stream).

This also significantly reworks the async BBQueue wrapper to allow for three flavors:

• bidi - a bidirectional, dual-spsc channel
• mpsc - a multiple (async mutex'd) producer, single consumer one-way channel
• spsc - a single producer, single consumer one-way channel.

So, as of right now, I have sort of an ambiguous structure around "registering" a driver, vs just spawning a driver task in the kernel.

The original role of "registering" was to act as a routing mechanism for userspace messages. This is why it rejects duplicate "kind" registrations.

IMO, this is sort of good, and sort of not.

I had an idea to essentially have two kinds of drivers:

• Ones that exist as tasks, but only exist in service of some other driver, e.g. they are not "discoverable", they are just a task that does stuff for other drivers that already have access to talking to them (kchannels, bbqueues, etc.)
• Ones that exist as a "discoverable" item, e.g. ones that can be queries by other drivers that are not tightly couple to eachother (e.g. "give me the serial port"), or can be queried/addressed from userspace.

I think the "standard library" will have a handful of official/discoverable driver services (like block devices, serial ports), that will generally be expected to exist. This will fulfill the interface for userspace standard library interfaces.

I also think there is room for "third party" drivers which come with their own userspace libraries, as well as a kernel service.

I think it might make sense to decouple the message types across the user/kernel comms layer, and have the service registrations work as some kind of UUID that defines the service. That way, in userspace, you can say "does the kernel have 1234ABC-... service?" and know whether you can operate with that.

Previously, all kernel async tasks would render within the timeline of the tokio executor that is running the kernel loop.

This change allows us to generate unique timelines per-task. This will only work for the kernel at the moment, and will require adjusting for also splitting off userspace task timelines.

It would be cool to have network capabilities from driverspace, and later pushed through to userspace via the mstd interface.

To start, it would be good to have:

• A driver service that handles the "net" layer, essentially a wrapper over smoltcp, or potentially embassy-net as it is already async
• A driver service that handles the "phy" layer, something on linux for melpo that plugs into smoltcp at layer 2/3. Later this could be used for the wifi or ethernet or usb-networking-gadget kind of interface

I'm not sure what the messaging service interface needs to look like, but my first guess is:

• Register the smoltcp-based net service w/ no phys
  • Maybe register two services, one for phy and one for net
• Start each phy, have it take a smoltcp-phy service client handle. They are not registered on service discovery, they are just tasks holding smoltcp-phy handles
• Users of the network interface can get a smoltcp-net/embassy-net service handle to open sockets, send/recv, etc.
• Write a melpo demo showing ping or a hand crafted GET response or something

we could maybe avoid the linear search of the registry if we had a way for userspace to look up the array index of a driver by its UUID, or something? then, you could do the linear search once for driver discovery, and then reference the driver by an index every time you want to talk to it.

Originally posted by @hawkw in https://github.com/tosc-rs/mnemos/pull/28#discussion_r928150063

There are two "won't scale, but fine for now" issues with the registry:

• Using UUIDs in every user message, meaning a +16 bytes fixed overhead
• Doing a linear search on registry elements

I'd like to keep using this in the "bad" form for a while, to see if any changes need to be made, but there's definitely room on the table for improvement if we keep this like this for now

There are a few things that need to be sorted before I can release the next version of mnemos.

Technical Blockers

• Crates.io release of Maitake
• Probably an updated release of Cordyceps (maybe?)
• An updated release mnemos-alloc
• Some decision around tracing
  • Currently there is sort of a mixed use of tracing 0.1 and tracing 0.2
  • The problem with tracing 0.1 is that it isn't no_std (it requires liballoc)
  • The problem with tracing 0.2 is that it isn't released
  • This can be solved one of many ways:
    • add feature gates for tracing 1/2, with "none" as a possible option
    • release a rogue version of tracing 2
    • switch to a non-tracing library (bad idea)

Bugs

• Probably

Needed Features

• Lots

At some point, I'll spawn userspace programs in the simulator. There are a couple ways I could model this. In no particular order:

Separate Process + TCP Port

I could have the kernel open a new program (e.g. std::process::Command), and pass it a TCP port. The TCP port will basically take the place of the user/kernel ring comms.

Forked process + shared mem

I could have the kernel actually spawn/fork the child process with some shared memory, and have the bbqueues actually live there. bbqueue should be IPC safe, but this will likely require a lot of cursed code

Dynamically loaded library

I could make the user process compile as a dylib, and load it with some fixed entry point. Since it is loaded in the same memory space, I can just share the bbqueue there.

Another question is how to model the programs "existing" on the block device. This might be removed by requiring a "program manager" driver of some kind that abstracts this away at the kernel level.