I took another whack at this using a few different methods. The time to run is still a bit longer than native GHA jobs, but with a self-hosted runner I think it would be pretty similar.

I'm building the image as the first step in the pipeline, then using that image to run the jobs on downstream. I have introduced cargo-chef to package the dependencies, without needing to compile the code itself.

Using minideb we launch a container that has the rust requirements (but without extras like cargo-chef). This runs the actual make steps, and uses native GHA caching for the rust modules.

I would be happy to setup and own a self-hosted runner and docker registry. I'm happy to chat about that in discord too (kuxaku) :)

Most of the build layers should cache across branches too (using buildkits native GHA cache), so the first time it runs should be the longest.

Biggest Risks:

• I got the caching wrong and the pipeline doesn't have correct dependencies someplace along the test/build process
• The build times are longer, impacting developer feedback times

Out of scope:

• I did not apply the docker image to the deploy step yet.

I kept the original two GHA jobs in the pipeline for reference. If/when this gets merged they can be ran alongside for a bit to ensure its working, or deleted in a second PR.

I'd like to propose using bash (with an aurae cli similar to kubectl or buildah) or Deno to script against aurae instead of moving forward with the Rhai implementation. I am not heavily invested in bash or Deno per se, but I am not sure that Rhai's unique properties serve the project better than other more popular approaches to scripting against API's. In short, why is Rhai better than python, javascript, bash, etc? I tried to focus my thinking on who aurae users probably will be and how that theoretical persona would want to interact with the system. In addition, I am working on a system with some of the same goals as aurae currently, and I'm sure that project also biases my opinions about aurae.

Soooo, from the readme,

AuraeScript follows a similar client paradigm to Kubernetes kubectl command. However, unlike Kubernetes this is not a command line tool like kubectl. AuraeScript is a fully supported programing language complete with a systems standard library. The Aurae runtime projects supports many clients, and the easiest client to get started building with is AuraeScript.

Does it make sense that the easiest client to get started building with is a scripting language users probably aren't familiar with? As a sysadmin, I would much rather reach for bash to get started hacking. I think buildah serves as a great model to emulate (tool intro here). buildah replaces dockerfiles the same way auraescript is attempting to replace yaml for infra configuration. It should be simple to spin up an instance of aurae and configure it with a simple bash script or interactively from the terminal. Bash is the most intuitive answer for this. Moving past simplicity, power users can get pretty far with bash, awk, and a well maintained gnu-style cli such as kubectl or buildah before needing to reach for a beefier (and crucially, more complex to integrate and maintain) scripting language such as python, javascript, perl, or lua.

And speaking of lua, an even simpler question to ask than above is why Rhai instead of lua? I've fiddled with it a bit to mess with my neovim configuration, but I am far from an expert. From my outsider's perspective, lua seems like an active success story for small, embed-able languages.

If the scripting language is important to provide a platform for the aurae standard library, then Deno becomes a very compelling option. Deno markets itself as a v8 runtime that is secure, hackable, and embed-able.

• The Deno global object (Deno.spawn, Deno.test, Deno.exit) can be modified to include runtime specific functionality. You can ship aurae users a full featured runtime & compiler to build their aurae scripts with.
• Javscript & Typescript is a very popular with developers these days, and might help make maintainable infrastructure code understandable to more web developers.
• Deno takes a lot of design inspiration from Go. In fact the standard library is loosely modeled after the Go stdlib. Using Deno gives us a lot of the upsides of using Go without some of the more prickly downsides (see bottom)

A Go Problemo

side note: when building docker we were struggling to provide flexibly replaceable extensions as Go had no support for dynamic loading of libraries. As Docker wanted to ship with "batteries included but changeable", we ended up with those CNI / CSI constructs, where core functionality was pushed to external processes with more or less nice interfaces.

It seems like differences in casing for the generated code is causing property/field values to be lost.

JS/TS use camelCase for properties while rust uses snake_case for fields.

@krisnova confirmed this as an issue on stream. A possible solution is to relax the casing requirements with another macro that tags all the fields with #[serde(alias = "camelCaseFieldName")] as Deno uses serde for serializing/deserializing.

*I actually have a macro for this, that probably needs a little adapting before I can contribute it. I'm just a little time constrained before the break, so wanted to leave this issue as a note for future me. For now, if anyone is trying stuff out, just use hardtoread field names so the casing will be the same throughout.

Upon merging a PR to main (or committing directly to the branch because I am a horrible person) a GitHub action is kicked off to update the static site using GitHub pages. Shortly after the event, the website aurae.io will return a 404.

Looking in the get GitHub pages settings for the repository the domain name value is unexpectedly missing.

Setting the value back to aurae.io fixes the 404 and the site is now updated with the most recent changes from the main branch.

these are all suggestions from the proto best practices guide.

tl;dr:

1. suffix packages with versions
2. suffix services with "Service"
3. prefix request/response with service names
4. include _UNSPECIFIED enum entries for value 0

thoroughly untested WIP

this should give us spans around the cells service and logging should be scoped to that but i'm not seeing any obvious output differences so i'm a bit stumped.

going to read around and see if i can figure out if i'm meant to be seeing anything new.

Can we please break the build and fail a pull request if code creates a rust warning?

We are trying to keep a clean set of code for the project, and we would like to leverage the Makefile commands to ensure that no warnings are generated.

This includes

• Documentation
• Linting
• Unused (dead) code
• etc

Should we consider isolating an Aurae cell at the namespace level as well? If so what are the sane defaults we should assume for every Aurae cell? Which namespaces should we consider, and what do we do given the various kernels and their support and awareness of each namespace?

I believe we will need to develop a logging cache that serves as an in-memory caching layer (Note: in the future it will need to also persist to disk!) for both stdout and stderr streams from an executable within a cell.

We will need to be able to hook into a stream at any moment and have some basic guarantees about the data and how we retrieve it.

WARNING: This PR introduces validation and validation_macros crates which are opinionated in their expectations of the conventions we should use but makes scaffolding code easier ONLY IF you have a cooperative IDE.

Some expectations:

• incoming messages should be validated ASAP.
• we should fail fast (return on first validation error)
• validating the incoming message should return a new type
• the new type will have the same fields, but the field types can, and often should, be different

As part of the PR, I have scaffolded validating the allocate endpoint on the CellService. Steps I had to take.

1. Create a validated struct based on the unvalidated proto message (field_type is needed because the type differs from the unvalidated type):

#[derive(ValidatingType)]
pub(crate) struct ValidatedAllocateCellRequest {
    #[field_type(Option<Cell>)]
    cell: ValidatedCell,
}

2. The macro generated the AllocateCellRequestTypeValidator trait, and an empty AllocateCellRequestValidator struct so I wrote:

impl AllocateCellRequestTypeValidator for AllocateCellRequestValidator {
    
}

3. CLion being a cooperative IDE (hopefully VSC will also be), it provided a "quick-fix" action to generate:

impl AllocateCellRequestTypeValidator for AllocateCellRequestValidator {
    fn validate_cell(
        cell: Option<Cell>,
        field_name: &str,
        parent_name: Option<&str>,
    ) -> Result<ValidatedCell, ValidationError> {
        todo!()
    }
}

4. Implement the todo

impl AllocateCellRequestTypeValidator for AllocateCellRequestValidator {
    fn validate_cell(
        cell: Option<Cell>,
        field_name: &str,
        parent_name: Option<&str>,
    ) -> Result<ValidatedCell, ValidationError> {
        let cell = validation::required(cell, field_name, parent_name)?;

        ValidatedCell::validate(
            cell,
            Some(&validation::field_name(field_name, parent_name)),
        )
    }
}

5. We need a ValidatedCell so (leaving out fields to be succinct here):

#[derive(ValidatedType)]
struct ValidatedCell {
    name: String,
    cpus: String,
}

6. Repeat step 2 but with CellTypeValidator and CellValidator, then step 3 and 4.

 Things to note:

• There are 2 derive macros (ValidatingType, ValidatedType). The only difference is which type has the validate function.
• You can skip decorating fields with the field_type attribute if the validated type is exactly the same as the unvalidated type, but Rust has a great type system, so making types such as struct CellName(String) can help reduce errors in code and provides a place to write reusable code. Ex:

struct CellName(String);

impl validation::ValidatedField<String> for CellName {
    fn validate(
        input: Option<String>,
        field_name: &str,
        parent_name: Option<&str>,
    ) -> Result<Self, ValidationError> {
        let input = validation::required_not_empty(
            input,
            field_name,
            parent_name,
        )?;

        validation::allow_regex(
            &name,
            &validation::DOMAIN_NAME_LABEL_REGEX,
            field_name,
            parent_name,
        )?;

        Ok(CellName(input))
    }
}

We could then adjust ValidatedCell to:

#[derive(ValidatedType)]
struct ValidatedCell {
    #[field_type(String)]
    name: CellName,
    cpus: String,
}

and implement validation like:

impl CellTypeValidator for CellValidator {
    fn validate_name(
        name: String,
        field_name: &str,
        parent_name: Option<&str>,
    ) -> Result<CellName, ValidationError> {
        CellName::validate(Some(name), field_name, parent_name)
    }

    fn validate_cpus(
        _cpus: String,
        _field_name: &str,
        _parent_name: Option<&str>,
    ) -> Result<String, ValidationError> {
        todo!()
    }
}

Hi! I'm brand new to the project and was running through building from source doc. Found a couple missing system dependencies that a few crates depend on.

• pkg-config
• libdbus-1-dev
• libseccomp-dev

I'm building on Ubuntu 22.04 jammy with a fresh Rust 1.65 install. Once installing those dependencies via apt, I was able to build both auraed and auraescript without issue.

I also updated the link to the aurae.io site for the standard library since it was linking to the (now archived?) auraed stdlib README. If that should be reverted and remain linking to the original source, let me know! As far as I read through, the content looked to be the same and I assumed that the doc site will be the authoritative reference for the stdlib going forward.

Log snippets from Make before adding those deps:

error: failed to run custom build command for `libdbus-sys v0.2.2`
...
  --- stderr
  pkg_config failed: Could not run `"pkg-config" "--libs" "--cflags" "dbus-1" "dbus-1 >= 1.6"`
  The pkg-config command could not be found.

error: failed to run custom build command for `libdbus-sys v0.2.2`
...
  --- stderr
  pkg_config failed: `"pkg-config" "--libs" "--cflags" "dbus-1" "dbus-1 >= 1.6"` did not exit successfully: exit status: 1
  error: could not find system library 'dbus-1' required by the 'libdbus-sys' crate

  --- stderr
  Package dbus-1 was not found in the pkg-config search path.
  Perhaps you should add the directory containing `dbus-1.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'dbus-1' found

error: failed to run custom build command for `libseccomp v0.0.2 (https://github.com/containers/youki.git?tag=v0.0.2#0f662dd9)`
...
  --- stderr
  `"pkg-config" "--libs" "--cflags" "libseccomp" "libseccomp >= 2.5"` did not exit successfully: exit status: 1
  error: could not find system library 'libseccomp' required by the 'libseccomp' crate

I think this is safe and have confirmed that the files are regenerated on make

The proto generation wasn't a dependency for making docs but should have been.

Implementation of #197.

Created to source comments on whether this is what we want: a more natural proto API that doesn't match the native cgroup API.

Follow-up to https://github.com/aurae-runtime/aurae/pull/226 so the first 16 commits are duplicated from that branch, unfortunately GitHub won't let me use that PR as the base since it's not under aurae-runtime.

This PR adds an initial Go client under client-go. It only has code generate by buf, a future PR (probably some time tmrw or tuesday) will have a more fleshed out client.

The PR boils down to adding this to buf.gen.yaml, and running make proto.

  - plugin: buf.build/protocolbuffers/go:v1.28.1
    out: client-go/pkg/api
    opt: paths=source_relative
  - plugin: buf.build/grpc/go:v1.2.0
    out: client-go/pkg/api
    opt: paths=source_relative

Follow-up to https://github.com/aurae-runtime/aurae/pull/224 so the first 6 commits are duplicated from that branch, unfortunately GitHub won't let me use that PR as the base since it's not under aurae-runtime.

The previous structure was incompatible with the gRPC Go plugin for the buf tool since the generated Go code would get generated into a single directory with each file having a different package. See https://github.com/aurae-runtime/aurae/pull/227 for this structure.

This restructure also moves the (pod|spawn|instance|cell) services into their own proto file under the runtime package to simply future client code.

The changes boil down to this:

api/v0/observe.proto -> api/observe/v0/observe.proto
api/v0/schedule.proto -> api/schedule/v0/schedule.proto
api/v0/runtime.proto *deleted*
api/runtime/cell/v0/cell.proto *created*
api/runtime/pod/v0/pod.proto *created*
api/runtime/instance/v0/instance.proto *created*
api/runtime/spawn/v0/spawn.proto *created*

See https://developers.google.com/protocol-buffers/docs/style#packages

The package names weren't unique nor did they have anything based on the project name. The renames were as follows:

observe.v0 -> aurae.observe.v0
runtime.v0 -> aurae.runtime.v0
schedule.v0 -> aurae.schedule.v0