A few known issues:

• [X] ~Python 3.8 doesn't know about the type syntax~ -- Cubicle was since rewritten in Rust
• [X] ~du is different~ -- no longer used on host
• [X] ~tar is different~ -- no longer used on host
• [X] ~Docker mounts are different. The $HOME path on the host and on the container are different.~ -- #9 adds Docker volume support and fixes $HOME path assumptions.
• [ ] X11 surely won't just work.
• [ ] Some hard-coding of amd64/x86-64.

There are probably more issues after that. And then there's a big performance question.

cc @darrengarvey

The window has crashed (reason: 'crashed', code: '133')

I'm not sure, but this might have started when I switched from running it in Bubblewrap to using Docker.

This seems to be reproducible when I take certain actions, like previewing README.md in this repo.

e.g.

[depends.debian]
"libstdc++-arm-none-eabi-newlib" = {}

gives

simon@Starbuck cubicle % cub package update qmk
Error: error scanning built-in packages in "/Users/simon/Github/cubicle/packages"

Caused by:
    0: could not read manifest for package "qmk": "/Users/simon/Github/cubicle/packages/qmk/package.toml"
    1: data did not match any variant of untagged enum DependencyOrTable for key `depends.debian` at line 1 column 1

same for

[depends.debian]
libusb-1.0-0={}

this running #38 if that makes a difference.

This is using #38 on arm/osx I cub enter bob3 do some things, then use exit to leave bob3. At which point i get an error

simon@bob3:~/w$ exit
Error: failed to run command in environment "bob3"

Caused by:
    Non-zero exit status (1) from docker exec

Stack backtrace:
   0: anyhow::backtrace::capture::Backtrace::capture
   1: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
   2: cubicle::docker::Docker::run_
   3: <cubicle::runner::CheckedRunner as cubicle::runner::Runner>::run
   4: cub::cli::run
   5: cub::main
   6: std::sys_common::backtrace::__rust_begin_short_backtrace
   7: std::rt::lang_start::{{closure}}
   8: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:280:13
      std::panicking::try::do_call
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/rt.rs:128:48
      std::panicking::try::do_call
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/rt.rs:128:20
   9: _main

I ran into some trouble trying to run Cubicle Docker environments as root:

11: Pulling from library/debian
10cff8997b4d: Already exists 
Digest: sha256:3e82b1af33607aebaeb3641b75d6e80fd28d36e17993ef13708e9493e30e8ff9
Status: Downloaded newer image for debian:11
 ---> 59ed139d178b
Step 2/8 : RUN echo Etc/UTC > /etc/timezone &&     ln -fs '/usr/share/zoneinfo/'Etc/UTC /etc/localtime
 ---> Running in b3346793084e
Removing intermediate container b3346793084e
 ---> e980d48c6ccd
Step 3/8 : RUN addgroup --gid 0 root || addgroup root &&     adduser --disabled-password --gecos '' --uid 0 --ingroup root root &&     adduser root sudo &&     mkdir /home/root/w &&    
 chown root:root /home/root/w
 ---> Running in 490bcc5504a4
addgroup: The group `root' already exists.
addgroup: The group `root' already exists.
The command '/bin/sh -c addgroup --gid 0 root || addgroup root &&     adduser --disabled-password --gecos '' --uid 0 --ingroup root root &&     adduser root sudo &&     mkdir /home/root
/w &&     chown root:root /home/root/w' returned a non-zero code: 1
Error: failed to update package: "configs-core"

Caused by:
    0: error building package "configs-core"
    1: failed to create environment "package-configs-core"
    2: failed to build "cub-cubicle-base" Docker image
    3: `docker build` exited with exit status: 1
root@debian:~/opt/cubicle# 

Seems like this should probably just pick another username, like "cubicle".

From https://doc.rust-lang.org/std/env/fn.current_exe.html#platform-specific-behavior:

If the executable was invoked through a symbolic link, some platforms will return the path of the symbolic link and other platforms will return the path of the symbolic link’s target.

Better make this more obvious, more resilient, and configurable.

cc @darrengarvey

Fix Issue #44: "docker fails with overly long hostnames"

I don't know if setting the hostname was useful or important anywhere other than the shell prompt. If so, it's easy to revert this.

• The core configs are now more extensible.
• asdf now manages Node and Go installs, and it's likely to be useful for other packages later.
• Hopefully these now work on aarch64/arm64 (#36).

Due to some renames and rewrites, users might need to clear out package environments and caches upon upgrade:

cub purge 'package-*'
rm ~/.cache/cubicle/packages/*.tar

Before this would write the seed tarball out, then read it. This is a little nicer because it's (likely) faster, and it gets rid of some shell script.

It'd be nice to have a runner that runs on virtual machines (for when you're doing something particularly icky or something that itself needs Docker). libvirt and crosvm are two relevant projects to consider.

New users should be able to download a binary, make it executable, and run it. Without reading any other setup instructions, something useful and reasonable should happen.

• [ ] Distribute static binaries (for {mac,linux}-{amd64,aarch64})
• [ ] Manage package definitions at runtime
• [ ] Set up a default configuration or have a configuration "wizard" when no config found

cc @darrengarvey

Having recently worked with qmk for keyboard firmware, I thought I'd take a stab at writing a package for it. You can see the attempt here https://github.com/superfell/cubicle/commit/2e0adeba98fe91ed705f617b8670cf33102c9e2a

qmk has a lot of dependencies. There are some base dependencies (git, python3) that are needed to get the qmk cli tool, and then that tool is used to set everything up including a long list of deb packages and more python packages.

when working with qmk, you work directly in the qmk_firmware tree, its not a separate library. qmk setup will clone the repo to the relevant location. I Used ~/w/qmk_firmware. but this seems wrong. But if its somewhere else and added to the provides, then that also seems wrong.

perhaps qmk isn't a tool and so isn't a good fit for cubicle? Or perhaps there some secret way of using qmk without your work being in the qmk_firmware tree.

The package definitions currently have a lot of shell scripts, which is bad for my sanity, as well as correctness and portability (across distros or versions of tools, or even operating systems). They're likely to get more complicated, too, thanks to #30, #31, #34, and #36.

It should be possible to use build_depends and write many of these scripts in a more suitable language. Rust is probably my top pick. It'd have challenges of convenience and compilation times, but I think we can find ways to overcome those.

using latest, attempting to build the go package target/debug/cubicle package update go results in

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Copying/extracting seed tarball
15.5KiB 0:00:00 [4.12MiB/s] [================================>] 100%            
Checking latest version of Go
./build.sh: 13: go: not found
./build.sh: 14: go: not found
Have  installed at 
Downloading go1.19.1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    75  100    75    0     0    681      0 --:--:-- --:--:-- --:--:--   681
100  141M  100  141M    0     0  80.6M      0  0:00:01  0:00:01 --:--:-- 92.1M
Unpacking go1.19.1
 141MiB 0:00:02 [54.8MiB/s] [============================================================================================================================================================================================================================================>] 100%            
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
ERROR: Have  installed at , which is still not right
Error: failed to update package: "go"

Caused by:
    0: error building package "go"
    1: failed to run command in environment "package-go"
    2: Non-zero exit status (1) from docker exec

Stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.66/src/backtrace/libunwind.rs:93:5
      backtrace::backtrace::trace_unsynchronized
             at /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.66/src/backtrace/mod.rs:66:5
   1: backtrace::backtrace::trace
             at /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.66/src/backtrace/mod.rs:53:14
   2: anyhow::backtrace::capture::Backtrace::create
             at /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/anyhow-1.0.61/src/backtrace.rs:216:13
   3: anyhow::backtrace::capture::Backtrace::capture
             at /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/anyhow-1.0.61/src/backtrace.rs:204:17
   4: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/anyhow-1.0.61/src/error.rs:533:25
   5: <T as core::convert::Into<U>>::into
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/convert/mod.rs:550:9
   6: anyhow::kind::Trait::new

poking around in build.sh and in the created docker image, it looks like the build.sh assumes ~/bin is in the path, but it doesn't appear to be. At least from docker exec -i cub-package-go sh from that shell, it lists the path as PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

The PATH issue may be a complete red herring, and the real issue is whatever causes the qemu logging. I'll continue to poke around.

Package builds currently print so much output that it's hard to tell what's going on. It'd be great to have a way to monitor progress without being overwhelmed by it.

(We might also want to think about parallel package builds and how to display that.)