I'm going crazy. Getting this error on a VM that was working fine. I recently hard reset the VM due to a memory issue. A vagga container was running at the time of the hard reset. Now when trying to run a container I get:

ERROR:main: Error executing _build: Can't mount pseudofs /home/andy/git/nginx-scripts/.vagga/.mnt/proc (, options: proc): permission denied

I think this might be a general linux container - and not a vagga issue. Just hoping you might have some thoughts / seen this before Paul..

andy@hop:~/git/nginx-scripts$ uname -a
Linux hop 3.12-1-amd64 #1 SMP Debian 3.12.6-2 (2013-12-29) x86_64 GNU/Linux

andy@hop:~/git/nginx-scripts$ cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux jessie/sid"
NAME="Debian GNU/Linux"
ID=debian
ANSI_COLOR="1;31"
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support/"
BUG_REPORT_URL="https://bugs.debian.org/"

andy@hop:~/git/nginx-scripts$ sysctl kernel.unprivileged_userns_clone
kernel.unprivileged_userns_clone = 1

andy@hop:~/git/nginx-scripts$ grep andy /etc/sub[ug]id
/etc/subgid:andy:100000:65536
/etc/subuid:andy:100000:65536

Hi there,

I've compiled vagga from the current master source. I get this error when trying to build a container:

ERROR:container::monitor: Can't run container wrapper: Error executing: permission denied

I have this in my vagga.yaml

containers:
  ubuntu:
    setup:
    - !Ubuntu trusty
    - !Install [python]

And I'm trying to build with:

andy@hop:~/tmp/vagga$ vagga _build ubuntu

My local user is andy. I have:

andy:100000:65536

In my /etc/sub[gu]id

Any advice for how to proceed? Thank you!

Hi, I was trying to make a container for launching scala notebooks using ipython/jupyter.

Everything goes well adding scala dependencies and ipython itself. But as soon as I try to make a child container to run iscala, vagga crashes. The vagga file looks like

containers:
  jvm:
    setup:
    - !UbuntuRelease { version: 15.04 }
    - !UbuntuUniverse
    - !Install [openjdk-8-jre-headless, ca-certificates]
  sbt:
    setup:
    - !Container jvm
    - !Install [curl]
    - !EnsureDir /work/.home/sbt
    - !Download
      url: https://raw.githubusercontent.com/paulp/sbt-extras/master/sbt
      path: /work/.home/sbt/sbt
      mode: 0o744
    - !Sh "sed -i '/^root/{s@/root@/work/.home@;}' /etc/passwd"
    volumes:
      /tmp: !Tmpfs
      size: 100Mi
      mode: 0o1777
    environ:
      PATH: "/work/.home/sbt:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
      HOME: /work/.home
  jupyter:
    setup:
    - !Container sbt
    - !PipConfig
      dependencies: true
    - !Py3Install ["ipython[notebook]==2.4.1"]
  iscala:
    setup:
    - !Container jupyter
commands:
  scala: !Command
    container: sbt
    run: sbt -sbt-create console
    description: Run a scala console
  sbt: !Command
    container: sbt
    run: [sbt]
    description: Run sbt
  iscala: !Command
    container: iscala
    run: bash
    description: Run a scala notebook
  sh: !Command
    container: sbt
    run: bash
    description: Enter a shell with sbt available

Even just adding the iscala container depending on jupyter is enough to trigger the crash.

To reproduce the crash, run vagga iscala on latest `vagga-testing'. The error I get is

papillon@marvin:~/esperimenti/vagga/scala$ vagga iscala
ERROR:vagga::wrapper: Error executing _build: Builder killed by signal SIGSEGV[11] (core dumped)
Command <Command "/proc/self/exe" ("vagga_wrapper") "_build" "iscala"; environ: {"_VAGGA_HOME"="/home/papillon","TERM"="xterm","RUST_LOG"="warn",}; uid_map=[UidMap { inside_uid: 0, outside_uid: 1000, count: 1 }, UidMap { inside_uid: 100000, outside_uid: 100000, count: 65536 }]; gid_map=[GidMap { inside_gid: 0, outside_gid: 1000, count: 1 }, GidMap { inside_gid: 100000, outside_gid: 100000, count: 65536 }]> exited with code 124

What does it mean?

src/builder/commands/pip.rs:331:25: 332:74 warning: unused result which must be used, #[warn(unused_must_use)] on by default
src/builder/commands/pip.rs:331                         version_req(hash,
src/builder/commands/pip.rs:332                             fname.parent().unwrap().join(req).as_path());

When using Vagga to work on a PHP project using Composer, how is one expected to manage dependencies?

When not using Vagga, the composer require command can be used to download dependencies and update the composer.json and composer.lock accordingly. The user can also edit the composer.json file directly and then run the composer update command.

When using Vagga, composer is only temporarily installed during the container build. And when building the container with a !ComposerDependencies, the composer install command is used to download dependencies. If the project folder contains a composer.lock file (as it should once put under version control), change to the composer.json will be ignored during container build. And the user as now way to have additional dependencies downloaded (or existing one updated).

Is the composer support not intended to be used the manage the project's dependencies, but only those of applications? This is not what the Laravel tutorial suggest.

One option is to create a copy of the composer binary during container build and expose it as a command. But since downloaded dependencies are stored in the immutable container (in /usr/local/lib/composer/vendor), it can not be used to update them.

Another option would be to not use Vagga composer support but install and then use composer on my own. Sadly, it means it can't re-use Vagga cache directory as they are not exposed to commands.

A third option would be to delete the composer.lock file when dependencies are changed, but this seems counter intuitive to composer users.

What would be the recommended way a manage my project dependencies with Composer?

I'm running a freshly installed Xubuntu 15.10 Linux 4.2.0-27-generic #32-Ubuntu SMP Fri Jan 22 04:49:08 UTC 2016 x86_64 GNU/Linux

I tried to run the flaky-network example, then when I ran vagga _create_netns it failed with the following output:

net.ipv4.ip_forward = 1
net.ipv4.conf.vagga.route_localnet = 1
modprobe: ERROR: could not insert 'ip_tables': Operation not permitted
iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
ERROR:vagga::setup_netns: Error running command <Command "/sbin/iptables" "-t" "nat" "-A" "POSTROUTING" "-o" "vagga_guest" "-j" "MASQUERADE"; environ: {"TERM"="xterm","PATH"="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games","RUST_LOG"="warn",}>: exited with code 3
vagga_setup_netns exited with code 1

Running sudo iptables -t nat -L shows that no iptables rule was added, so I added the rule manually, then ran vagga run-normal but I was unable to view the page at http://172.18.255.2:8000.

Trying traceroute 172.18.255.2 shows that it tries to route to my local network, which means that the routing was not setup by vagga _create_netns.

Networking between containers seems to work since I tried a django + postgres setup and the two containers were able to talk to each other.

Just checking - is the best way to discuss new ideas? Would a mailing list be better?

So this is very much a feature request...

I find myself wanting to be able to cache !Sh sections.

I.e., something like you have a !Sh section that takes a while to build, but is stable. You then want to build on top of that with additional !Sh sections. If you add multiple !Sh sections, could a cache be taken after each one, and only rebuilt if that !Sh section changes?

image-cache-url uses wget to download the images and doesn't support using images from a local driver or a network mounted drive.

It should probably do something like this:

if [ -d "$(dirname ${the_path})]; then
  rsync -qa ${the_path} ${image-cache-url-path}/
else
  wget ${the_path} -O ${image-cache-url-path}/$(basename ${the_path})
fi

Currently it says:

wget: bad address ''

And starts rebuilding the containers.

When trying to use the mount syscall to mount a filesystem image I get the next error:

Trace: { [Error: EPERM, Operation not permitted '/work/out/latest']
  errno: 1,
  code: 'EPERM',
  path: '/work/out/latest',
  syscall: 'mount' }
    at onerror (/lib/node_modules/nodeos-mount-filesystems/index.js:34:13)
    at /lib/node_modules/nodeos-mount-filesystems/index.js:297:26

Could it be possible I can't be able to use the mount syscall from inside vagga? And if so, what alternatives do I have? I know I can be able to set a volume, but they are only mapping to a host folder and don't allow to mount disk images. I could live with this, but can I be able later to use mount bind and mount move from inside vagga? NodeOS use them to create a clean filesystem hierarchy... :-/

If I'm reading the source correctly, vagga sets up /etc/apk/cache, but it doesn't seem to work. Every time I the container gets rebuilt, all APK packages get downloaded again. Does this need to be manually activated?

I'm using the Vagga 0.5.0 binaries from the download page.

Thanks!

Currently, the only directory shared between the host machine and the container is the directory vagga.yaml resides in (known as /work inside the container). It would be useful to be able to share another directory with the container in the form of volumes. For example, maybe allow a prefix that makes vagga know that the directory is outside the container and in the host machine.

This has a few use cases:

• It could help with making vagga cross-platform: applications that create symlinks in directories (such as postgresql in its data directory) can be installed on vagrant machines running on Windows.
• It could help with people not wanting to litter the /work directory with artifacts such as logs. This would mean that you wouldn't have to have a logs directory in your repository with a .gitignore with something like this inside:

*
!.gitignore

Not sure if this idea is worth it, but putting it here since this is blocking me from installing postgresql in a vagga container on an Ubuntu virtual machine created by vagrant.

vagga --mount-bind $HOME/.cargo:/work/target/.cargo run

At the moment we can emulate such a behavior:

vagga.yaml:

commands:
  run-ci: !Command
   <<: *run-cmd
    volumes:
      /work/target/.cargo: !BindRW /volumes/cargo

export VAGGA_SETTINGS="
external-volumes:
  cargo: $HOME/.cargo"
vagga run-ci

Think it is too complex to be really useful.

Possible options:

• --mount-bind
• --mount-bind-ro
• --mount-tmpfs
• --mount-snapshot

Either we can have a single --mount option and pass mount type encoded inside an argument.

Implemented:

• Build step options: registry, image, tag, path, insecure
• Short form: !DockerImage python, !DockerImage python:3.10-slim
• Insecure registries
• Registries that don't support authentication
• Cache layers; wait if another process is downloading the layer

Have a PoC of the !DockerImage build step:

containers:
  python:
    setup:
    - !DockerImage
      image: library/python
      version: 3.10-slim

What do you think about it?

TODO:

• [x] Build step options: registry, image, tag, path
• [x] Short form: !DockerImage python, !DockerImage python:3.10-slim
• [ ] Auth with credentials; credentials from vagga settings
• [x] Registries that don't support authentication
• [ ] Detect OS release
• [x] Cache layers; wait if another process is downloading the layer
• [ ] Downloading layers progress
• [ ] Maximum layers cache size
• [ ] Push image to a docker registry
• [ ] Use docker registry as an image cache
• [ ] Image entrypoint
• [ ] Capsule command to run docker images?

Superseeds #502

i have been facing busybox issue every time when i downloaded from the external source. latesti downloaded busybox pro from their and also working fine but after sometime facing issue.

thsi issue from app fault or developer side ?

Hi, I need help with an issue I'm encountering consistently on my project. I couldn't minimize it yet - maybe you can tell what's wrong already. This issue occurs when directly building final container pavetta-sdk-custom, but if we first build pavetta-redist, it goes away. There's also a warning I don't know how to handle:

 WARN 2019-08-19T12:26:09Z: vagga::builder::commands::subcontainer: Can't write image usage info: No such file or directory (os error 2)

➜  seidhe git:(master) ✗ vagga pavetta run Iss/libraries/sysid
<ubuntu setup logs>
Setting up g++ (4:7.4.0-1ubuntu2.3) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
update-alternatives: warning: skip creation of /usr/share/man/man1/c++.1.gz because associated file /usr/share/man/man1/g++.1.gz (of link group c++) doesn't exist
Setting up build-essential (12.4ubuntu1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
 WARN 2019-08-19T12:26:09Z: vagga::builder::commands::subcontainer: Can't write image usage info: No such file or directory (os error 2)
ERROR 2019-08-19T12:26:09Z: vagga::builder: Error building container "pavetta-sdk-custom": step SubConfig { source: Directory, path: "vagga.yaml", container: "pavetta-sdk-common", cache: None } failed: sub-step Build { container: "pavetta-redist", source: "/usr/local/bin", path: Some("/usr/local/bin"), temporary_mount: None, content_hash: false, rules: ["/pavetta"] } failed: can't read "/vagga/base/.roots/pavetta-redist.a9d21258/root/usr/local/bin": No such file or directory (os error 2)
ERROR 2019-08-19T12:26:09Z: vagga::wrapper: Error executing _build: Builder exited with code 1
Command <Command "/proc/self/exe" "__wrapper__" "_build" "pavetta-sdk-custom"; environ[7]; uid_map=[UidMap { inside_uid: 0, outside_uid: 1000, count: 1 }, UidMap { inside_uid: 1, outside_uid: 100000, count: 65535 }]; gid_map=[GidMap { inside_gid: 0, outside_gid: 1000, count: 1 }, GidMap { inside_gid: 1, outside_gid: 100000, count: 65535 }]> exited with code 124

Relevant configs are:

containers:
  pavetta-sdk-common:
    setup:
      - !Ubuntu bionic
      - !UbuntuUniverse
      - !Install [build-essential]
      - !Build
        container: pavetta-redist
        source: /usr/local/bin
        path: /usr/local/bin
        rules:
        - /pavetta
      - !Build
        container: ciri-redist
        source: /usr/local/bin
        path: /usr/local/bin
        rules:
        - /ciri
      - !Build
        container: ciri-lua
        source: /opt/lua
        path: /opt/lua
      - !Install [liblua5.2-0]
      - !Text
        /usr/local/bin/Config.toml: |
          test_suites = [
              "/work/ciri/test_suites/",
          ]

          [ciri]

          path = "/usr/local/bin/ciri"
          config_path = "/usr/local/bin/ciri-Config.toml"
      - !Text
        /usr/local/bin/ciri-Config.toml: |
          lock_file_path = "/tmp/lock"
          lua_prefix_path = "/opt/lua/"
      - !SubConfig
        container: sdk-deps

  pavetta-sdk-custom:
    environ:
      HOME: /work/.vagga-ws
    setup:
      - !SubConfig
        container: pavetta-sdk-common
      - !SubConfig
        container: sdk-deps
      - !Build
        container: sdk-custom
        source: /opt
        path: /opt

  sdk-deps:
    setup:
      - !Install
        - dosfstools
        - doxygen
        - jq
        - libasound2
        - libcurl3-gnutls
        - libglib2.0-0
        - libpixman-1-0
        - libpulse0
        - libsdl2-2.0-0
        - libx11-6
        - libxml2
        - make
        - mtd-utils
        - parted
        - python
        - python3
        - virtualenv

  pavetta-redist:
    environ:
      HOME: /tmp
    setup:
      - !Container rust-build
      - !Depends 
        path: pavetta
        rules:
        - /src
        - /Cargo.toml
      - !Sh |
         cd pavetta
         cargo build --target-dir /work/.vagga-ws/target/pavetta
      - !Sh "cp /work/.vagga-ws/target/pavetta/debug/pavetta /usr/local/bin"

  rust:
    environ:
      HOME: /work/.vagga-ws
    setup:
      - !Ubuntu bionic
      - !UbuntuUniverse
      - !TarInstall
        url: https://static.rust-lang.org/dist/rust-1.35.0-x86_64-unknown-linux-gnu.tar.gz
        script: ./install.sh --prefix=/usr
      - !TarInstall
        url: https://static.rust-lang.org/dist/rust-1.35.0-x86_64-unknown-linux-musl.tar.gz
        script: ./install.sh --prefix=/usr --components=rust-std-x86_64-unknown-linux-musl

  rust-build:
    environ:
      HOME: /work/.vagga-ws
    setup:
      - !Ubuntu bionic
      - !UbuntuUniverse
      - !Build
        container: rust
        source: /usr
        path: /usr
      - !Install [build-essential, curl, ca-certificates]
      - !CacheDirs
        /tmp/.cargo: cargo

  ciri-redist:
    environ:
      HOME: /tmp
    setup:
      - !Container ciri-build
      - !CacheDirs
        /tmp/.cargo: cargo
      - !Depends 
        path: ciri
        rules:
        - /src
        - /Cargo.toml
      - !Sh |
         cd ciri
         cargo build --target-dir /work/.vagga-ws/target/ciri
      - !Sh "cp /work/.vagga-ws/target/ciri/debug/ciri /usr/local/bin/ciri"

  ciri-build:
    environ:
      HOME: /work/.vagga-ws
    setup:
      - !Container rust-build
      - !Install [liblua5.2-dev, liblua5.2-0, pkg-config]

  ciri-lua:
    setup:
      - !Ubuntu bionic
      - !UbuntuUniverse
      - !Install
        # Lua and Lua rocks build dependencies
        - build-essential
        - ca-certificates
        - curl
        - libreadline-dev
        - unzip
        # Lua native rocks extensions
        - liblua5.2-dev
        # Luarocks installation
        - python-pip
        # Luarocks GitHub installation support
        - git
        # luasec native dependency
        - libssl-dev
      - !Sh "pip install hererocks"
      - !EnsureDir /opt
      - !Cmd
        - /bin/bash
        - -exc
        - |
          hererocks -l5.2 -rlatest /opt/lua
      - !CacheDirs
        /tmp/.cache/luarocks: luarocks-cache
      - !Depends
        path: ciri
        rules:
        - /rocks.sh
      - !Cmd
        - /bin/bash
        - -exc
        - |
          . /opt/lua/bin/activate
          /work/ciri/rocks.sh

  sdk-custom:
    setup:
      - !Container base
      - !Sh |
         echo "In case next step fails, look at .vagga/sdk-parser/sdk.yaml"
      - !SubConfig
        source: !Container sdk-spec-parser
        path: sdk.yaml
        container: sdk

  sdk-spec-parser-base:
    setup:
    - !Container empty
    - !Depends prepare-sdk-yaml.py
    - !PipConfig 
      dependencies: true
    - !Py3Install [PyYAML, pystache]

  sdk-spec-parser:
    setup:
    - !Container sdk-spec-parser-base
    - !Depends .vagga-ws/sdk-spec.yaml
    - !Sh |
       /work/prepare-sdk-yaml.py /work/.vagga-ws/sdk-spec.yaml /sdk.yaml

  empty:
    setup:
      - !Alpine v3.10

prepare-sdk-yaml.py:

#!/usr/bin/env python3

import sys
import yaml
import pystache

template_head = """
containers:
  sdk:
    setup:
"""

template_steps_base = """
      - !RunAs
        script: |
          if [ -f "/work/.vagga-ws/{{id}}/{{base_name}}-{{id}}_{{version}}_amd64.deb" ]; then
            mkdir /tmp/{{id}}
            cp /work/.vagga-ws/{{id}}/{{base_name}}-{{id}}_{{version}}_amd64.deb /tmp/{{id}}
          else
            jfrog rt download sdk/{{id}}/*_{{version}}_* | tee /work/.vagga-ws/jfrog-result | jq -e ".totals.success == 1 and .totals.failure == 0" || cat /work/.vagga-ws/jfrog-result
          fi
        work-dir: /tmp
      - !Sh |
          echo debconf {{base_name_lower}}-{{id_lower}}/eula select true | debconf-set-selections
          echo debconf {{base_name_lower}}-{{id_lower}}/eula seen true | debconf-set-selections
          dpkg -i /tmp/{{id}}/{{base_name}}-{{id}}_{{version}}_amd64.deb
"""

def main():
    spec_file_path = sys.argv[1]
    out_file_path = sys.argv[2]

    with open(spec_file_path, 'r') as spec_file:
        try:
            with open(out_file_path, 'w') as out_file:
                spec = yaml.safe_load(spec_file)

                result = pystache.render(template_head, spec)
                out_file.write(result)

                for s in spec.values():
                    s['id_lower'] = s['id'].lower()
                    if not 'base_name' in s:
                        s['base_name'] = "KasperskyOS"

                    s['base_name_lower'] = s['base_name'].lower()
                    
                    template_steps = pystache.render(template_steps_base, s)
                    result = pystache.render(template_steps, s)
                    out_file.write(result)

        except yaml.YAMLError as exc:
            print(exc)


if __name__ == "__main__":
    main()

sdk-spec.template.yaml:

# Copy this file to .vagga-ws/sdk-spec.yaml and change as you like
# Then use container 'sdk-custom'. It will have specified SDK installed

komset:
  id: Komset-001
  version: 0.11
myoffice:
  id: MyOffice-001
  version: 0.39
  # Initial part of deb package name
  # By default, it's KasperskyOS
  base_name: KasperskyOS-SDK