New commands for Dockerfile

Overview

Dockerfile+

This project provides Dockerfile syntax extensions that have been rejected by the moby project or haven't been addressed in a long time.

Currently, the project adds an INCLUDE+ Dockerfile directive that allows you to import the content of another file into your Dockerfile. There are plans to add more features in the near future.

Getting started

  • First, you need to make sure you are running a compatible version of Docker: you need at least Docker 18.09. If you are using an older version, you are out of luck. Sorry!
  • Export the following environment variables to enable Buildkit: DOCKER_BUILDKIT=1 and COMPOSE_DOCKER_CLI_BUILD=1 (this might not be necessary for newer installations of Docker)

Once your Docker is set, you just need to add the following line as your first line in your Dockerfile:

# syntax = edrevo/dockerfile-plus

That's it!

Features

INCLUDE+

Right now there is just one extra instruction: INCLUDE+. All Dockerfile+ commands will end up with a + sign to avoid any potential future collisions with Dockerfile commands.

INCLUDE+ will import the verbatim contents of another file into your Dockerfile. Here's an example Dockerfile which uses the INCLUDE+ instruction:

# syntax = edrevo/dockerfile-plus

FROM alpine

INCLUDE+ Dockerfile.common

ENTRYPOINT [ "mybin" ]

If Dockerfile.common contained a single line that said RUN echo "Hello World", then the resulting Docker image would be identical to the one generated by this Dockerfile:

FROM alpine

RUN echo "Hello World"

ENTRYPOINT [ "mybin" ]

Roadmap

The next features in line would be:

  • ENVFILE+ command, which would read a .env file and import all of those environment variable definitions into the Dockerfile
  • RUN+ --no-cache, which would disable the cache only for a specific RUN step (useful for non-idempotent commands, for example those that clone git repos)
  • TAG command
  • improvements to .dockerignore, like recursive dockerignore files

Feedback

Found a bug? Want to contribute a PR? Want to improve documentation or add a cool logo for the project? All contributions are welcome!

Development environment

Install cargo (you can use rustup.rs) and run:

$ cargo build

Creating a local release of the Buildkit frontend

$ docker build -f dockerfile-plus/Dockerfile .
Comments
  • Getting Started documentation is lacking

    Getting Started documentation is lacking

    I'm not sure what I need to do to get started. Do I have to clone the repo somewhere? Do I have to "install" dockerfile-plus so docker will recognize the 'syntax' command? Please elaborate on how to get dockerfile-plus properly build/configured so the rest of the Getting Started example works.

    Thanks.

    opened by joehays 11
  • How to use relative or absolute paths with INCLUDE+

    How to use relative or absolute paths with INCLUDE+

    I get the following error when the 'Dockerfile.common' is not in the same directory.

    failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Code(128) desc = Could not read file " ../../../mbKinDyn-algos/docker.requirements". Remember that the file path is relative to the build context, not the Dockerfile path.

    I also tried using an absolute path and got the same error.

    The third test was to make the CWD a parent of both the Dockerfile folder and the 'docker.requirements' folder and then I used a relative path to both the Dockerfile and 'docker.requirements' from the parent folder. This produced the same error.

    I have multiple git repositories that are related to the Docker container that I'm building. Each repository has a 'docker.requirements' file that documents the dependencies that it needs backed into the docker image/container. So, I need the ability to keep these INSERT+ files in locations other than the main Dockerfile.

    Thoughts?

    opened by joehays 2
  • Does INCLUDE+ work with docker-compose?

    Does INCLUDE+ work with docker-compose?

    docker build works great

    docker-compose up dies with the error

    dockerfile parse error line 2: unknown instruction: INCLUDE+

    My a.Dockerfile looks like:

    # syntax = edrevo/dockerfile-plus
    INCLUDE+ /Dockerfile
    
    opened by terrisgit 1
  • Fails in GitHub Actions

    Fails in GitHub Actions

    I get this error:

    failed to solve with frontend dockerfile.v0: failed to create LLB definition: dockerfile parse error line 3: unknown instruction: INCLUDE+
    

    Here is the version of Docker:

    Client:
     Version:           20.[10]
     API version:       1.41
     Go version:        go1.16
     OS/Arch:           linux/amd64
     Context:           default
     Experimental:      true
    
    Server:
     Engine:
      Version:          20.10.11+azure-3
      API version:      1.41 (minimum version 1.12)
      Go version:       go1.16.12
      OS/Arch:          linux/amd64
      Experimental:     false
     containerd:
      Version:          1.4.12+azure-2
     runc:
      Version:          1.0.3
    

    I have set the environment variables:

        env:
          DOCKER_BUILDKIT: 1
          COMPOSE_DOCKER_CLI_BUILD: 1
    
    opened by aminya 1
  • Error when building on arm64 device

    Error when building on arm64 device

    I was able to build this repo in raspberry pi (arm64v8) with minor changes in the Dockerfile. However when I used the built output in another dockerfile (use INCLUDE+) it would fail with “failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: exit code: 1”. Do you have any idea why it would fail?

    The changes I made in this repo:

    1. Use arm64v8/rust as the base image
    2. Change the target to aarch64-unknown-Linux-gnu
    opened by clsx524 1
  • Multiple levels of include

    Multiple levels of include

    I really like the INCLUDE+ instruction, since it allows to avoid code duplication. The only problem I encountered was using multiple levels of INCLUDE+ (i.e. including a file using INCLUDE+ as well).

    Not sure if i am doing something wrong, there is a limitation or it just is not implemented. I think it is just not considered in the implementation (i.e. missing recursive call on the included file) but i could be wrong, since i do not know rust.

    My files look like this:

    Dockerfile.A:

    # syntax = edrevo/dockerfile-plus
    FROM alpine
    RUN echo ">A"
    INCLUDE+ Dockerfile.A2
    RUN echo "<A"
    

    Dockerfile.A2:

    # syntax = edrevo/dockerfile-plus
    RUN echo ">A2"
    INCLUDE+ Dockerfile.A3
    RUN echo "<A2"
    

    Dockerfile.A3:

    # syntax = edrevo/dockerfile-plus
    RUN echo ">A3"
    RUN echo "<A3"
    

    Here you can find the files: https://gitlab.com/barcode/test/-/tree/ci/multi_level_include

    The build output is here: https://gitlab.com/barcode/test/-/jobs/1214501093

    So is this a problem with my files or is it something else?

    opened by barcode 0
  • Instructions on how to debug and see logs from frontend container

    Instructions on how to debug and see logs from frontend container

    When we do

    # syntax = edrevo/dockerfile-plus
    

    in a dockerfile, a docker container is launched from edrevo/dockerfile-plus image which contains the grpc server.

    Is there any way to get logs from this frontend container for the purpose of debugging the build? This frontend container is short lived, so is there any way on getting the logs from this container?

    I can see that some logs are being written to stderr in here https://github.com/edrevo/dockerfile-plus/blob/master/dockerfile-plus/src/dockerfile_frontend.rs#L111

    Is there any way of seeing these logs? Even just being able to view stderr from the frontend container would suffice.

    opened by aditya211935 1
  • Doesn't work with DOCKER_BUILDKIT=1 and heredocs

    Doesn't work with DOCKER_BUILDKIT=1 and heredocs

    I'm using INCLUDE+ to include a Dockerfile that includes heredocs, which means that I'm setting DOCKER_BUILDKIT=1 before invoking docker build. I'm getting the same error as when I run the included Dockerfile without setting DOCKER_BUILDKIT: namely, docker build is choking on the contents of the heredoc because it's trying to parse them as if they're part of the Dockerfile.

    My best guess as to what's going on is that heredocs are processed at some early stage of Dockerfile parsing, and the INCLUDE+ directive is getting parsed later -- so when it interpolates the contents of the included file, they're not being parsed correctly. Not being expert in ruby OR Docker, that's about as far as I can get in tracking down the problem.

    opened by budney 0
  • doesn't work for windows

    doesn't work for windows

    hello

    when trying to use INCLUDE+ syntax I am getting the error below

    PS C:\work\bitbucket\devops\Docker> docker build -f ./qa-windows/Dockerfile -t win_diab . Sending build context to Docker daemon 58.37kB Error response from daemon: dockerfile parse error line 66: unknown instruction: INCLUDE+

    OS: windows Docker version : 20.10.17, build 100c701

    opened by eyla4ever 0
  • Does not seem to work on ARM32

    Does not seem to work on ARM32

    This was tested on a

    OS: Raspbian GNU/Linux 11 (bullseye) armv7l
    Host: Raspberry Pi 4 Model B Rev 1.2
    Kernel: 5.15.32-v7l+
    Uptime: 1 hour, 49 mins
    Packages: 651 (dpkg)
    Shell: bash 5.1.4
    Terminal: /dev/pts/0
    CPU: BCM2711 (4) @ 1.500GHz
    Memory: 127MiB / 3838MiB
    

    with Docker 20.10.14

    which causes this:

    $ DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 docker build -t ghcr.io/tbrodbeck/qoffee-maker-arm:0.1.1 -f DockerfileArm . && docker run --name qoffeeMaker --rm -itp 8887:8887 --env-file .env ghcr.io/tbrodbeck/qoffee-maker-arm:0.1.1
    [+] Building 2.8s (3/3) FINISHED
     => [internal] load build definition from DockerfileArm                                                                                                                                                    0.2s
     => => transferring dockerfile: 575B                                                                                                                                                                       0.0s
     => [internal] load .dockerignore                                                                                                                                                                          0.1s
     => => transferring context: 90B                                                                                                                                                                           0.0s
     => ERROR resolve image config for docker.io/edrevo/dockerfile-plus:latest                                                                                                                                 1.5s
    ------
     > resolve image config for docker.io/edrevo/dockerfile-plus:latest:
    ------
    failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: no match for platform in manifest sha256:d234bd015db8acef1e628e012ea8815f6bf5ece61c7bf87d741c466919dd4e66: not found
    
    opened by tbrodbeck 0
  • Does cause visual Problems in VS code

    Does cause visual Problems in VS code

    This extension is awesome! But it causes some ugliness in VS code like: Screen Shot 2022-05-05 at 16 25 20

    Screen Shot 2022-05-05 at 16 25 34

    Since VS code is one of the most used editors - does anyone know a solution or workaround to this?

    opened by tbrodbeck 0
  • Does not work over SSH

    Does not work over SSH

    I'm building on an embedded system, using Visual Studio Code's remote build facility which works over SSH. When I try to use dockerfile-plus I get the following:

    => [internal] load build definition from Dockerfile                                                                                                                                                                                          0.0s
     => => transferring dockerfile: 1.19kB                                                                                                                                                                                                        0.0s
     => [internal] load .dockerignore                                                                                                                                                                                                             0.1s
     => => transferring context: 34B                                                                                                                                                                                                              0.0s
     => ERROR resolve image config for docker.io/edrevo/dockerfile-plus:latest                                                                                                                                                                    0.4s
    ------
     > resolve image config for docker.io/edrevo/dockerfile-plus:latest:
    ------
    failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = error getting credentials - err: exit status 1, out: `Cannot autolaunch D-Bus without X11 $DISPLAY`
    

    I presume this is because it is trying to use the new fancy output format which won't work over SSH, is that correct?

    opened by rnwhitehead 0
Releases(0.1.0)
Owner
Ximo Guanter
Ximo Guanter
Workflows make it easy to browse, search, execute and share commands (or a series of commands)--without needing to leave your terminal.

Workflows The repo for all public Workflows that appear within Warp and within commands.dev. To learn how to create local or repository workflows, see

Warp 369 Jan 2, 2023
Workflows make it easy to browse, search, execute and share commands (or a series of commands)--without needing to leave your terminal.

Workflows The repo for all public Workflows that appear within Warp and within commands.dev. To learn how to create local or repository workflows, see

Warp 227 Jun 1, 2022
Dash is a CLI tool that rapidly sets up new projects by running a series of pre-defined commands.

Dash Dash is a CLI tool that rapidly sets up new projects by running a series of pre-defined commands. Features Quick Initialization: Initialize the c

Kunal Bagaria 4 Nov 7, 2023
CLI utility to move (or rename) your files to a new location and redirect all of its symbolic links, to the new path

Move Links CLI utility to move (or rename) your files to a new location and redirect all of its symbolic links, to the new path (or name). Usage execu

Ben Mefteh 18 May 22, 2022
Shell Of A New Machine: Quickly configure new environments

Shell Of A New Machine soanm is a dead-simple tool for easily configuring new UNIX machines, with almost zero prerequisites on the target machine. All

Ben Weinstein-Raun 41 Dec 22, 2022
:stars: Manage your shell commands.

Pueue Pueue is a command-line task management tool for sequential and parallel execution of long-running tasks. Simply put, it's a tool that processes

Arne Beer 3.3k Jan 8, 2023
Executes commands in response to file modifications

watchexec Software development often involves running the same commands over and over. Boring! watchexec is a simple, standalone tool that watches a p

null 3.6k Jan 1, 2023
Remember me my commands

mme is a search tool that helps you to print commands and their description using a custom file. Installation The binary name for rememberme is mme. A

Roberto 27 Dec 23, 2022
Turn static CLI commands into TUIs with ease

lazycli Turn static CLI commands into TUIs with ease Demo: Usage Pick a command that spits out either a list or table of content, like ls, docker ps,

Jesse Duffield 255 Dec 20, 2022
🐢 Atuin replaces your existing shell history with a SQLite database, and records additional context for your commands

Atuin replaces your existing shell history with a SQLite database, and records additional context for your commands. Additionally, it provides optional and fully encrypted synchronisation of your history between machines, via an Atuin server.

Ellie Huxtable 4.6k Jan 1, 2023
🤖 just is a handy way to save and run project-specific commands.

just just is a handy way to save and run project-specific commands. (非官方中文文档,这里,快看过来!) Commands, called recipes, are stored in a file called justfile

Casey Rodarmor 8.2k Jan 5, 2023
1 library and 2 binary crates to run SSH/SCP commands on a "mass" of hosts in parallel

massh 1 library and 2 binary crates to run SSH/SCP commands on a "mass" of hosts in parallel. The binary crates are CLI and GUI "frontends" for the li

null 2 Oct 16, 2022
🌌⭐cosmo is a wrapper for Git essentially, allowing you to compress multiple commands into one

❯ Cosmo Git tooling of the future New feature: Cosmo hooks! Click here for more info! ❯ ?? Features Config files (with defaults!) Fast Easy to use Fri

Jack 1 Oct 31, 2021
Tiny Commands Toolchain

Tiny Commands Toolchain: TCT A 1.62MB (release profile) monolithic commands kit for casual terminal usage without any run-time dependency. MMade in 17

Defmc 9 Dec 1, 2022
Save cli commands and fuzzy find them later

crow - cli command memorizer What is crow? | Installation | Usage | FAQ What is crow? crow (command row) is a CLI tool to help you memorize CLI comman

sandstorm 7 Feb 17, 2022
koyo is a cli tool that lets you run commands as another user. It is similar to doas or sudo.

koyo is a cli tool that lets you run commands as another user. It is similar to doas or sudo.

null 3 Nov 27, 2021
allows you to run multiple cargo commands in a row

Cargo Do Allows you to put multiple cargo commands on one line, e.g. $ cargo do clean, update, build Installation cargo install cargo-do From Source

Paul Woolcock 20 Sep 11, 2022
Lazygit - simple terminal UI for git commands

A simple terminal UI for git commands, written in Go with the gocui library. Rant time: You've heard it before, git is powerful, but what good is that

Jesse Duffield 31.8k Jan 3, 2023
Tldr - 📚 Collaborative cheatsheets for console commands

What is tldr-pages? The tldr-pages project is a collection of community-maintained help pages for command-line tools, that aims to be a simpler, more

tldr pages 42.4k Dec 29, 2022
OOLANG - an esoteric stack-based programming language where all instructions/commands are differnet unicode O characters

OOLANG is an esoteric stack-based programming language where all instructions/commands are differnet unicode O characters

RNM Enterprises 2 Mar 20, 2022