A Yocto setup and management tool that helps you keep your environment up-to-date and in-sync with your team

Related tags

Command-line yb
Overview

yb (Yocto Buddy)

yb is designed to make it easy to setup and (perhaps more importantly) keep Yocto environments up-to-date and in-sync with your team. It is early in development, but we are releasing it now as it is already useful.

Motivation

This tool was heavily inspired by kas, myrepos, and Google's repo tool. We are also familiar with whisk.

All of these tools are great for doing initial environment setup for CI and/or new developers coming onboard. In our estimation, however, that is the easy part. The harder part is ensuring your environment stays up-to-date as your product(s) evolve through development:

  • Layers get added, removed, and updated
  • DISTRO and MACHINE configurations are added
  • Recommended local.conf settings may drift over time: perhaps new SSTATE_MIRRORS or BB_HASHSERVE servers come on-line.

Historically, it has been painful to keep all of this in-sync, usually manifesting as emails sent team-wide everytime bblayers.conf needs to change.

Goals and non-goals

yb strives to be a tool for helping developers with their everyday development tasks. Unlike kas, it does not enforce separation of build environment and host. yb is designed to complement the Yocto workflow you're already used to - for example, there is no yb shell command. You'll run the bitbake command as usual.

Specs and streams: keeping in-sync

Much like kas' configuration files (see https://kas.readthedocs.io/en/latest/userguide.html), yb has specs which are also .yaml files. In fact, the format is nearly the same (though interoperability is not guaranteed - if that's a feature you want please open an issue).

A basic spec looks like this:

Basic spec (click to expand)
header:
  version: 1
  name: "nightly"

repos:
  poky:
    url: "git://git.yoctoproject.org/poky"
    refspec: "honister"
    layers:
      meta:
      meta-poky:

  meta-openembedded:
    url: "git://git.openembedded.org/meta-openembedded"
    refspec: "honister"
    layers:
      meta-networking:
      meta-python:
      meta-filesystems:
      meta-webserver:
      meta-oe:

Specs live in streams. A stream is just a git repo that you've hosted somewhere accessible by your developers.

If you need to add a layer to your build, just do it in the spec and commit the change to the stream. Developers using that stream with yb will automatically have the stream refreshed the next time they run yb status or yb sync (see below).

Installation

The easiest way to install yb is to use the pre-compiled, statically-linked binary available here: https://github.com/Agilent/yb/releases/tag/0.0.11. Simply download and unpack into PATH. It should run on any modern-ish 64-bit Linux system. If you want binaries for other systems (e.g. Windows or 32-bit Linux) please file an issue.

Alternatively, you can build yb yourself. You'll need a nightly Rust compiler. To build and run, use cargo run -- --help (equivalent to doing yb --help).

Basic usage

yb supports two kinds of environments ("envs" for short): vanilla Yocto and yb. You'll know you have a yb env if you see a hidden .yb/ directory inside your yocto/ directory.

Vanilla Yocto

A vanilla Yocto env is one in which you haven't (yet) used yb init to initialize a yb env. In this case, yb operates with reduced functionality but can still be extremely useful. See for example the yb status command below. To use:

  1. In a terminal, activate your Yocto env. This is usually a matter of doing source setupsdk or source oe-init-build-env.
  2. Try running yb status

Converting vanilla Yocto env to yb env

To do the conversion, simply activate your Yocto env as usual and then run yb init:

  1. source setupsdk or source oe-init-build-env
  2. yb init (or yb init -s PATH_TO_STREAM)
  3. Try running yb status

For a demo stream you can use right now, see https://github.com/Agilent/yb-demo-specs.

Creating a new yb env from scratch

You can create a new yb env (and skeleton yocto/ directory) by running yb init outside of any existing environments:

  1. Ensure you are not in the context of an existing yb or vanilla Yocto env. If you are, launch a new terminal and/or cd somewhere else.
  2. yb init (or yb init -s PATH_TO_STREAM)
  3. cd yocto

Note that even if you pass a stream to yb init, no layers are cloned yet. You'll need yb sync for that (see below).

For a demo stream you can use right now, see https://github.com/Agilent/yb-demo-specs.

Commands

yb activate: set the active spec

Vanilla Yocto env yb env
✔️

Use this command to set the active spec. It doesn't actually make any changes to your layers or confs. You'll need yb sync for that (see below).

yb activate nightly

yb status: report env status

Vanilla Yocto env yb env
✔️ ✔️

The yb status command was designed to provide useful status information across all the repos in your Yocto env.

Better yet, it also works in vanilla Yocto envs. As long as you run yb status in a terminal in which you have an activated Yocto env (i.e. which bitbake prints a path), yb will find the path to where your layers live and report their statuses.

yb status with vanilla Yocto env
yb status is run in the context of a vanilla Yocto env.

Use the --skip-unremarkable / -s flag to hide repos for which there is no actionable status information. The result is a more concise summary.

yb status with vanilla Yocto env
yb status is run in the context of a vanilla Yocto env with the --skip-unremarkable / -s flag.

When run in the context of a yb env, however, yb can help even more. If a yb env is found, yb will fetch the current stream to see if any specs were updated. Then it will report how your current env differs from that of the activated spec.

yb status with yb env
yb status is run in the context of a yb env with an activated spec.

yb sync: make my env match the active spec

Vanilla Yocto env yb env
✔️

yb sync with the -a/--apply flag will do what is needed to make your env reflect that of the activated spec. It currently supports these actions:

  • Clone repos
  • Add/remove layers from bblayers.conf (creating it first if necessary)
  • Switch branches
  • Do fast-forward git pull
  • Create local tracking branch
  • Reset working directory (only if given --force flag)

As a precaution, yb sync does nothing but report what would have been done. To actually make changes you need to pass the -a/--apply flag.

When used within a yb env, yb sync will first pull any stream updates.

yb sync and status
yb sync is first run in dry-run only mode (the default) to show what would be done. Then it is run again with --apply/-a flag. Finally, yb status is run to show that the env is up-to-date.

yb run: run a command for each repo

Vanilla Yocto env yb env
✔️ ✔️

This works in either yb or Yocto env. It doesn't matter what directory you run it in as long as yb can find the env.

yb run -n -- git branch --show-current
yb run
yb run using the -no-return-codes/-n flag to display just the current branch of each repo.

Project status

What's working:

  • Everything described above, plus a few other utility commands (e.g. yb list to view specs and streams)

TODO:

  • Reconstitute the auto-update feature deployed internally
  • Support modifications to local.conf in specs
  • Some kind of matrix build support (multiple MACHINE/DISTRO/?)
  • Usage in CI environment
  • Documentation...
  • Tests
  • User-friendly output for yb sync
  • Make updating streams more robust than just a git pull
  • Support some kind of local stream? (for now, just use a local git repo - file:// works as a URI)
  • Make --porcelain flag more than just dumping internal structures as JSON which are liable to change; and make it work for other commands

Possible TODOs (i.e. features I probably won't use but that other tools support - submit issue reports please)

  • Layer patches

Ideas:

  • Maybe menuconfig like kas?
  • Multiconfig support
  • Some kind of yb stash/yb stash pop (like git stash/git stash pop) across all layers/local.conf at once. Would be useful for doing a quick build in between experiementing.

Why not Python?

Basically because this. Rust lets you build statically-linked binaries with no hassle. There is no beating that for distribution. Also, the type system and ecosystem are great.

License

Copyright 2022 Agilent Technologies, Inc.

This software is licensed under the MIT license.

Some portions have been adapted from git2-rs which is dual-licensed MIT and Apache 2.0. We have chosen to use it as MIT licensed.

Disclaimer

This is not an official Agilent product. No support is implied.

You might also like...
Get up-to-date departure times for Munich public transport in your terminal.
Get up-to-date departure times for Munich public transport in your terminal.

MVG Fahrinfo MVG Fahrinfo is a CLI tool to keep up-to-date with latest departure times of Munich public transport. The app is a native binary and uses

This is a `Rust` based package to help with the management of complex medicine (pill) management cycles.
This is a `Rust` based package to help with the management of complex medicine (pill) management cycles.

reepicheep This is a Rust based package to help with the management of complex medicine (pill) management cycles. reepicheep notifies a person(s) via

⚡️(cd with env) Is a configurable cd wrapper that lets you define your environment per directory.
⚡️(cd with env) Is a configurable cd wrapper that lets you define your environment per directory.

⚡️cdwe (cd with env) A simple configurable cd wrapper that provides powerful utilities for customizing your envionment per directory. (For ZSH / BASH

Scriptable tool to read and write UEFI variables from EFI shell. View, save, edit and restore hidden UEFI (BIOS) Setup settings faster than with the OEM menu forms.
Scriptable tool to read and write UEFI variables from EFI shell. View, save, edit and restore hidden UEFI (BIOS) Setup settings faster than with the OEM menu forms.

UEFI Variable Tool (UVT) UEFI Variable Tool (UVT) is a command-line application that runs from the UEFI shell. It can be launched in seconds from any

RustRedOps is a repository dedicated to gathering and sharing advanced techniques and malware for Red Team, with a specific focus on the Rust programming language. (In Construction)

RustRedOps In Construction.... The project is still under development Overview RustRedOps is a repository that houses various tools and projects relat

cpa is a cli tool for ultra fast setup of Rust & Python projects
cpa is a cli tool for ultra fast setup of Rust & Python projects

CPA: Create-Python-App cpa is a cli tool for ultra fast setup of new Python & Rust projects. It automates the creation of config files like style & li

A Turing-complete but dead-simple spaced repetition CLI that helps you learn stuff.

Forne — Learn Stuff Forne is a Turing-complete spaced repetition engine to help you learn stuff your way. What does that mean? Well, there are a few p

This is a command line utility written in Rust that helps you plan factories in Satisfactory.

Satisfactory Factory Planning Utility This is a command line utility written in Rust that helps you plan factories in Satisfactory. Tell it what you w

A project management tool for data science and bioinformatics. If you want it, Kerblam it!
A project management tool for data science and bioinformatics. If you want it, Kerblam it!

Warning kerblam run and kerblam package are complete but still untested. Please do use them, but be careful. Always have a backup of your data and cod

Comments
  • yb `sync`/`status` somehow needs to handle

    yb `sync`/`status` somehow needs to handle "related" repos

    Suppose you have two specs, "a" and "b". Both declare a meta-clang repo. But let's say "a" uses https://github.com/kraj/meta-clang.git but "b" uses https://github.com/Xilinx/meta-clang.git.

    Currently when you try to switch between "a" and "b" (either direction), yb will give an error that "meta-clang" has the same name as a spec repo but not the expected remote. It currently refuses to yb sync in this situation.

    I think the desired behavior is that yb is smart enough to realize that, for example, syncing from "a" => "b" means we want to add a remote tracking branch for https://github.com/Xilinx/meta-clang.git and do a checkout. The challenge is teaching yb how to identify these "related" repositories.

    There are a few approaches I can think of:

    1. Trust that since the "a" on-disk repo is called "meta-clang", it's the same "meta-clang" called for in the "b" spec (and vice-versa). This would impose an annoying requirement that all repos be named uniformly across specs.
    2. Allow declaring repo "aliases". This is a bit like "extra-remotes" (and as a reminder, these are already considered when trying to map on-disk repos to spec repos). For it to work in this situation you'd need to declare aliases in both "a" and "b" specs (in fact, across any spec in which you use meta-clang). Alternatively, the alias could be persisted in some .yb/config file.
    3. Prompt the user for confirmation. Decisions would probably be persisted in a .yb/config file.
    4. Automatically establish that repos are "related" by looking for common commits. See https://stackoverflow.com/a/58287565.

    I am currently working on a hybrid of (4) and (3). yb will attempt to automatically find "related" repos. If it mistakenly identifies two repos as related, you'll be able to override the decision.

    I am still amenable to implementing (1) and/or (2) if people would prefer them. In that case, the choice between method would be user-configurable either on a per-yb env basis or globally.

    bug enhancement 
    opened by chris-laplante 0
  • `yb init` is confusing because it has two overloaded meanings

    `yb init` is confusing because it has two overloaded meanings

    We've had some internal confusion over yb init because it does two different things depending on the context:

    1. In a Yocto environment (i.e. bitbake is a valid command in your terminal), it upgrades your env to a yb env.
    2. Otherwise, it creates a new yb env and a new yocto/ directory.

    It's pretty easy to forget that you have an activated Yocto env. For example, you may cd somewhere else entirely from where your Yocto env lives. In this case, you probably want behavior #2 but what you're going to get is #1.

    I think it is early enough in the lifecycle of yb that it makes sense to make a breaking change to resolve this confusion. What I'm proposing is to add a new command, yb upgrade, which will only do #1. yb init will be changed to always initialize a new yb env (i.e. #2). It will issue a warning (and maybe ask for confirmation) if it detects that you may have wanted to use yb upgrade instead.

    opened by chris-laplante 0
  • Consider implementing layer patches like kas

    Consider implementing layer patches like kas

    Please :heart: or :+1: this issue if it's something you care about so I can prioritize it.

    kas supports applying patches that are declared in the configuration files to repos after cloning them: https://kas.readthedocs.io/en/latest/userguide.html#usage

    This could be useful to implement in yb.

    enhancement 
    opened by chris-laplante 2
Releases(0.0.11)
Owner
null
Deadliner helps you keep track of the time left for your deadline by dynamically updating the wallpaper of your desktop with the time left.

Deadliner Watch the YouTube video What's Deadliner? Deadliner is a cross-platform desktop application for setting deadline for a project and keeping t

Deadliner 34 Dec 16, 2022
gfold is a CLI-driven application that helps you keep track of multiple Git repositories.

gfold is a CLI-driven application that helps you keep track of multiple Git repositories.

Nick Gerace 215 Jan 4, 2023
Quickly setup your development environment on your Chromebook/ChromeOS or any Linux distribution 🐧 ❄️ 💻 🚀 ✨

Crosup ?? ?? ?? ✨ Crosup is a CLI tool to help you quickly setup your development environment on a new Chromebook (ChromeOS) or any Linux distribution

Tsiry Sandratraina 11 Jun 15, 2023
Rust File Management CLI is a command-line tool written in Rust that provides essential file management functionalities. Whether you're working with files or directories, this tool simplifies common file operations with ease.

Rust FileOps Rust File Management CLI is a command-line tool written in Rust that provides essential file management functionalities. Whether you're w

Harikesh Ranjan Sinha 5 May 2, 2024
Warp is a blazingly fast, Rust-based terminal that makes you and your team more productive at running, debugging, and deploying code and infrastructure.

Warp is a blazingly fast, Rust-based terminal that makes you and your team more productive at running, debugging, and deploying code and infrastructure.

Warp 10.4k Jan 4, 2023
Sleek is a CLI tool for formatting SQL. It helps you maintain a consistent style across your SQL code, enhancing readability and productivity.

Sleek: SQL Formatter ✨ Sleek is a CLI tool for formatting SQL. It helps you maintain a consistent style across your SQL code, enhancing readability an

Nick Rempel 40 Apr 20, 2023
rpm (Rust project manager) is a tool that helps you to manage your rust projects

rpm rpm (Rust project manager) is a open source tool for managing your rust project in an organized way Installation # make sure you have rust install

Dilshad 4 May 4, 2023
Get your loadshedding schedule in your calendar and never be left in the dark! Open-source, up-to-date, and developer friendly.

Loadshedding schedules in your digital calendar. No apps, no ads, up-to-date, and developer friendly. Get it • Key Features • Using the data • Project

Boyd Kane 117 Apr 26, 2023
A Rust CLI tool that helps you enforce Git policies through Git hooks both server and client side

GitPolicyEnforcer This is a command line utility written in Rust, that helps you utilize Git hooks, to enforce various policies. It currently supports

Vagelis Prokopiou 4 Aug 14, 2022
🔣 nerdfix helps you to find/fix obsolete Nerd Font icons in your project.

?? nerdfix nerdfix helps you to find/fix obsolete Nerd Font icons in your project. ?? Why Nerd Fonts is used in many projects for a beautiful UI. It p

Loi Chyan 80 Apr 8, 2023