GRM — Git Repository Manager

Overview

GRM — Git Repository Manager

GRM helps you manage git repositories in a declarative way. Configure your repositories in a TOML file, GRM does the rest.

Take a look at the official documentation for installation & quickstart.

Why?

I have a lot of repositories on my machines. My own stuff, forks, quick clones of other's repositories, projects that never went anywhere ... In short, I lost overview.

To sync these repositories between machines, I've been using Nextcloud. The thing is, Nextcloud is not too happy about too many small files that change all the time, like the files inside .git. Git also assumes that those files are updated as atomically as possible. Nextcloud cannot guarantee that, so when I do a git status during a sync, something blows up. And resolving these conflicts is just no fun ...

In the end, I think that git repos just don't belong into something like Nextcloud. Git is already managing the content & versions, so there is no point in having another tool do the same. But of course, setting up all those repositories from scratch on a new machine is too much hassle. What if there was a way to clone all those repos in a single command?

Also, I once transferred the domain of my personal git server. I updated a few remotes manually, but I still stumble upon old, stale remotes in projects that I haven't touched in a while. What if there was a way to update all those remotes in once place?

This is how GRM came to be. I'm a fan of infrastructure-as-code, and GRM is a bit like Terraform for your local git repositories. Write a config, run the tool, and your repos are ready. The only thing that is tracked by git it the list of repositories itself.

Future & Ideas

  • Operations over all repos (e.g. pull)
  • Show status of managed repositories (dirty, compare to remotes, ...)

Optional Features

  • Support multiple file formats (YAML, JSON).
  • Add systemd timer unit to run regular syncs

Crates

  • toml for the configuration file
  • serde because we're using Rust, after all
  • git2, a safe wrapper around libgit2, for all git operations
  • clap, console and shellexpand for good UX

Links

Comments
  • Made a single error message not stall the repo finding process

    Made a single error message not stall the repo finding process

    Love the idea of this project. But on my machine it crashed on a single repo, so I refactored it so that it continues processing after encountering a erroneous repo. With this changes it works perfectly on my machine, it shows the error and picks up all other repos. It also picks up the repo that gave rise to the error in some way which I do not really understand but will look into.

    The git changes look a bit all over the place, but essentially I moved the code after match open_repo into the match statement in the function find_repos.

    I see you do not have any contributing guidelines or license in the repo but I assume you appreciate PRs and I allow my changes to be integrated in the code and licensed under any open source license.

    opened by douweschulte 5
  • handling nested repositories

    handling nested repositories

    So I have some projects that simply hold git repositories inside other git repositories (usually in .gitignore folders). And I want to keep that directory structure. Is this possible without introducing submodules?

    opened by pythys 1
  • catch the error returned by add_repo_status

    catch the error returned by add_repo_status

    Hi, thanks for the tool ! I'm playing around with it to see if I can use it and I encountered a small issue:

    If add_repo_status returns an error it will eventually be printed to the user but there won't be any repository information:

    [✘] Error getting status: No branch checked out
    

    This patch catches the error earlier so we can print the repository name:

    [✘] Error: freebsd-src: Couldn't add repo status: No branch checked out
    

    (ideally repos status wouldn't fail if there's no branch checked out but that's another story).

    opened by vrischmann 1
  • Handle worktrees in subdirectories properly

    Handle worktrees in subdirectories properly

    Currently, the following behaviour is quite weird:

    $ grm wt add x/branch
    

    This creates a branch called branch (unexpected) inside the directory x/branch (expected). It's also weird, because the following will fail:

    $ grm wt add x/branch
    $ grm wt delete x/branch
    [✘] Branch "branch" is checked out in worktree, this does not look correct
    

    The docs say:

    The branch inside the worktree is always the same as the directory name of the worktree.

    So the branch name should be x/branch as well. Note that the worktree cannot contain slashes, so this may be a bit ugly to set up.

    opened by hakoerber 1
  • Add contributing guidelines

    Add contributing guidelines

    Write it down in a CONTRIBUTING.md and/or in docs.

    Contribution workflow:

    • Open issue for "bigger" changes / ideas so we can discuss them
    • Make just check work
      • Formatting: cargo fmt
      • Linting: cargo clippy
      • Tests: See below
    • Usual github pull request workflow against develop.
    • Add yourself to CONTRIBUTORS if wanted

    Testing:

    • Unit / integ tests in rust (very small test suite) => just test-unit test-integration
    • e2e test suite in e2e_tests. Should catch all scenarios that grm can be used in. Might make sense to write a test here that fails when making changes, to prevent regressions in the future. => just test-e2e
    opened by hakoerber 1
  • Use YAML for the configuration

    Use YAML for the configuration

    As it's deeply nested, TOML was not a good choice. There is a lot of duplication and the section headers get quite verbose.

    YAML is the better choice in hindsight.

    Note that a possible global config would still use TOML.

    idea 
    opened by hakoerber 1
  • Output group name if not found

    Output group name if not found

    When using grm repos sync with a --group that does not exist, the output is just this:

    Error: 404 Group Not Found
    

    It should at least include the group name.

    enhancement good first issue 
    opened by hakoerber 0
  • build(deps): bump openssl-src from 111.21.0+1.1.1p to 111.22.0+1.1.1q

    build(deps): bump openssl-src from 111.21.0+1.1.1p to 111.22.0+1.1.1q

    Bumps openssl-src from 111.21.0+1.1.1p to 111.22.0+1.1.1q.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Update README

    Update README

    • [x] "Future & Ideas" is out of date
    • [x] "Crates" is out of date
    • [x] No reference to YAML
    • [x] No mention of worktrees
    • [x] Add mirrors
    documentation 
    opened by hakoerber 0
  • Streamline & document worktree handling

    Streamline & document worktree handling

    Currently, some things are a bit hit-and-miss:

    • Remotes
    • Remote tracking branches
    • Handling of local branches if a remote branch with the same name exists
    • Subdirectories
    • Branches with slashes

    Remotes

    Overview:

    • If there is only one remote, this is implicitly used as the "default"
    • If there is a remote defined as default (via default_remote), this is the "default"
    • If there is more than one remote and no default, well, there is no default remote

    In case we have default remote, everything is easy: That's the remote we use to set up remote tracking branches and pull/push/fetch from.

    If there are multiple remotes without a default, the remote will always have to be specified for push/pull/fetch and tracking branches. I guess this is a very rare setup, so it will be fine to emit warnings and point users to default_remote.

    • [x] Verify the above with e2e tests
    • [x] Document the behaviour

    Remote tracking branches

    The desired behaviour is:

    • The remote tracking branch is always the same name as the local name, except when either:
      • There is an explicit override with --track
      • There is a default prefix in the configuration file

    When we create a worktree, there are a few different cases to handle:

    • A local branch exists => use it. If it is set to track a remote branch that does not match the naming, emit a warning, but leave the remote tracking branch alone.
    • A local branch does not exist, but a remote branch does => use it and set tracking
    • Neither => Create a new local branch without tracking, except when tracking is configured (track.default = true)
    • [x] Verify the above with e2e tests
    • [x] Document the behaviour

    Worktrees & Subdirectories

    The rule is: Branch name == Worktree name. As a branch name can contain a slash, we cannot really handle subdirectories, as there would be no way to tell a subdirectory and a branch prefix apart. So subdirectories need to be removed.

    • [x] Remove subdirectory handling from worktrees
    • [x] Document the behaviour
    • [x] Make sure that subdirectories are removed together with the subdirectory
    opened by hakoerber 0
  • Use

    Use "origin" instead of provider-dependent remote name as default

    Right now, e.g. when using the GitLab provider, the remote will be called gitlab. It's less surprising to use origin instead.

    If required, it can still be overriden with --remote-name.

    opened by hakoerber 0
  • identifying nested repositories

    identifying nested repositories

    Great work, I really like the project and thank you for your initiative.

    In reference to #48 it would be great if grm repos find local would not stop as soon as it finds a repository but it would continue to walk the directory tree to find other possible git repos. Some projects have a pattern of adding some directory to .gitignore and then putting sub repositories over there without adding them as submodules.

    opened by pythys 2
  • Enable serde(deny_unknown_fields) for config structs

    Enable serde(deny_unknown_fields) for config structs

    Right now, the configuration files can contain additional keys without problems, they are just ignored. This is good for backwards compatibility during development, but can lead to confusing situation (e.g. when mistyping a key).

    Enabling #[serde(deny_unknown_fields)] for those structs would reject those configuration files.

    • [x] Enable the setting
    • [ ] Check the error codes

    The errors are very unhelpful:

    grm repos sync config -c ~/grm.toml
    [✘] Error parsing configuration file "grm.toml": data did not match any variant of untagged enum Config
    
    future 
    opened by hakoerber 0
  • Putting names in toml hash table headings

    Putting names in toml hash table headings

    I wonder if you see any merit it putting names into using a slightly simplified (I think) configuration. This uses quite a few arrays as is that contain names for things that are already atomic. For example, directory names and remote names already require uniqueness.

    In my view, this would make a slightly more terse and readable configuration in any format. For example, this config.toml:

    [[trees]]
    root = "~/projects"
    
    [[trees.repos]]
    name = "github.com/theherk/terraform-aws-apigateway-route-builder"
    worktree_setup = false
    
    [[trees.repos.remotes]]
    name = "origin"
    url = "[email protected]:theherk/terraform-aws-apigateway-route-builder.git"
    type = "ssh"
    
    [[trees.repos]]
    name = "github.com/theherk/helix"
    worktree_setup = false
    
    [[trees.repos.remotes]]
    name = "origin"
    url = "[email protected]:theherk/helix.git"
    type = "ssh"
    
    [[trees.repos.remotes]]
    name = "upstream"
    url = "https://github.com/helix-editor/helix.git"
    type = "https"
    

    would become:

    root = "~/projects"
    
    ["github.com".theherk.terraform-aws-apigateway-route-builder]
    worktree_setup = false
    
    ["github.com".theherk.terraform-aws-apigateway-route-builder.remotes.origin]
    type = "ssh"
    url = "[email protected]:theherk/terraform-aws-apigateway-route-builder.git"
    
    ["github.com".theherk.helix]
    worktree_setup = false
    
    ["github.com".theherk.helix.remotes.origin]
    type = "ssh"
    url = "[email protected]:theherk/helix.git"
    
    ["github.com".theherk.helix.remotes.upstream]
    type = "https"
    url = "https://github.com/helix-editor/helix.git"
    

    That would have the byproduct of forcing names not be duplicated in these cases, and it feels a bit more nice to me.

    Of course, you have put together a wonderful tool. Thank you.


    Also, I think this could be further simplified using some very sensible heuristics to get the same configuration from:

    root = "~/projects"
    
    ["github.com".theherk.terraform-aws-apigateway-route-builder]
    worktree_setup = false
    
    ["github.com".theherk.helix]
    worktree_setup = false
    
    ["github.com".theherk.helix.remotes.upstream]
    url = "https://github.com/helix-editor/helix.git"
    
    opened by theherk 2
  • Allow multiple forges

    Allow multiple forges

    Different remotes could be used at the same time, with the repositories being combined.

    What information to use to determine whether two projects are the same on both forges?

    Could be:

    • :question: Full path: Would only work in certain circumstances.
    • :x: Repository name: Not possible, as it's not unique per forge (repos with the same name can be in different groups)

    There could also be a mapping that allows namespaces to be mapped between forges. Something like --map=github:github_user=gitlab=gitlab_user. Or in config:

    [[mapping]]
    github = "github_user"
    gitlab = "gitlab_user"
    

    But: What should be given to --group or --user then? Better to have a mapping from a forge-independent handle to a name for each forge. E.g:

    --map=myuser@github:github_user@gitlab:gitlab_user --user=myuser

    Or in config:

    [mapping.myuser]
    github = "github_user"
    gitlab = "gitlab_user"
    

    Whenever GRM then encounters a (namespace, name) tuple from a forge, it checks whether namespace or any parent matches the mapping, and replaces it accordingly.

    E.g.:

    Imagine the mapping from above:

    | Namespace in | Namespace out | | ------------- | ------------- | | /myuser/ | /github_user/ | /myuser/mysubgroup/| /github_user/mysubgroup/ | /group/myuser/ | No change, as the mapping is not a prefix | /myuserfoo/ | No change, as the mapping only specifies part of the path component

    This mapping would also transfer to the rest of GRM like local path names for clones.

    This would also be cool to put into a global config in $XDG_CONFIG_HOME

    enhancement idea research required 
    opened by hakoerber 0
  • Set push.default = upstream only when necessary

    Set push.default = upstream only when necessary

    push.default = upstream is only necessary when the remote branch name differs from the local branch name. Otherwise, simple is much more ergonomic.

    upstream leads to weird behaviour if there are multiple remotes. The problem is that only one remote can be the one with the upstream branch. Pushing to others without an explicit remote branch leads to the following error:

    --- [origin] ---
    fatal: You are pushing to remote 'origin', which is not the upstream of
    your current branch 'develop', without telling me what to push
    to update which remote branch.
    

    It might make sense to print a warning in that case (multiple remotes with branch naming mismatch), which should be rare enough.

    Currently, push.default is just set during worktree init/clone/conversion. In the future, this would also need to be done during each sync, as the number of remotes may change and the setting may have to be changed accordingly.

    Also, the setting is currently repo-specific. But actually, it would be worktree-specific. Some worktrees might match the remote branch (master/main/develop being common), while others may not (I like to use a prefix with feature branches for example). Only the latter should use push.default = upstream when using multiple remotes.

    enhancement 
    opened by hakoerber 0
Owner
Hannes Körber
Coffee | Me > Code
Hannes Körber
Verify that registry crates in your Cargo.lock are reproducible from the git repository

cargo-goggles Verify that registry crates in your Cargo.lock are reproducible from the git repository. This cargo subcommand analyzes the following pr

M4SS - Industrial IoT Solutions 36 Jul 16, 2024
Simple git/hg tui client focused on keyboard shortcuts

verco A simple Git/Hg tui client focused on keyboard shortcuts Screenshots Platforms This project uses Cargo and pure Rust stable and works on latest

Matheus Lessa Rodrigues 214 Dec 26, 2022
🌌⭐ Git tooling of the future.

❯ Glitter Git tooling of the future. ❯ ?? Features Config files Fast Easy to use Friendly errors ❯ ?? Documentation For proper docs, see here ❯ ✋ What

Milo 229 Dec 22, 2022
A small tool to clone git repositories to a standard location, organised by domain name and path.

A small tool to clone git repositories to a standard location, organised by domain name and path. Runs on BSD, Linux, macOS, Windows, and more.

Wesley Moore 68 Dec 19, 2022
Stacked branch management for Git

git-stack Stacked branch management for Git Dual-licensed under MIT or Apache 2.0 Documentation About Installation Getting Started Reference FAQ Compa

Ed Page 1 Jul 18, 2022
Lintje is an opinionated linter for Git.

Lintje Lintje is an opinionated linter for Git. It lints commit messages based on a preconfigured set of rules focussed on promoting communication bet

Tom de Bruijn 26 Dec 18, 2022
Replay git history with some tweaks

Git-replay Overview Git-replay is a simple tool that replays the history of a Git repository but with some tweaks (i.e., it can change the author name

Tak-gun Na 3 Mar 22, 2022
A git hook to manage derivative files automatically.

git-derivative A git hook to manage derivative files automatically. For example if you checked out to a branch with different yarn.lock, git-derivativ

Sung Jeon 3 Oct 30, 2022
Simple and fast git helper functions

Simple and fast git helper functions

LongYinan 126 Dec 11, 2022
Verbump - A simple utility written in rust to bump and manage git semantic version tags.

Verbump - A simple utility written in rust to bump and manage git semantic version tags.

Sarat Chandra 6 May 6, 2022
A git prepare-commit-msg hook for authoring commit messages with GPT-3.

gptcommit A git prepare-commit-msg hook for authoring commit messages with GPT-3. With this tool, you can easily generate clear, comprehensive and des

Roger Zurawicki 3 Jan 19, 2023
Original Version Management System based on Git

nss (noshishi) Original Version Management System based on Git. Learning git and rust for good developer. Usage Install cargo install nssi how to nss

nopeNoshishi 4 Feb 13, 2023
Easily add emojis to your git commit messages 😎

gimoji A CLI tool that makes it easy to add emojis to your git commit messages. It's very similar to (and is based on) gitmoji-cli but written in Rust

Zeeshan Ali Khan 12 May 29, 2023
Command palette-style Git client for blazing-fast commits.

?? About Commit Commit is the world's simplest Git client. Open it with a keyboard shortcut, write your commit, and you're done! Commit will automatic

Miguel Piedrafita 190 Aug 18, 2023
🌲 Open the current remote repository in your browser

gitweb Some of the flags and options are subject to change in the future. Ideas are welcome. Ideas are bulletproof (V). gitweb is a command line inter

Yoann Fleury 26 Dec 17, 2022
Repository for Public Impervious Releases

Impervious Releases This is the repository for impervious releases and supporting files and documentation. Binaries The binaries are now released and

null 67 Dec 20, 2022
Clean up the lines of files in your code repository

lineman Clean up the lines of files in your code repository NOTE: While lineman does have tests in place to ensure it operates in a specific way, I st

Joseph T. Lyons 4 Nov 25, 2021
[DEPRECATED] password manager thing

NOTE: this is pretty much abandoned, I recommend Bitwarden now. There is an export command in the CLI that produces CSV suitable for import into Bitwa

null 312 Oct 22, 2022
Remote Secret Editor for AWS Secret Manager

Barberousse - Remote Secrets Editor About Usage Options Printing Editing Copying RoadMap 1.0 1.1 Future About A project aimed to avoid downloading sec

Mohamed Zenadi 18 Sep 28, 2021