Git mirroring daemon

Overview

lohr

lohr is a Git mirroring tool.

I created it to solve a simple problem I had: I host my own git server at https://git.alarsyo.net, but want to mirror my public projects to GitHub / GitLab, for backup and visibility purposes.

GitLab has a mirroring setting, but it doesn't allow for multiple mirrors, as far as I know. I also wanted my instance to be the single source of truth.

How it works

Gitea is setup to send webhooks to my lohr server on every push update. When lohr receives a push, it clones the concerned repository, or updates it if already cloned. Then it pushes the update to all remotes listed in the .lohr file at the repo root.

Destructive

This is a very destructive process: anything removed from the single source of truth is effectively removed from any mirror as well.

Installing

lohr is published on crates.io, so you can install it with cargo install:

$ cargo +nightly install lohr

We currently require a nightly compiler because Rocket needs one to compile (a 0.5.0 which compiles on stable Rust is in the making, stay tuned!). You can install a nightly toolchain with the following command:

$ rustup install nightly

Setup

Quickstart

Setting up lohr should be quite simple:

  1. Create a Rocket.toml file and add your configuration.

  2. Export a secret variable:

    $ export LOHR_SECRET=42 # please don't use this secret
    
  3. Run lohr:

    $ cargo run # or `cargo run --release` for production usage
    
  4. Configure your favorite git server to send a webhook to lohr's address on every push event.

    I used Gitea's webhooks format, but I think they're similar to GitHub and GitLab's webhooks, so these should work too! (If they don't, please file an issue!)

    Don't forget to set the webhook secret to the one you chose above.

  5. Add a .lohr file containing the remotes you want to mirror this repo to:

    [email protected]:you/your_repo
    

    and push it. That's it! lohr is mirroring your repo now.

Configuration

Home directory

lohr needs a place to clone repos and store its data. By default, it's the current directory, but you can set the LOHR_HOME environment variable to customize it.

Shared secret

As shown in the quickstart guide, you must set the LOHR_SECRET environment variable.

Extra remote configuration

You can provide lohr with a YAML file containing additional configuration. You can pass its path to the --config flag when launching lohr. If no configuration is provided via a CLI flag, lohr will check the LOHR_CONFIG environment variable. If the environment variable isn't set either, it will check in LOHR_HOME is a lohr-config.yaml file exists, and try to load it.

This file takes the following format:

default_remotes:
    - "git@github:user"
    - "git@gitlab:user"

additional_remotes:
    - "[email protected]:~user"

blacklist:
    - "private-.*"
  • default_remotes is a list of remotes to use if no .lohr file is found in a repository.
  • additional_remotes is a list of remotes to add in any case, whether the original set of remotes is set via default_remotes or via a .lohr file.
  • blacklist is a list of regular expressions to match against the full repository names. Any that matches will not be mirrored, even if it contains a .lohr file.

Both settings take as input a list of "stems", i.e. incomplete remote addresses, to which the repo's name will be appended (so for example, if my default_remotes contains [email protected]:alarsyo, and a push event webhook is received for repository [email protected]:some/long/path/repo_name, then the mirror destination will be [email protected]:alarsyo/repo_name.

Contributing

I accept patches anywhere! Feel free to open a GitHub Pull Request, a GitLab Merge Request, or send me a patch by email!

Why lohr?

I was looking for a cool name, and thought about the Magic Mirror in Snow White. Some furious wikipedia searching later, I found that the Magic Mirror was probably inspired by the Talking Mirror in Lohr am Main. That's it, that's the story.

License

lohr is distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT for details.

You might also like...
Git FIDO Helper - Sign your Git commits with multiple resident SSH keys

gfh Git FIDO helper, or God Fucking Help me. gfh is a tool for helping you sign your commits in Git with resident SSH keys stored on multiple FIDO dev

A spotify daemon

Spotifyd An open source Spotify client running as a UNIX daemon. Spotifyd streams music just like the official client, but is more lightweight and sup

Synapse BitTorrent Daemon

synapse Synapse is a flexible and fast BitTorrent daemon. It currently supports most systems which implement epoll or kqueue, with a focus on 64-bit l

Docker daemon API in Rust

Bollard: an asynchronous rust client library for the docker API Bollard leverages the latest Hyper and Tokio improvements for an asynchronous API cont

A highly efficient daemon for streaming data from Kafka into Delta Lake

kafka-delta-ingest The kafka-delta-ingest project aims to build a highly efficient daemon for streaming data through Apache Kafka into Delta Lake. Thi

Daemon and tools to control your ASUS ROG laptop. Supersedes rog-core.

asusctl for ASUS ROG - Asus Linux Website asusd is a utility for Linux to control many aspects of various ASUS laptops but can also be used with non-a

A wallpaper daemon for Wayland compositors

Paper A wallpaper daemon for Wayland compositors implementing the layer-shell protocol. Features Supports png and jpg format Tiled wallpapers Bordered

Process killer daemon for out-of-memory scenarios

bustd: Available memory or bust! bustd is a lightweight process killer daemon for out-of-memory scenarios for Linux! Features Small memory usage! bust

⏳ trackie is a private, daemon-less time tracker for your CLI.
⏳ trackie is a private, daemon-less time tracker for your CLI.

⏳ trackie `trackie` is a private, daemon-less time tracker running in your CLI. Trackie offers an easy CLI to track the time you spent on your various

A highly efficient daemon for streaming data from Kafka into Delta Lake

A highly efficient daemon for streaming data from Kafka into Delta Lake

Simple daemon built with Rust to track metrics.

Marvin - Metrics Tracker What I cannot create, I do not understand. — Richard Feynman Simple daemon built with Rust to track metrics. The goal is run

`haproxy_autconfd` is a daemon that automatically assembles a HAProxy config and restarts HAProxy if the config changes

haproxy_autconfd Welcome to haproxy_autconfd 🎉 haproxy_autconfd is a daemon that automatically assembles a HAProxy config and restarts HAProxy if the

LeftHK - A hotkey daemon written in Rust

LeftHK LeftHK - A hotkey daemon written in Rust THIS IS BETA SOFTWARE The configuration file should be created in ~/.config/lefthk/ and called config.

bustd is a lightweight process killer daemon for out-of-memory scenarios for Linux!

bustd: Available memory or bust! bustd is a lightweight process killer daemon for out-of-memory scenarios for Linux! Features Small memory usage! bust

s3d is an S3 daemon for the Edge written in Rust
s3d is an S3 daemon for the Edge written in Rust

s3d is an S3 daemon for the Edge written in Rust The goal of s3d is to provide a daemon for edge platforms (e.g. factory servers 🏭 planes 🛩 ships 🚢

NymDrive is a complete, end-to-end encrypted file syncing daemon that runs over the Nym network.

NymDrive NymDrive is a complete, end-to-end encrypted file syncing daemon that runs over the Nym network. Features Active file monitoring of changes i

Wallpaper daemon for Wayland

wpaperd wpaperd is a minimal wallpaper daemon for Wayland. It allows the user to choose a different image for each output (aka for each monitor) just

A Simple Audio Control and Notifications Daemon

SACAND This is sacand, a Simple Audio Control and Notifications Daemon As it name oaths to imply, this is intended to be a simple audio control daemon

Rust Audio Player Daemon

Rust Audio Player Daemon Cause mpd was annoying What rapd trys to do Rapd is not a spotify client, or an advanced music player. Its an audio/music dae

Comments
  • Support cloning private repos

    Support cloning private repos

    Currently pushing to private repos works because the user can provide SSH urls if he wants to; but if the mirrored repo is private, using the webhook's clone_url field won't do, as it uses https by default. We should use ssh_url instead.

    opened by alarsyo 0
  • Support git push options

    Support git push options

    Motivating example: Sourcehut's push options includes a way to set the repository's visibility when pushing to it. Lohr could either try to be smart and set the corresponding option automatically when pushing a repository, or it should be configurable using the .lohr file.

    I'm more partial to it being set as a general config, as I would rather avoid having to write .lohr files.

    opened by ambroisie 4
Releases(v0.4.2)
Owner
Antoine Martin
Computer Science student
Antoine Martin
The Git Commit Message and Changelog Generation Framework :book:

git-journal ?? The Git Commit Message and Changelog Generation Framework Table of contents: TL;DR Installation Usage Default output Template output Co

Sascha Grunert 551 Jan 4, 2023
gstats — command line tool to print a developer handy summary of all git repositories below current directory

gstats Simple Rust tool to get quick summary info on git repos showing latest tag, branch, state. I implemented this to help me work with a the not to

Boon at Shift 12 Jun 10, 2021
A parallel universal-ctags wrapper for git repository

ptags A parallel universal-ctags wrapper for git repository Description ptags is a universal-ctags wrapper to have the following features. Search git

null 107 Dec 30, 2022
By mirroring traffic to and from your machine, mirrord surrounds your local service with a mirror image of its cloud environment.

mirrord lets you easily mirror traffic from your Kubernetes cluster to your development environment. It comes as both Visual Studio Code extension and

MetalBear 2.1k Jan 3, 2023
Mirroring remote repositories to s3 storage, with atomic updates and periodic garbage collection.

rsync-sjtug WIP: This project is still under development, and is not ready for production use. rsync-sjtug is an open-source project designed to provi

SJTUG 57 Feb 22, 2023
A Rust CLI tool that helps you enforce Git policies through Git hooks both server and client side

GitPolicyEnforcer This is a command line utility written in Rust, that helps you utilize Git hooks, to enforce various policies. It currently supports

Vagelis Prokopiou 4 Aug 14, 2022
git-cliff can generate changelog files from the Git history by utilizing conventional commits as well as regex-powered custom parsers.⛰️

git-cliff can generate changelog files from the Git history by utilizing conventional commits as well as regex-powered custom parsers. The changelog template can be customized with a configuration file to match the desired format.

Orhun Parmaksız 5k Jan 9, 2023
First Git on Rust is reimplementation with rust in order to learn about rust, c and git.

First Git on Rust First Git on Rust is reimplementation with rust in order to learn about rust, c and git. Reference project This project refer to the

Nobkz 1 Jan 28, 2022
A git sub-command to view your git repository in the web browser

git-view A git sub-command to view your git repository in the web browser! About Are you also frustrated from moving your hands away from the keyboard

Hamothy 5 Sep 26, 2022
Git Explorer: cross-platform git workflow improvement tool inspired by Magit

Gex Git workflow improvement CLI tool inspired by Magit. This project is still under initial development, but I am actively dogfooding it and features

Peter Hebden 204 Jan 6, 2023