There are some optional fields that weren't being added to the Packages archive. This would cause packages that have those fields to give repeated update messages.

hi @mmstick, first up thanks for a sweet tool for Debian packaging! :100:

i'd like to use debrep for building Debian packages for a project PeachCloud :peach: :cloud: , which will run on (Raspberry Pi) ARM devices, so i'm keen to add cross-compilation support to deprep.

here's a pull request that seems to work:

• adds architectures field to suite config, which defaults to [amd64, i386]
• runs sbuild for each architecture (in a iter.try_for_each)
• architecture matches support all official Debian architectures: https://www.debian.org/ports/

here's a demo using this to build a basic Rust package for amd64, i386, arm64, and armhf: https://github.com/peachcloud/peach-packages.

happy to implement this feature however you want, let me know if anything is not okay, or if you want something done differently, or if i'm missing a better way to do this, etc. i kept the changes pretty minimal to get it working and start the conversation, but for example at the moment this compiles every architecture in series and then writes the record object without any architecture information, maybe you have a better idea.

cheers! :smiley_cat:

Currently testing locally

• Built source packages will now be recorded at /records/${suite}/package-name
• Builds now occur at /build/${suite}/
• Metapackages are now moved to /metapackages/${suite}/
• Package logs will be retained for each suite at /logs/${suite}/package-name
• Debian packaging data will now be separated at /debian/${suite}/package-name

This will fix source builds on cosmic so that the CUDA packages will build on both bionic and cosmic separately.

• Refactoring the config module to separate the source, repo, and direct structures.
• Added a new repos field for handling repo-based sources.
• Added a new repos module to repo::download for fetching sources from a repo.
• Split the web crawling functionality out to a new url-scraper, url-crawler, and apt-repo-crawler crate

Example

[[repos]]
repo = "http://ppa.launchpad.net/js-reynaud/kicad-5/ubuntu/pool/main/"
version = { is = "ubuntu18.04" }
arch = { not = "i386" }

[[repos]]
repo = "http://ppa.launchpad.net/freecad-maintainers/freecad-stable/ubuntu/pool/main/"
version = { is = "ubuntu18.04" }
arch = { not = "i386" }

# Peek GTK application
[[repos]]
repo = "http://ppa.launchpad.net/peek-developers/stable/ubuntu/pool/main/"
version = { is = "ubuntu18.04" }
arch = { not = "i386"}

# Updated thunderbird packages, and firefox-esr
[[repos]]
repo = "http://ppa.launchpad.net/mozillateam/ppa/ubuntu/pool/main/"
version = { is = "18.04" }
arch = { is = "amd64|all" }

Debian versions require either the timestamp or revision count to be included in the version in order for packages to be properly incremented when building git updates. Timestamps can be fetched with git show -s --format=%ct, and revision count with git rev-list --count --first-parent HEAD.

Closes #33

• dsc files may now be specified as a possible source location variant
• The git source location variant now supports specifying a branch and commit
• Enable automatic changelog updates of git sources, to append the commit to the changelog

Example

[DEBUG] debrep: running "git" "-C" "/home/mmstick/Sources/repo-curated-free/build/cosmic/piserver" "rev-parse" "master"
[DEBUG] debrep: running "dch" "-D" "cosmic" "-l" "~40a78a" "-c" "/home/mmstick/Sources/repo-curated-free/build/cosmic/piserver/debian/changelog" "automatic build of commit 40a78a"
[INFO] debrep: building piserver at changelog version 0.3~40a78a1

building ion-shell at changelog version 1.0.0-alpha0-cosmic2~0bff9e1

[[source]]
name = "ion-shell"
version = "1.0.0-alpha0-cosmic2"

    [source.location]
    dsc = "https://launchpad.net/~mmstick76/+archive/ubuntu/${name}/+sourcefiles/${name}/${version}/${name}_${version}.dsc"

[[source]]
name = "ion-shell"
build_on = "changelog"

    [source.location]
    git = "https://gitlab.redox-os.org/redox-os/ion/"
    branch = "master"
    commit = "0bff9e12"

• This is required to support building Tensorflow 1.11 for CUDA 10.0
• Also fixes extra sbuild package dependencies, when multiple versions exist

Closes #15

When generating dist files, older versions were getting picked up instead of new versions. This will use the version comparison function that the repo fetching config also uses, so that 10.0 is considered newer than 9.2.

To better support importing software which has already been packaged for Debian.

• [ ] Support fetching dsc files remotely with dget
• [ ] Support building from locally-stored dsc / tar files
• [ ] Support applying additional quilt patches

The lone sources.toml file has been done away with to now offer a suites directory for storing toml configs for each suite to be built. No changes were made outside of the location for storing suite configrations. The new layout will be:

suites/
    bionic.toml
    cosmic.toml

The parallel-getter dependency has also been updated to use the version on Crates.io.

• Fixes version comparisons, as found when attempting to backport rustc 1.28.0 from cosmic.
• No longer assumes that received apt entries will be in order

• [ ] Provide a GitHub / GitLab repo, or a list of individual repos
• [ ] Provide a destination repo which we can compare against / submit to
• [ ] Fetch each repo and their branches to the "CPI" system
• [ ] Watch for new branches, and updated branches
• [ ] Generate a list of repos that need to be built
• [ ] Generate packages, and submit packages

It should be possible to:

• Specify the base URL of a repo
• Specify the suite to pull from
• Fetch packages from the packages list(s) for that suite

The implementation belongs in apt-fetcher.

[ ] Support capability to determine what packages were changed between the last dists update [ ] Insert details into the dists file at the correct location, rather than regenerating them from scratch

• [x] Support automatically generating and appending an entry to a package's changelog (debchange)
• [ ] Support repacking a remote deb archive with an updated changelog