Download Apple's open source code from opensource.apple.com

Overview

Apple Open Source Downloader

This repository defines a Rust crate and CLI program to automate the downloading of Apple's open source code from https://opensource.apple.com/.

The primary goal of this project is to enable more intuitive usage and inspection of Apple's open source code. Using this tool you can:

  • Convert the history of an Apple component (like the xnu core OS primitives) to a Git repository and easily view differences between releases.
  • Convert the history of a set of open source components (such as everything comprising macOS) to a Git repository and easily view differences between releases.
  • Query for all available open source components and releases.

The canonical home for this project is https://github.com/indygreg/apple-opensource-downloader. Please report issues or submit enhancements there.

Installing

# From crates.io
$ cargo install apple-opensource-downloader

# From Git
$ cargo install --git https://github.com/indygreg/apple-opensource-downloader.git --branch main

Using

The apple-opensource-downloader CLI is provided. It defines sub-commands to perform various actions. Run apple-opensource-downloader help to see the help.

Download a Single Component to a Git Repository

The component-to-git sub-command will download all versions of a named software component (see the component list at https://opensource.apple.com/tarballs/) and write their contents as Git commits to a Git repository. This allows you to see differences between the versions of a component.

$ apple-opensource-downloader component-to-git --no-bare xnu aos/xnu

   
    

$ cd aos/xnu
$ git log
commit c2011455c3d75195791bd20d189abae4917c8c81 (HEAD -> main, tag: 7195.141.2)
Author: Apple Open Source 
    
     
Date:   Fri Jan 1 00:00:00 2021 +0000

    xnu 7195.141.2

    Downloaded from https://opensource.apple.com/tarballs/xnu/xnu-7195.141.2.tar.gz

commit e76ea20b5519ae2eaf2b74698bd2331141e028fa (tag: 7195.121.3)
Author: Apple Open Source 
     
      
Date:   Fri Jan 1 00:00:00 2021 +0000

    xnu 7195.121.3

    Downloaded from https://opensource.apple.com/tarballs/xnu/xnu-7195.121.3.tar.gz

...

     
    
   

The Git trees and commit objects should be deterministic provided that the version of this software is identical and the Apple-hosted source archives don't change. i.e. different machines should produce Git commits with the same commit IDs.

Download all Components to Git Repositories

The components-to-gits sub-command will download each available component and write each to separate Git repositories. It is equivalent to running component-to-git for every named component.

Download An Apple Software Release to a Git Repository

The release-to-git command can be used to download all components in a logical Apple software release (such as macOS or iOS) to a Git repository. This enables you to view the history and changes of open source components between releases.

$ apple-opensource-downloader release-to-git --no-bare macos aos/macOS
$ cd aos/macOS
$ git log


Known Issues

The HTML parsing isn't the most robust and may not scrape all available software.

If Apple changes the HTML on opensource.apple.com, it will break this tool.

If Apple imposes throttling on their servers, it will likely break this tool.

Created Git repositories don't use packfiles and their performance may be sub-optimal. Run git gc after Git repo creation to optimize the Git repositories.

We don't support incrementally updating Git repositories. Git repositories have their history recreated from scratch on every invocation. This is obviously inefficient.

Various advertised URLs on opensource.apple.com result in an HTTP 404. These are sometimes ignored by this tool.

When importing software releases (such as macOS), the components from one release to the next may vary. e.g. SQLite could be there in release A, gone in release B, and reappear in release C. This may make git diff output non-representative.

Git commits have a hard-coded date that has no basis in reality.

The naming and layout of Apple's components can at times be confusing and inconsistent. We don't yet make an effort to reconcile this.

Legal Compliance

The content downloaded by this tool may be governed by license and usage restrictions defined outside this tool. Check for usage restrictions posted at https://opensource.apple.com/ and within the downloaded content.

i.e. if you redistribute the downloaded content, Apple may take an issue with that.

Owner
Gregory Szorc
Gregory Szorc
MeiliSearch is a powerful, fast, open-source, easy to use and deploy search engine

MeiliSearch is a powerful, fast, open-source, easy to use and deploy search engine. Both searching and indexing are highly customizable. Features such as typo-tolerance, filters, and synonyms are provided out-of-the-box. For more information about features go to our documentation.

MeiliSearch 19.6k Nov 24, 2021
An open source virtual tabletop that is currently purely a concept.

An open source virtual tabletop that is currently purely a concept.

null 3 Sep 27, 2021
Blueboat is an open-source alternative to Cloudflare Workers. The monolithic engine for serverless web apps.

Blueboat Blueboat is an open-source alternative to Cloudflare Workers. Blueboat aims to be a developer-friendly, multi-tenant platform for serverless

Heyang Zhou 674 Nov 29, 2021
Simple MHV6 extension used to download custom songs efficiently and effectively.

nong-downloader-extension A simple MegaHack v6 extension that helps you download NONG songs directly to GD. Compiling. Why would you want to compile??

Alpha 4 Nov 24, 2021
Automatically download minecraft server jars in one line

MCDL Automatically download minecraft server jars in one line (or one click) Installation Download (Windows, Linux) Install via cargo: cargo install m

Isaac Hirschfeld 1 Oct 26, 2021
Source code for the Telegram channel @pixiv_daily

PixivDaily (Rust) This repository contains the source code of the program running the Telegram channel @pixiv_daily. Usage First, you'll need to clone

K4YT3X 4 Sep 6, 2021
ttasm is an assembler for assembling source code to TTVM executable.

ttasm is an assembler for assembling source code to TTVM executable.

maDeveloper 1 Nov 8, 2021
unFlow is a Design as Code implementation, a DSL for UX & backend modeling. DSL to Sketch file, Sketch to DSL, DSL to code.

unflow 是一个低代码、无代码设计语言。unFlow is a Design as Code implementation, a DSL for UX & backend modeling. DSL to Sketch file, Sketch to DSL, DSL to code.

Inherd OS Team (硬核开源小组) 56 Nov 10, 2021
This is a public snapshot of Fly's init code. It powers every Firecracker microvm we run for our users.

Fly Init This is a public snapshot of Fly's init code. It powers every Firecracker microvm we run for our users. It is Rust-based and we thought makin

fly.io 108 Nov 22, 2021
Rust ABI safe code generator

CGlue offers an easy way to ABI (application binary interface) safety. Just a few annotations and your trait is ready to go!

Auri 55 Nov 22, 2021
A boiler plate code to create dynamic link library in rust.

?? rust-dll-bp This is a boiler plate code that will be generated as a dll binary. I personally cache this here for me but if you're intend to create

s3pt3mb3r 2 Nov 7, 2021
Advent of Code 2015, done entirely in Rust both for the challenge and as a way to learn

Advent of Code 2015 In preparation for Advent of Code 2021, I wanted to go back and try some of the older challenges. I figured it made the most sense

Matt 1 Nov 27, 2021
The working code for my Safe Pay app - check out my tutorial!

Safe Pay Tutorial This repository contains the source code for my blog post Using PDAs and SPL Token in Anchor. The code contains the following: Solan

Daniel Pyrathon 7 Nov 23, 2021
A basic rp2040-hal project with blinky and rtt logging example code.

A basic rp2040-hal project with blinky and rtt logging example code. With this you can quickly get started on a new rp2040 project

rp-rs 25 Nov 28, 2021
A stupid macro that compiles and executes Rust and spits the output directly into your Rust code

inline-rust This is a stupid macro inspired by inline-python that compiles and executes Rust and spits the output directly into your Rust code. There

William 1 Nov 22, 2021
Download a file using multiple threads in parallel for faster download speeds.

multidl Download a file using multiple threads in parallel for faster download speeds. Uses 0 external dependencies. Usage Usage: multidl [--help] ADD

Divyanshu Agrawal 2 Sep 12, 2021
ANISE provides an open-source and open-governed library and algorithmic specification for most computations for astrodynamics

ANISE provides an open-source and open-governed library and algorithmic specification for most computations for astrodynamics. It is heavily inspired by NAIF SPICE, and may be considered as an open-source modern rewrite of SPICE.

ANISE 2 Nov 22, 2021
Opensource diagnostic software for Daimler vehicles, inspired by Xentry and DAS, written in Rust

OPENSTAR An opensource diagnostic application for Daimler vehicles inspired by DAS and Xentry. Some of the work here is based on OpenVehicleDiag If yo

Ashcon Mohseninia 11 Oct 31, 2021
virtualization-rs provides the API of the Apple Virtualization.framework in Rust language.

virtualization-rs Rust bindings for Virtualization.framework virtualization-rs provides the API of the Apple Virtualization.framework in Rust language

suzu 24 Nov 20, 2021
Punic is a remote caching CLI built for Apple's .xcframework

Punic is a remote caching CLI built for Carthage that exclusively supports Apple's .xcframeworks.

Shred Labs 24 Oct 23, 2021
A CLI for extracting libraries from Apple's dyld shared cache file

dyld-shared-cache-extractor As of macOS Big Sur, instead of shipping the system libraries with macOS, Apple ships a generated cache of all built in dy

Keith Smiley 89 Nov 23, 2021
Bad Apple!! for the Uxn stack-machine

bad-apple-uxn Bad Apple!! for the Uxn stack-machine by Hundred Rabbits. Quick start cargo run --release -- --input assets/badapple.mp4 --output badapp

Karol Belina 10 Sep 22, 2021
A work-in-progress, open-source, multi-player city simulation game.

Citybound is a city building game with a focus on realism, collaborative planning and simulation of microscopic details. It is independently developed

Citybound 6.6k Nov 26, 2021
4fun open-source Cave Story reimplementation written in Rust

doukutsu-rs Download latest Nightly builds (Requires being logged in to GitHub) A re-implementation of Cave Story (Doukutsu Monogatari) engine written

null 251 Nov 22, 2021
The open source design documentation tool for everybody

Heads up: reimagining artifact 3.0, check it out at artifact_py Artifact: design documentation for everybody Note: this project, and the python re-wri

Rett Berg 567 Nov 20, 2021
An extensible open-source framework for creating private/permissioned blockchain applications

Exonum Status: Project info: Community: Exonum is an extensible open-source framework for creating blockchain applications. Exonum can be used to crea

Exonum 1.1k Nov 23, 2021
Oso is an open source policy engine for authorization that’s embedded in your application

Oso What is Oso? Oso is an open source policy engine for authorization that’s embedded in your application. It provides a declarative policy language

oso 1.8k Nov 26, 2021
Open-source Autonomy Software in Rust-lang with gRPC for the Roomba series robot vacuum cleaners

CleanIt Open-source Autonomy Software in Rust-lang with gRPC for the Roomba series robot vacuum cleaners Motivation Motivation is to build a complete

Kristoffer Rakstad Solberg 174 Nov 12, 2021
Open Source Rust kernel; Runs WASM and WASI as lightweight containers.

?? etheryal Kernel etheryal kernel is an Open Source capability-based Kernel written in the Rust programming language. The kernel allows implementing

null 19 Sep 5, 2021