Remote-Archive is a utility for exploring remote archive files without downloading the entire contents of the archive.

Overview

[WIP] REMOTE-ARCHIVE

Remote-Archive is a utility for exploring remote archive files without downloading the entire contents of the archive.

The idea behind it is to use the Range HTTP header to read the entries' metadata and skip the content of the entries in the archive.

  • Option for using custom offsets
  • Specificing the thread count
  • Option for downloading a specific file
  • File types
    • ZIP
    • TAR (USTAR)
    • RAR
    • 7z

an example usage (exploring the zip file from the middle of it (450GB)):

$ cargo run -- --url 'http://foo.bar/a.zip'  --offset=255194890075 --proxy socks5://127.0.0.1:9050

[src/remote_file.rs:54] &range = 255194890075..255194894171
[src/remote_file.rs:54] &range = 255194890078..255194890082
[src/remote_file.rs:54] &range = 255194890082..255194890100
[src/remote_file.rs:54] &range = 255194890100..255194890100
[src/main.rs:61] &entry = Ok(
    Entry {
        entry: EndOfCentralDir(
            EndOfCentralDir {
                disk_of_end_of_central_dir: 0,
                disk_of_central_dir: 0,
                num_central_dir_entries_on_disk: 1,
                num_central_dir_entries_total: 1,
                len_central_dir: 88,
                ofs_central_dir: 11507406,
                len_comment: 0,
                comment: "",
            },
        ),
        range: 255194890078..255194890100,
    },
)
[src/remote_file.rs:54] &range = 255194890100..255194890104
[src/remote_file.rs:54] &range = 255194890104..255194890130
[src/remote_file.rs:54] &range = 255194890130..255194890247
[src/main.rs:61] &entry = Ok(
    Entry {
        entry: LocalFile(
            LocalFile {
                header: LocalFileHeader {
                    version: 20,
                    flags: 2048,
                    compression_method: Deflated,
                    file_mod_time: 20964,
                    file_mod_date: 21694,
                    crc32: 2242091140,
                    len_body_compressed: 641317,
                    len_body_uncompressed: 664719,
                    len_filename: 117,
                    len_extra: 28,
                    filename: "Y.xlsx",
                },
                body: None,
            },
        ),
        range: 255194890100..255195531592,
    },
)
You might also like...
KAIVM is a multiplatform Command Line Interface (CLI) designed to simplify the process of downloading, managing, configuring, and running different versions of Shinkai Node
KAIVM is a multiplatform Command Line Interface (CLI) designed to simplify the process of downloading, managing, configuring, and running different versions of Shinkai Node

KAIVM - Shinkai Version Manager KAIVM is a multiplatform Command Line Interface (CLI) designed to simplify the process of downloading, managing, confi

A simple tool for extracting files from iOS backup archive.
A simple tool for extracting files from iOS backup archive.

iBackupExtractor A simple tool for extracting files from iOS backup archive. iOS backup files are not stored with their original directory layouts. Re

Tool to allow parsing large JSON files without laoding into memory
Tool to allow parsing large JSON files without laoding into memory

Tool to allow parsing large JSON files without laoding into memory. Developed in Rust with adapters in other programming langauges for easy adoption

mdBook is a utility to create modern online books from Markdown files.

Create book from markdown files. Like Gitbook but implemented in Rust

fixred is a command line utility to fix outdated links in files with redirect URLs.
fixred is a command line utility to fix outdated links in files with redirect URLs.

fixred fixred is a command line utility to fix outdated links in files with redirect URLs. Installation fixred is installed via cargo package manager.

This utility traverses through your filesystem looking for open-source dependencies that are seeking donations by parsing README.md and FUNDING.yml files
This utility traverses through your filesystem looking for open-source dependencies that are seeking donations by parsing README.md and FUNDING.yml files

This utility traverses through your filesystem looking for open-source dependencies that are seeking donations by parsing README.md and FUNDING.yml files

A utility written in Rust for dumping binary information out of Mach-O files inspired by objdump

Mach-O Dump (macho-dump) An objdump like tool for exploring and manipulating Mach-O files. Note: This project is in an early stage and most of the fea

 This CLI utility facilitates effortless manipulation and exploration of TOML, YAML, JSON and RON files.
This CLI utility facilitates effortless manipulation and exploration of TOML, YAML, JSON and RON files.

📁💻🔍🔧 This CLI utility facilitates effortless manipulation and exploration of TOML, YAML, JSON and RON files.

A utility to do detailed analysis of gzip files.

GZInspector A robust command-line tool for inspecting and analyzing GZIP/ZLIB compressed files. GZInspector provides detailed information about compre

Owner
2nd acc
null
A filesystem driver that allows you to view your Blackboard course contents as if they were normal files and folders on your system!

BlackboardFS Blackboard: noun A website so bad that it might as well be a network drive. BlackboardFS is a filesystem driver that allows you to view y

null 22 Sep 4, 2023
apkeep - A command-line tool for downloading APK files from various sources

apkeep - A command-line tool for downloading APK files from various sources Installation Precompiled binaries for apkeep on various platforms can be d

Electronic Frontier Foundation 561 Dec 29, 2022
BoilerFiles is a CLI for downloading your boilerplate files from a public GitHub template repo.

BoilerFiles Available on crates.io. BoilerFiles is a CLI for downloading your boilerplate files from a public GitHub template repo. Features Are you t

Jan Müller 4 Oct 1, 2022
An interactive shell environment for exploring the p2panda protocol

An interactive shell environment for exploring the p2panda protocol. Uses a mock node and clients to simulate network logic.

null 4 Dec 12, 2021
✨ A simple and cute tool for exploring Wikipedia from the terminal ✨

Wikipe ✨ Wikipe ✨ is a tool to search Wikipedia from the command-line. Given a search query, it returns a list of matching Wikipedia articles, which c

Harvey 4 Dec 20, 2022
A TUI for exploring crates.io using Ratatui

crates-tui crates-tui is a simple terminal user interface explorer for crates.io based on Ratatui. crates-tui.mov It supports features like: copy carg

Ratatui 28 Mar 11, 2024
Failed experiment in downloading random cat image, turning it into ascii and displaying it in Neovim.

cat.nvim Failed experiment in downloading random cat image, turning it into ascii and displaying it in Neovim. Failed as I realized far too late, that

James Vero 4 Aug 5, 2022
Desktop app for reading and downloading manga. With clean distraction-free design and no clutter

Tonbun Tonbun is a desktop app for reading and downloading manga. With clean distraction-free design and no clutter. Build with Rust, Tauri, Vue.js, a

null 23 Nov 30, 2022
A multi-threaded Twitch chat archiving and downloading tool.

Twitch Chat Downloader ??️ tcd is a multi-threaded Twitch Chat Downloader built in Rust ?? . Usage: tcd [OPTIONS] <--channel <CHANNEL>|--video <VIDEO>

Matthew Polak 6 Dec 19, 2022
Automatically replace Discord CDN links with local file links, by downloading the images to the public folder

Auto Undiscord Did you hear that Discord will be blocking external websites from using images hosted on their servers? Did you host every image on you

Dot32 5 Nov 17, 2023