Yet another youtube (and more) down loader

Overview

yaydl

Crates.io

yet another youtube (and more) down loader

% yaydl "https://www.youtube.com/watch?v=jNQXAC9IVRw"

How? What? Why?

% yaydl --help

Features

  • Can download videos.
  • Can optionally keep only the audio part of them.
  • Could convert the resulting file to something else (requires the ffmpeg binary).
  • Comes as a single binary (once compiled) - take it everywhere on your thumbdrive, no Python cruft required.

Currently supported sites

  • porndoe.com · vidoza.net · vimeo.com · vivo.sx · voe.sx · watchmdh.to · xhamster.com · youtube.com

There is an easy way to add more supported sites, see below for details.

Non-features

The list of features is deliberately kept short:

  • No output quality choice. yaydl assumes that you have a large hard drive and your internet connection is good enough, or else you would stream, not download.
  • No complex filters. This is a downloading tool.
  • No image file support. Videos only.

How to install

From the source code

Install Rust (e.g. with rustup), then:

using Fossil:

% fossil clone https://code.rosaelefanten.org/yaydl
% cd yaydl
% cargo build --release

using Git:

% git clone https://github.com/dertuxmalwieder/yaydl
% cd yaydl
% cargo build --release

From Cargo

% cargo install yaydl

From your package manager

pkgsrc (with pkg_add):

% pkg_add yaydl

pkgsrc (with pkgin):

% pkgin install yaydl

Other package managers:

  • Nobody has provided any other packages for yaydl yet. You can help!

How to use the web driver (very beta, at your own risk!)

For some video sites, yaydl needs to be able to parse a JavaScript on them. For this, it needs to be able to spawn a headless web browser. It requires Google Chrome, Microsoft Edge or Mozilla Firefox to be installed and running on your system.

  1. Install and run ChromeDriver (if you use Chrome), the typically named Microsoft Edge WebDriver (if you use Edge) or geckodriver (if you use Firefox) for your platform.
  2. Tell yaydl that you have a web driver running: yaydl --webdriver . (The drivers usually run on port 4444 or 9515, please consult their documentation if you are not sure.) Hint: If you need this feature regularly, you can also use the environment variable YAYDL_WEBDRIVER_PORT to set the port number for all further requests.
  3. In theory, it should be possible to use more sites with yaydl now. :-)

How to contribute code

  1. Read and agree to the Code of Conduct Merit.
  2. Implicitly agree to the LICENSE. Nobody reads those. I don't either.
  3. Find out if anyone has filed a GitHub Issue or even sent a Pull Request yet. Act accordingly.
  4. Send me a patch, either via e-mail (yaydl at tuxproject dot de), on the IRC or as a GitHub Pull Request. Note that GitHub only provides a mirror, so you'd double my work if you choose the latter. :-)

If you do that well (and regularly) enough, I'll probably grant you commit access to the upstream Fossil repository.

Add support for new sites

  1. Implement definitions::SiteDefinition as handlers/.rs.
  2. Push the new handler to the inventory: inventory::submit! { &YourSiteHandler as &dyn SiteDefinition }
  3. Add the new module to handlers.rs.
  4. Optionally, add new requirements to Cargo.toml.
  5. Send me a patch, preferably with an example. (I cannot know all sites.)

Minimal example that does nothing

(&'a self, url: &'a str, webdriver_port: u16, onlyaudio: bool) -> Result { // Return the direct download URL of the video (or its audio version) here. // Exception: If is_playlist() is true, return the playlist URL here instead. Ok("".to_string()) } fn find_video_file_extension<'a>(&'a self, url: &'a str, webdriver_port: u16, onlyaudio: bool) -> Result { // Return the designated file extension of the video (or audio) file here. Ok("mp4".to_string()) } fn display_name<'a>(&'a self) -> String { // For cosmetics, this is the display name of this handler. "NoopExample" } fn web_driver_required<'a>(&'a self) -> bool { // Return true here, if the implementation requires a web driver to be running. false } } // Push the site definition to the list of known handlers: inventory::submit! { &NoopExampleHandler as &dyn SiteDefinition }">
// handlers/noop.rs

use anyhow::Result;
use crate::definitions::SiteDefinition;

struct NoopExampleHandler;
impl SiteDefinition for NoopExampleHandler {
    // Parameters sent to the handler by yaydl:
    // - url:            The video page's URL.
    // - webdriver_port: The port that runs the WebDriver client.
    //                   Defaults to 0 if there is no WebDriver configured.
    // - onlyaudio:      true if only the audio part of the video should be
    //                   kept, else false.
    fn can_handle_url<'a>(&'a self, url: &'a str, webdriver_port: u16) -> bool {
        // Return true here if  can be covered by this handler.
        // Note that yaydl will skip all other handlers then.
        true
    }

    fn does_video_exist<'a>(&'a self, url: &'a str, webdriver_port: u16) -> Result<bool> {
        // Return true here, if the video exists.
        Ok(false)
    }

    fn is_playlist<'a>(&'a self, url: &'a str, webdriver_port: u16) -> Result<bool> {
    	// Return true here, if the download link is a playlist.
    	Ok(false)
    }

    fn find_video_title<'a>(&'a self, url: &'a str, webdriver_port: u16) -> Result<String> {
        // Return the video title from  here.
        Ok("".to_string())
    }

    fn find_video_direct_url<'a>(&'a self, url: &'a str, webdriver_port: u16, onlyaudio: bool) -> Result<String> {
        // Return the direct download URL of the video (or its audio version) here.
        // Exception: If is_playlist() is true, return the playlist URL here instead.
        Ok("".to_string())
    }

    fn find_video_file_extension<'a>(&'a self, url: &'a str, webdriver_port: u16, onlyaudio: bool) -> Result<String> {
        // Return the designated file extension of the video (or audio) file here.
        Ok("mp4".to_string())
    }

    fn display_name<'a>(&'a self) -> String {
        // For cosmetics, this is the display name of this handler.
        "NoopExample"
    }

    fn web_driver_required<'a>(&'a self) -> bool {
        // Return true here, if the implementation requires a web driver to be running.
        false
    }
}

// Push the site definition to the list of known handlers:
inventory::submit! {
    &NoopExampleHandler as &dyn SiteDefinition
}

Fix some bugs or add new features

  1. Do so.
  2. Send me a patch.

Donations

Writing this software and keeping it available is eating some of the time which most people would spend with their friends. Naturally, I absolutely accept financial compensation.

Thank you.

Contact

Comments
  • Remove unsafe code?

    Remove unsafe code?

    Rust's most important feature is memory safety and using unsafe removes this feature.

    At my forked repo, I have pushed a branch that removed all unsafe code in spankbang.rs and can be used to remove all unsafe code in the project

    https://github.com/egdv/yaydl/tree/no-unsafe

    opened by egdv 7
  • Youtube regex parse error

    Youtube regex parse error

    I have a simple fix for this error. Provide access to upstream and I can test direct upstream update as you have suggested?

         Running `target\debug\yaydl.exe https://youtu.be/jNQXAC9IVRw`
    Fetching from YouTube.
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Syntax(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    regex parse error:
        (?:v=|\.be\/)(.*?)(&.*)*$
                  ^^
    error: unrecognized escape sequence
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    )', src\handlers\youtube.rs:174:65
    

    or fix it by deleting a char

    let id_regex = Regex::new(r"(?:v=|\.be\/)(.*?)(&.*)*$").unwrap();
                                          ^--- fix: delete this backslash 1 char before e 
    
    
    opened by egdv 6
  • Keep file on error

    Keep file on error

    There is this msg which is same as youtube-dl when a download from music.youtube.com is finished : 'list' is not recognized as an internal or external command, operable program or batch file.

    youtube-dl keeps the file whereas yay-dl doesn't (even though the download is successful).

    Please keep it/ add a flag to allow keeping it :)

    opened by ZuixAlias 5
  • yaydl does not support playlists yet, resulting in weird errors.

    yaydl does not support playlists yet, resulting in weird errors.

    hi,

    i came across your project when i was looking for a command line tool for video download from voe.sx. i tried to download a voe.sx and youtube video, but unfortunately it doesn't seem to work:

    ./yaydl https://www.youtube.com/watch?v=dd-dZJYwz4E
    Fetching from YouTube.
    Error: https://r4---sn-4g5ednss.googlevideo.com/videoplayback?expire=1620331434&ei=SveTYP3ILdWBpATu8qmYBA&ip=5.146.60.98&id=o-AG6g819X4du50aR2CnKMZVNRXOXHFy72i8kg0_DNq4ED&itag=22&source=youtube&requiressl=yes&mh=e6&mm=31%2C26&mn=sn-4g5ednss%2Csn-5hne6nsz&ms=au%2Conr&mv=m&mvi=4&pl=22&initcwndbps=1472500&vprv=1&mime=video%2Fmp4&ns=1vURrODGNxa3BwdwimkfATMF&cnr=14&ratebypass=yes&dur=2316.260&lmt=1620244524736849&mt=1620309655&fvip=4&fexp=24001373%2C24007246&c=WEB&txp=6316222&n=L-CEZ-2WwCP4G5BvM&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cns%2Ccnr%2Cratebypass%2Cdur%2Clmt&sig=AOq0QJ8wRQIhAMyUVWpPMOlChPHKyjgbsy498jNyksonQD1IAu0F2J-7AiAUs1pwUyg7OgBtrrwloAkAgLxwTpDX2pmKRRF8wbTAXw%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgdNrBNwDgmnVEjAakEClTSIzMso59awCNczx0YUC3b5ECIAD2FVwkQOoQOd7-MIbEBjnRHVwee-K9Gje74krBhk2q: 
    Dns Failed: failed to lookup address information: Name or service not known
    
    ./yaydl "https://voe.sx/5mqmzgck2jx3#tabs-482b6qc1r"
    Fetching from Voe.
    The video could not be found. Invalid link?
    
    bug help wanted good first issue 
    opened by D1Ck3n 5
  • Avoid web driver in instances where is not needed

    Avoid web driver in instances where is not needed

    Webdriver is a really useful component to simulate an user browsing the web and running javascript. However it introduces new requirements to the app, like having a compatible web browser installed. It also makes usage and configuration harder for users.

    While the webdriver is useful, it really only shines when we need to evaluate Javascript. If the webpage is server statically we don't really need a webdriver. This is the case of the two sites I cover in the MR. We can replace the webdriver with a simple HTTP request and parse the html. This is way faster and way more convenient

    opened by LuisMayo 3
  • YouTube download fails on Windows, but ok on Linux

    YouTube download fails on Windows, but ok on Linux

    yaydl https://youtu.be/gIqsBUo82-Q -v Fetching from YouTube. The requested video was found. Processing... Title: Why is an 8-Track Tape Called an 8-Track Tape? Starting the download. Error: The filename, directory name, or volume label syntax is incorrect. (os error 123)

    yaydl --version yaydl 0.11.2

    opened by egdv 1
  • yaydl can't download from YouTube playlists yet.

    yaydl can't download from YouTube playlists yet.

    Here's a link I tried downloading:

    https://www.youtube.com/watch?v=F8sZRBdmqc0&list=WL&index=10&t=1040s
    

    yaydl can't parse this. It's a bit of a complicated link because it's part of a playlist and there's a timestamp attached. However, it can parse this link

    https://www.youtube.com/watch?v=F8sZRBdmqc0
    

    I think with a little bit of regex, the extractor could parse these links better. Probably in the future, it would be good to parse it and recognize it's part of a list and give the option to download the whole playlist, but an easy solution for now is to throw everything past the watch ID out and send it to the downloader. The timestamp can probably be thrown out in almost all cases.

    opened by wkrettek 8
Owner
Cthulhux
Creator of BlogC++ and SublimeTodoTxt, knower of things. Hates Git.
Cthulhux
Bevy asset loader that transparently supports loading over http(s)

Bevy Web Asset This is a tiny crate that that wraps the standard bevy asset loader, and adds the ability to load assets from http and https urls. Supp

Johan Klokkhammer Helsing 25 Nov 21, 2022
YAC (Yac is Another Chat) is an example of Chat using WebSocket.

YAC (Yac is Another Chat) is an example of Chat using WebSocket. Because often the example you find in internet are "not so production ready", I would like to try to implement something more robust.

Tommaso Allevi 8 May 24, 2022
Rust library that helps you change the domain of the link to another domain 🦀🔐

Rust library that helps you to change the domain of the link to another domain, the library helps with privacy. It can be used to change the domain of sites that do not care about privacy to another that does.

TheAwiteb 2 Mar 28, 2022
An end-to-end encrypted, anonymous IP-hiding, decentralized, audio/video/file sharing/offline messaging multi-device platform built for both communications and application security and performance.

An end-to-end encrypted, anonymous IP-hiding, decentralized, audio/video/file sharing/offline messaging multi-device platform built for both communications and application security and performance.

null 2 Apr 27, 2022
A fast, stable, efficient, and lightweight intranet penetration, port forwarding tool supports multiple connections, cascading proxy, and transmission encryption

A fast, stable, efficient, and lightweight intranet penetration, port forwarding tool supports multiple connections, cascading proxy, and transmission encryption

editso 1.2k Nov 30, 2022
A proxy implement with http / socks5 in-bound and vmess out-bound, written in Rust and tokio.rs

tokio-vmess an Asynchronous proxy implement with http / socks5 in-bound and vmess out-bound, written in Rust and tokio Run example first, Fill out the

irumeria 7 Oct 3, 2022
A simple web server(and library) to display server stats over HTTP and Websockets/SSE or stream it to other systems.

x-server-stats A simple web server(and library) to display server stats over HTTP and Websockets/SSE or stream it to other systems. x-server(in x-serv

Pratyaksh 11 Oct 17, 2022
The gRPC library for Rust built on C Core library and futures

gRPC-rs gRPC-rs is a Rust wrapper of gRPC Core. gRPC is a high performance, open source universal RPC framework that puts mobile and HTTP/2 first. Sta

TiKV Project 1.6k Nov 25, 2022
🥧 Savoury implementation of the QUIC transport protocol and HTTP/3

quiche is an implementation of the QUIC transport protocol and HTTP/3 as specified by the IETF. It provides a low level API for processing QUIC packet

Cloudflare 6.9k Nov 27, 2022
Imagine your SSH server only listens on an IPv6 address, and where the last 6 digits are changing every 30 seconds as a TOTP code...

tosh Imagine your SSH server only listens on an IPv6 address, and where the last 6 digits are changing every 30 seconds as a TOTP code... Inspired fro

Mark Vainomaa 409 Oct 23, 2022
Library + CLI-Tool to measure the TTFB (time to first byte) of HTTP requests. Additionally, this crate measures the times of DNS lookup, TCP connect and TLS handshake.

TTFB: CLI + Lib to Measure the TTFB of HTTP/1.1 Requests Similar to the network tab in Google Chrome or Mozilla Firefox, this crate helps you find the

Philipp Schuster 22 Nov 7, 2022
A versatile and efficient proxy framework with nice features suitable for various use cases.

A versatile and efficient proxy framework with nice features suitable for various use cases.

null 1.6k Dec 1, 2022
A multiplayer web based roguelike built on Rust and WebRTC

Gorgon A multiplayer web-based roguelike build on Rust and WebRTC. License This project is licensed under either of Apache License, Version 2.0, (LICE

RICHΛRD ΛNΛYΛ 2 Sep 19, 2022
Simple and fast layer 4 proxy in Rust

Fourth 这一波在第四层。 English Fourth是一个Rust实现的Layer 4代理,用于监听指定端口TCP流量,并根据规则转发到指定目标。 功能 监听指定端口代理到本地或远端指定端口 监听指定端口,通过TLS ClientHello消息中的SNI进行分流 安装方法 为了确保获得您架构

Rui Li 17 Nov 8, 2022
Drop-in proxy for Discord gateway connections and sessions allowing for zero downtime deploys

gateway-proxy This is a very hacky project, so it might stop working if Discord changes their API core. This is unlikely, but keep that in mind while

Jens Reidel 38 Nov 10, 2022
A remote shell, TCP tunnel and HTTP proxy for Replit.

Autobahn A remote shell, TCP tunnel and HTTP proxy for Replit. Hybrid SSH/HTTP server for Replit. Based on leon332157/replish. Autobahn runs a WebSock

Patrick Winters 12 Sep 24, 2022
A DHCP parser and encoder for DHCPv4/DHCPv6

dhcproto A DHCP parser and encoder for DHCPv4/DHCPv6. dhcproto aims to be a functionally complete DHCP implementation. Many common option types are im

BlueCat Engineering 74 Dec 2, 2022
The Graph is a protocol for building decentralized applications (dApps) quickly on Ethereum and IPFS using GraphQL.

Graph Node The Graph is a protocol for building decentralized applications (dApps) quickly on Ethereum and IPFS using GraphQL. Graph Node is an open s

Mindy.wang 2 Jun 18, 2022