Twitch Chat Downloader
ποΈ
tcd is a multi-threaded Twitch Chat Downloader built in Rust
Usage: tcd [OPTIONS] <--channel <CHANNEL>|--video <VIDEO>>
Options:
-c, --channel <CHANNEL> The channel(s) to download
-i, --client-id <CLIENT_ID> The Twitch client ID to use in the request headers
-f, --format <FORMAT> Used with --output or --stdout [default: csv] [possible values: json, csv]
-l, --limit <LIMIT> Downloads the first n videos from each channel
-e, --live If specified, polls for new videos every `poll` seconds
-o, --output <OUTPUT> If specified, pipes data to the file
-p, --postgres [<POSTGRES>] The PostgreSQL connection string [default: DATABASE_URL env]
-q, --quiet Whether to print download progress
-s, --stdout If specified, pipes data to stdout
-t, --threads <THREADS> The number of threads to use [default: 10]
-v, --video <VIDEO> The video ids to download the chat for
-w, --wait <WAIT> The number of minutes to wait between polls (`live` only) [default: 30]
-h, --help Print help information
-V, --version Print version information
Pipe the chat messages of the first 5 videos of Atrioc, Linkus7 and Aspecticor to the file hitman.csv
tcd --channel atrioc --channel linkus7 --channel aspecticor --limit 5 --output hitman.csv
Save the chat from the videos with id 1649326959 and 1648474855 to the connected PostgreSQL database.
tcd --video 1649326959 --video 1648474855 --postgres
Building from source
# build the binary
cargo build --release
# execute the binary
target/release/tcd --help
Generating datasets
Some pre-made dataset scripts are located in the queries directory. You can run these with cargo run -p queries --example <name>.
Using pre-made datasets
Pre-made datasets can be downloaded from the Mediafire folder.
Piping data to a database
tcd supports saving data directly to a PostgreSQL database. First, apply the Prisma schema with the following commands:
# apply schema.prisma to the database
# note: this WILL wipe all database content
cargo prisma migrate dev --name init
# generate the Prisma client
cargo prisma generate
Or execute the migration.sql SQL statements against your database. Then, set the DATABASE_URL environment variable (a .env file works too), or supply the connection URL with --postgres <url>.
Output format
Data piped to a file or stdout will be in the following format:
--format csv
channel_id,video_id,comment_id,commenter_id,created_at,text
23211159,1642642569,3f445ae2-2f6e-4256-b367-df8132454786,157032028,"2022-11-03 21:25:22.754 +00:00","poggies"
--format json
[
{
"channelId": "i64",
"videoId": "i64",
"commentId": "string",
"commenterId": "i64",
"createdAt": "string",
"text": "string"
},
{
"channelId": 23211159,
"videoId": 1642642569,
"commentId": "3f445ae2-2f6e-4256-b367-df8132454786",
"commenterId": 157032028,
"createdAt": "2022-11-03 21:25:22.754 +00:00",
"text": "poggies"
}
]
4 May 20, 2022
9 Dec 23, 2022
4 Aug 17, 2023
4 Oct 1, 2022
4 Nov 7, 2022
5 Nov 17, 2023
0 Dec 5, 2021
38 Apr 25, 2023
13 Jan 25, 2023
172 Dec 30, 2022
59 Dec 10, 2022
6 May 3, 2024
561 Dec 29, 2022
23 Nov 30, 2022
4 Aug 5, 2022
7 May 1, 2024
20 Nov 11, 2022