Rust Robots-txt Resources

FizzBuzzFS root@8a2db3fc6292:/# cd /mnt/FizzBuzz/ root@8a2db3fc6292:/mnt/FizzBuzz# ls -l total 9007199254740992 -rw-r--r-- 1 501 dialout 9223372036854

🕷️ crawly A lightweight and efficient web crawler in Rust, optimized for concurrent scraping while respecting robots.txt rules. 🚀 Features Concurren

3FakeIM 📱️🚫️🌝️💾️ 3FakeIM is a joke program meant to imitate various fictional characters, and the "[CHARACTER] CALLED ME AT 3:00 AM" clickbait tre

Domo Arigato A simple command line tool which quickly audits the Disallow entries of a site's robots.txt. Disallow entries can be used to stop search

csv, excel toolkit written in Rust rsv is a command line tool to deal with small and big CSV, TXT, EXCEL files (especially 10G). rsv has following fe

Bonsai 盆栽 Rust implementation of Behavior Trees Contents Quick intro to Behavior Trees Concepts Examples Development Guide Kanban Board Honorable Ment

security.txt This library defines a macro, which allows developers to provide easy-to-parse information to security researchers that wish to contact t

Texting Robots Crate texting_robots is a library for parsing robots.txt files. A key design goal of this crate is to have a thorough test suite tested

Learning Rust I started learning the Rust programming language before using GitHub, but increased its usage afterwards. I have found it to be a fast a

Getting Started This is a minimal grep command-line utility built on Rust. It provides searching for plain-text files for lines that match a given str

rbdt 🚨 🚨 🚨 🚨 rbdt is a work in progress, currently being extracted out of another (private) project for the purpose of open sourcing and better so

Webots controllers in Rust Status: experimental This is a reference project that shows how to build controllers for the Webots robot simulator using t

Say No To Poetry Did you grow up using just pip or conda? Are you being forced to use poetry everywhere at work? They try to convince you poetry just

CleanIt Open-source Autonomy Software in Rust-lang with gRPC for the Roomba series robot vacuum cleaners Motivation Motivation is to build a complete

robotstxt A native Rust port of Google's robots.txt parser and matcher C++ library. Native Rust port, no third-part crate dependency Zero unsafe code