A simple, cross-platform GUI automation module for Rust.

Overview

Docs Crates.io Travis Build Status Appveyor Build Status

AutoPilot

AutoPilot is a Rust port of the Python C extension AutoPy, a simple, cross-platform GUI automation library for Python. For more information, see the README on that repo.

Currently supported on macOS, Windows, and X11 with the XTest extension.

Examples

The following will move the mouse across the screen as a sine wave:

extern crate autopilot;
extern crate rand;
use rand::Rng;

const TWO_PI: f64 = std::f64::consts::PI * 2.0;
fn sine_mouse_wave() {
    let screen_size = autopilot::screen::size();
    let scoped_height = screen_size.height / 2.0 - 10.0; // Stay in screen bounds.
    let mut rng = rand::thread_rng();
    for x in 0..screen_size.width as u64 {
        let y = (scoped_height * ((TWO_PI * x as f64) / screen_size.width).sin() + 
                 scoped_height).round();
        let duration: u64 = rng.gen_range(1, 3);
        autopilot::mouse::move_to(autopilot::geometry::Point::new(
            x as f64,
            y as f64
        )).expect("Unable to move mouse");
        std::thread::sleep(std::time::Duration::from_millis(duration));
    }
}

This will enter the keys from the string "Hello, world!" and then prompt an alert with the same text:

extern crate autopilot;

fn main() {
    autopilot::key::type_string("Hello, world!", &[], 200., 0.);
    let _ = autopilot::alert::alert("Hello, world!", None, None, None);
}

Contributing

If you are interested in this project, please consider contributing. Here are a few ways you can help:

License

This project is licensed under either the Apache-2.0 or MIT license, at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Comments
  • Relicense under dual MIT-or-Apache-2.0

    Relicense under dual MIT-or-Apache-2.0

    I'd like to relicense this project from Apache 2.0 to an MIT-or-Apache dual license for consistency with the Rust community, and to allow GPLv2 compatibility, which I was previously unaware was disallowed by Apache.

    For more information, see the explainer text used when the community switched over, or one of the many automatically generated issues on popular projects.

    To do this, it’s necessary to get explicit approval from each contributor (aside from small typo fixes, etc., as described in the above links). I’ve cc’d everyone here and will be checking off boxes as people show up.

    • [x] @msanders
    • [x] @greizgh
    • [x] @rlewismd
    • [ ] ~~@zrnsm~~ (n/a)
    • [x] @waywardmonkeys
    • [x] @pheki
    • [x] @murlakatamenka
    • [x] @VojtechStep
    • [x] @Ivshti

    To agree to relicensing, please comment with:

    I license past and future contributions under the dual MIT-or-Apache-2.0 license, allowing licensees to choose either at their option.

    (Feel free to unsubscribe from this issue after leaving your comment.)

    Once everyone has commented, I'll open a PR making the change. Will be doing this for the sister projects here as well.

    Thanks your help!

    opened by msanders 11
  • [Question] Easy way to do same automation in multiple times?

    [Question] Easy way to do same automation in multiple times?

    Is there any easy way to do gui automation easily for multiple time for same app in parallel or concurrently?

    Currently what is do is create multiple users etc. That seems abit tedious right? If there is better way let me know :)

    Thanks.

    question 
    opened by shirshak55 6
  • Cannot compile on raspbian stretct (raspberry pi ARM)

    Cannot compile on raspbian stretct (raspberry pi ARM)

    Running `rustc --crate-name autopilot /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/autopilot-0.2.0/src/lib.rs --color always --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=63661520e1f9d40b -C extra-filename=-63661520e1f9d40b --out-dir /home/pi/autopy/target/release/deps -L dependency=/home/pi/autopy/target/release/deps --extern image=/home/pi/autopy/target/release/deps/libimage-bf2a33ff0b6e5287.rlib --extern libc=/home/pi/autopy/target/release/deps/liblibc-92d8ac3aaa24033c.rlib --extern rand=/home/pi/autopy/target/release/deps/librand-d4987583c82169cf.rlib --extern scopeguard=/home/pi/autopy/target/release/deps/libscopeguard-ecb84c47962aa564.rlib --extern x11=/home/pi/autopy/target/release/deps/libx11-67a32f63197588b9.rlib --cap-lints allow -l X11 -l Xtst`
    error[E0308]: mismatched types
       --> /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/autopilot-0.2.0/src/key.rs:190:40
        |
    190 |             x11::xlib::XStringToKeysym(character.encode_utf8(&mut buf).as_ptr() as *const i8)
        |                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected u8, found i8
        |
        = note: expected type `*const u8`
                   found type `*const i8`
    
    error: aborting due to previous error
    
    For more information about this error, try `rustc --explain E0308`.
    error: Could not compile `autopilot`.
    
    
    bug 
    opened by plamenh 3
  • Run clippy and tests on travis

    Run clippy and tests on travis

    Now that you have a CI, let's run tests on it! It also runs clippy, which will fail on warning. This behavior can be tuned to only fail on error: what do you prefer?

    opened by greizgh 3
  • Q: AutoPilot and AutoPy relations

    Q: AutoPilot and AutoPy relations

    Hi, I'm confused, in README.md you stated that:

    AutoPilot is a Rust port of the Python C extension AutoPy, a simple, cross-platform GUI automation library for Python. For more information, see the README on that repo.

    But AutoPy, which is also your project, just python wrapper around AutoPilot.

    So is it true, that you wrote autopilot and than autopy? Or you wrote autopy on C++, than ported it to rust than used it in autopy?

    PS: They both have first commits in Oct 2017.

    question 
    opened by banderlog 2
  • Convenient way of typing strings with symbols like underscore?

    Convenient way of typing strings with symbols like underscore?

    Hi, is there a convenient way of typing strings with symbols like underscore?

    Reasoning

    use autopylot::key;
    
    key::type_string("test_subject", &[], 0., 0.);
    

    will type test-subject instead of desired test_subject.

    Current workaround

    use autopylot::key;
    
    key::type_string("test", &[], 0., 0.);
    key::type_string("-", &[key::Flag::Shift], 0., 0.); // or "_"
    key::type_string("test", &[], 0., 0.);
    

    isn't very convenient but does the job done.

    Probably I'm missing something? Using Linux here.

    P.S. Thanks for the crate <3

    bug help wanted 
    opened by murlakatamenka 2
  • Fix Linux Arrow Keycodes

    Fix Linux Arrow Keycodes

    Using the python binding I ran into the same issue that was logged here: https://github.com/autopilot-rs/autopy/issues/10

    I think the issue is that codes for actual arrow glyphs are being used rather than codes for arrow keypresses (e.g. U+2193 ↓).

    opened by zrnsm 2
  • Fix OSX / Linux builds

    Fix OSX / Linux builds

    Looks like OSX and Linux were both not fully updated with the Insert key

    Build results (after applying the github actions patch in my other pull request):

    https://github.com/shish/autopilot-rs/runs/2866406979?check_suite_focus=true

    opened by shish 1
  • Fix target os detection for crosscompilation

    Fix target os detection for crosscompilation

    cfg attribute provides target_os for build.rs itself, not for the main project. For example, if we are crosscompiling from linux to windows it will print "cargo:rustc-flags=-l X11 -l Xtst" and fail with message 'cannot find -lX11'.

    opened by alianse777 1
  • key::tap triple delay when using flags

    key::tap triple delay when using flags

    When using autopilot::key::tap(&Character('c'), &[autopilot::key::Flag::Alt], 25); you'll get a total delay of 75ms, 25 for the "tap" and an extra 25 for each use of the modifier (down and up). Is this intentional behaviour?

    I would've thought there would just be a 25ms delay between the use of c + modifier down and c + modifier up.

    enhancement 
    opened by Zeldern 1
  • README: fix example to compile with rand 0.6.x

    README: fix example to compile with rand 0.6.x

    this mut is needed to compile this example on with Rust 2018 and rand 0.6.x; not sure if it wasn't needed in a previous version of rnand or the example was always broken, but whatever the case is, it will compile after this change

    opened by Ivshti 1
  • thread 'main' panicked at 'cannot transmute_copy if U is larger than T', /rustc/897e37553bba8b42751c67658967889d11ecd120\library\core\src\mem\mod.rs:1057:5

    thread 'main' panicked at 'cannot transmute_copy if U is larger than T', /rustc/897e37553bba8b42751c67658967889d11ecd120\library\core\src\mem\mod.rs:1057:5

    thread 'main' panicked at 'cannot transmute_copy if U is larger than T', /rustc/897e37553bba8b42751c67658967889d11ecd120\library\core\src\mem\mod.rs:1057:5
    stack backtrace:
       0: std::panicking::begin_panic_handler
                 at /rustc/897e37553bba8b42751c67658967889d11ecd120/library\std\src\panicking.rs:584
       1: core::panicking::panic_fmt
                 at /rustc/897e37553bba8b42751c67658967889d11ecd120/library\core\src\panicking.rs:142
       2: core::mem::transmute_copy<winapi::um::winuser::KEYBDINPUT,winapi::um::winuser::INPUT_u>
                 at /rustc/897e37553bba8b42751c67658967889d11ecd120\library\core\src\mem\mod.rs:1057
       3: autopilot::key::system_toggle<autopilot::key::Character>
                 at C:\Users\root\.cargo\registry\src\mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd\autopilot-0.4.0\src\key.rs:486
       4: autopilot::key::toggle<autopilot::key::Character>
                 at C:\Users\root\.cargo\registry\src\mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd\autopilot-0.4.0\src\key.rs:164
       5: autopilot::key::tap<autopilot::key::Character>
                 at C:\Users\root\.cargo\registry\src\mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd\autopilot-0.4.0\src\key.rs:146
       6: demo::main
                 at .\src\demo.rs:23
       7: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
                 at /rustc/897e37553bba8b42751c67658967889d11ecd120\library\core\src\ops\function.rs:248
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    error: process didn't exit successfully: `target\debug\demo.exe` (exit code: 101)
    
    Process finished with exit code 101
    
    opened by Ykong1337 1
  • Shrink unsafe blocks

    Shrink unsafe blocks

    In this function you use the unsafe keyword for almost the entrie function body. However, I found that only 1 function are real unsafe operations (see the list below).

    We need to mark unsafe operations more precisely using unsafe keyword. Keeping unsafe blocks small can bring many benefits. For example, when mistakes happen, we can locate any errors related to memory safety within an unsafe block. This is the balance between Safe and Unsafe Rust. The separation is designed to make using Safe Rust as ergonomic as possible, but requires extra effort and care when writing Unsafe Rust. Real unsafe operation list:

    1. the mouse_event() function(unsafe function)

    @msanders Hope this PR can help you. Best regards. References https://doc.rust-lang.org/nomicon/safe-unsafe-meaning.html https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html

    opened by peamaeq 0
  • Strange spurious key presses when using key::toggle

    Strange spurious key presses when using key::toggle

    Run the following code (warning it causes an infinite loop):

    use std::thread::sleep;
    use std::time::Duration;
    use autopilot::key::Character;
    
    loop {
        autopilot::key::toggle(&Character('s'), true, &[], 0);
        sleep(Duration::from_millis(100));
        autopilot::key::toggle(&Character('s'), false, &[], 0);
    
        sleep(Duration::from_millis(100));
    }
    

    This code will infinitely press the s key. While it is pressing the s key, repeatedly tap the y key on your keyboard.

    You will see a pattern like this:

    sssssssy;y;y;syy;y;y;y;y;yssy;y;y;sysy;y;y;y;y;yy;y;y;y;y;y;sssssssssssssysy;y;y;yy;y;y;y;yy;y;;sss
    

    Even though the code is not pressing the ; key, and you did not press the ; key, it still causes spurious ; presses.

    I don't know if this is a bug with autopilot, XTest, or perhaps my specific hardware. It's a very strange bug.

    opened by Pauan 1
  • Build failure on mac: Insert key missing

    Build failure on mac: Insert key missing

    cargo build fails on Mac (macOS Mojave 10.14.6, so I am lacking behind):

    error[E0004]: non-exhaustive patterns: `Insert` not covered
       --> src/key.rs:323:15
        |
    43  | / pub enum KeyCode {
    44  | |     F1,
    45  | |     F2,
    46  | |     F3,
     ...  |
    79  | |     Insert,
        | |     ------ not covered
    ...   |
    107 | |     NumEnter,
    108 | | }
        | |_- `key::KeyCode` defined here
     ...
    323 |           match code {
        |                 ^^^^ pattern `Insert` not covered
        |
           = help: ensure that all possible cases are being handled, possibly by adding wildcards or more match arms
           = note: the matched value is of type `key::KeyCode`
    
    error: aborting due to previous error
    
    For more information about this error, try `rustc --explain E0004`.
    error: could not compile `autopilot`
    
    To learn more, run the command again with --verbose.
    warning: build failed, waiting for other jobs to finish...
    error: build failed
    

    Mac keyboards do not have Insert key. This maybe explains why my core_graphics package does not include the definition of that key so I am not able to add it. When I just brutally removed the offending "Insert" from enum KeyCode (src/key.rs:82) then the build succeeded on my Mac. This obviously is not the way to fix the issue, so I am not submitting a pull request... ;-)

    opened by mitizhi 0
Owner
null
Simple and portable (but not inflexible) GUI library in C that uses the native GUI technologies of each platform it supports.

libui: a portable GUI library for C This README is being written. Status It has come to my attention that I have not been particularly clear about how

Pietro Gagliardi 10.4k Dec 31, 2022
A cross-platform GUI library for Rust, inspired by Elm

Iced A cross-platform GUI library for Rust focused on simplicity and type-safety. Inspired by Elm. Features Simple, easy-to-use, batteries-included AP

Héctor Ramón 17.5k Jan 2, 2023
Truly cross platform, truly native. multiple backend GUI for rust

WIP: Sauron-native a rust UI library that conquers all platforms ranging from desktop to mobile devices. An attempt to create a truly native, truly cr

Jovansonlee Cesar 627 Jan 5, 2023
A cross-platform GUI library for Rust focused on simplicity and type-safety

A cross-platform GUI library for Rust, inspired by Elm

Héctor Ramón 17.5k Jan 8, 2023
Cross-platform GUI toolkit written in Rust

Tuix is a cross-platform GUI toolkit written in Rust. The driving principle behind tuix is to be a self-contained, small-as-possible, but still fast,

George Atkinson 166 Dec 13, 2022
A cross-platform GUI toolkit in Rust

NXUI - Native X UI A cross-platform GUI toolkit in Rust NXUI is a GUI toolkit that calls OS native APIs as much as possible to achieve fast operation.

らて 11 Jun 3, 2022
A cross-platform GUI library for Rust, inspired by Elm

Iced A cross-platform GUI library for Rust focused on simplicity and type-safety. Inspired by Elm. Features Simple, easy-to-use, batteries-included AP

null 17.5k Dec 28, 2022
A single-header ANSI C immediate mode cross-platform GUI library

Nuklear This is a minimal-state, immediate-mode graphical user interface toolkit written in ANSI C and licensed under public domain. It was designed a

Immediate Mode UIs, Nuklear, etc. 6.9k Jan 8, 2023
A lightweight cross-platform system-monitoring fltk gui application based on sysinfo

Sysinfo-gui A lightweight cross-platform system-monitoring fltk gui application based on sysinfo. The UI design is inspired by stacer. The svg icons a

Mohammed Alyousef 22 Dec 31, 2022
A simple news reading GUI app built in Rust

Headlines [WIP] A native GUI app built with Rust using egui. Uses newsapi.org as the source to fetch news articles. This is a WIP and the current stat

creativcoder 89 Dec 29, 2022
A simple GUI version of the pH calibration tool written in egui, based on the eframe template.

caliphui A simple GUI version of the pH calibration tool written in egui, based on the eframe template. Usage Native binaries are provided under relea

Peter Dunne 0 Dec 29, 2021
Rust bindings to the minimalist, native, cross-platform UI toolkit `libui`

Improved User Interface A cross-platform UI toolkit for Rust based on libui iui: ui-sys: iui is a simple (about 4 kLOC of Rust), small (about 800kb, i

Rust Native UI Group 865 Dec 27, 2022
Cross-platform native Rust menu library

A cross-platform Rust library for managing the native operating system menus.

Mads Marquart 16 Jan 6, 2023
A Rust binding of the wxWidgets cross platform toolkit.

wxRust master: / mac(0.10): This is a Rust binding for the wxWidgets cross platform toolkit. API wxRust API documentation How it works The wxRust libr

KENZ 129 Jan 4, 2023
Alerion is a cross-platform Rust rewrite of Pterodactyl Wings

alerion ?? A complete rewrite of pterodactyl wings. Caution Here be dragons. This project is still a huge work in progress and is not ready for produc

Pyro 5 Apr 15, 2024
A cross-platform Mod Manager for RimWorld intended to work with macOS, linux and Windows

TODOs are available here. Discussions, PRs and Issues are open for anyone who is willing to contribute. rrm Inspired by Spoons rmm. This is a cross-pl

Alejandro Osornio 7 Sep 5, 2022
SDL2 module for Deno

Deno SDL2 Cross platform and stable bindings to SDL2. Have fun! Features Bindings to Video, Graphics, Font and Mixer subsystems. (Uses rodio instead o

Divy Srivastava 102 Jan 4, 2023
An easy-to-use, 2D GUI library written entirely in Rust.

Conrod An easy-to-use, 2D GUI library written entirely in Rust. Guide What is Conrod? A Brief Summary Screenshots and Videos Feature Overview Availabl

PistonDevelopers 3.3k Jan 1, 2023
Rust bindings for the FLTK GUI library.

fltk-rs Rust bindings for the FLTK Graphical User Interface library. The FLTK crate is a crossplatform lightweight gui library which can be statically

Mohammed Alyousef 1.1k Jan 9, 2023