Rust bindings for the Python interpreter

Overview

PyO3

Actions Status codecov crates.io minimum rustc 1.41 Join the dev chat

Rust bindings for Python. This includes running and interacting with Python code from a Rust binary, as well as writing native Python modules.

A comparison with rust-cpython can be found in the guide.

Usage

PyO3 supports Python 3.6 and up. The minimum required Rust version is 1.41.

Building with PyPy is also possible (via cpyext) for Python 3.6, targeted PyPy version is 7.3+. Please refer to the pypy section in the guide.

You can either write a native Python module in Rust, or use Python from a Rust binary.

However, on some OSs, you need some additional packages. E.g. if you are on Ubuntu 18.04, please run

sudo apt install python3-dev python-dev

Using Rust from Python

PyO3 can be used to generate a native Python module.

Cargo.toml

[package]
name = "string-sum"
version = "0.1.0"
edition = "2018"

[lib]
name = "string_sum"
# "cdylib" is necessary to produce a shared library for Python to import from.
#
# Downstream Rust code (including code in `bin/`, `examples/`, and `tests/`) will not be able
# to `use string_sum;` unless the "rlib" or "lib" crate type is also included, e.g.:
# crate-type = ["cdylib", "rlib"]
crate-type = ["cdylib"]

[dependencies.pyo3]
version = "0.13.2"
features = ["extension-module"]

src/lib.rs

use pyo3::prelude::*;
use pyo3::wrap_pyfunction;

/// Formats the sum of two numbers as string.
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
    Ok((a + b).to_string())
}

/// A Python module implemented in Rust.
#[pymodule]
fn string_sum(py: Python, m: &PyModule) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;

    Ok(())
}

On Windows and Linux, you can build normally with cargo build --release. On macOS, you need to set additional linker arguments. One option is to compile with cargo rustc --release -- -C link-arg=-undefined -C link-arg=dynamic_lookup, the other is to create a .cargo/config with the following content:

[target.x86_64-apple-darwin]
rustflags = [
  "-C", "link-arg=-undefined",
  "-C", "link-arg=dynamic_lookup",
]

[target.aarch64-apple-darwin]
rustflags = [
  "-C", "link-arg=-undefined",
  "-C", "link-arg=dynamic_lookup",
]

While developing, you can symlink (or copy) and rename the shared library from the target folder: On MacOS, rename libstring_sum.dylib to string_sum.so, on Windows libstring_sum.dll to string_sum.pyd, and on Linux libstring_sum.so to string_sum.so. Then open a Python shell in the same folder and you'll be able to import string_sum.

To build, test and publish your crate as a Python module, you can use maturin or setuptools-rust. You can find an example for setuptools-rust in examples/word-count, while maturin should work on your crate without any configuration.

Using Python from Rust

If you want your Rust application to create a Python interpreter internally and use it to run Python code, add pyo3 to your Cargo.toml like this:

[dependencies.pyo3]
version = "0.13.2"
features = ["auto-initialize"]

Example program displaying the value of sys.version and the current user name:

use pyo3::prelude::*;
use pyo3::types::IntoPyDict;

fn main() -> Result<(), ()> {
    Python::with_gil(|py| {
        main_(py).map_err(|e| {
          // We can't display Python exceptions via std::fmt::Display,
          // so print the error here manually.
          e.print_and_set_sys_last_vars(py);
        })
    })
}

fn main_(py: Python) -> PyResult<()> {
    let sys = py.import("sys")?;
    let version: String = sys.get("version")?.extract()?;
    let locals = [("os", py.import("os")?)].into_py_dict(py);
    let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'";
    let user: String = py.eval(code, None, Some(&locals))?.extract()?;
    println!("Hello {}, I'm Python {}", user, version);
    Ok(())
}

Our guide has a section with lots of examples about this topic.

Tools and libraries

  • maturin Zero configuration build tool for Rust-made Python extensions.
  • setuptools-rust Setuptools plugin for Rust support.
  • pyo3-built Simple macro to expose metadata obtained with the built crate as a PyDict
  • rust-numpy Rust binding of NumPy C-API
  • dict-derive Derive FromPyObject to automatically transform Python dicts into Rust structs
  • pyo3-log Bridge from Rust to Python logging
  • pythonize Serde serializer for converting Rust objects to JSON-compatible Python objects
  • pyo3-asyncio Utilities for working with Python's Asyncio library and async functions

Examples

  • hyperjson A hyper-fast Python module for reading/writing JSON data using Rust's serde-json
  • html-py-ever Using html5ever through kuchiki to speed up html parsing and css-selecting.
  • point-process High level API for pointprocesses as a Python library
  • autopy A simple, cross-platform GUI automation library for Python and Rust.
    • Contains an example of building wheels on TravisCI and appveyor using cibuildwheel
  • orjson Fast Python JSON library
  • inline-python Inline Python code directly in your Rust code
  • Rogue-Gym Customizable rogue-like game for AI experiments
    • Contains an example of building wheels on Azure Pipelines
  • fastuuid Python bindings to Rust's UUID library
  • wasmer-python Python library to run WebAssembly binaries
  • mocpy Astronomical Python library offering data structures for describing any arbitrary coverage regions on the unit sphere
  • tokenizers Python bindings to the Hugging Face tokenizers (NLP) written in Rust
  • pyre Fast Python HTTP server written in Rust
  • jsonschema-rs Fast JSON Schema validation library
  • css-inline CSS inlining for Python implemented in Rust
  • cryptography Python cryptography library with some functionality in Rust

License

PyO3 is licensed under the Apache-2.0 license. Python is licensed under the Python License.

Comments
  • Added Rust initialisation of Python-allocated bytes

    Added Rust initialisation of Python-allocated bytes

    Adds new_with<F: Fn(&mut [u8])>(py: Python<'_>, len: usize, init: F) -> &PyBytes as suggested by @davidhewitt in #617. This allows initialising new PyBytes in Rust like such:

    let py_bytes = PyBytes::new_with(py, 10, |b: &mut [u8]| {
        b.copy_from_slice(b"Hello Rust");
    });
    

    Currently, it follows the semantics of PyBytes::new in that it panics if a memory error occurs in Python. Maybe my implementation of that could be improved.

    opened by MomoLangenstein 53
  • FromPyObject derivation for structs and enums

    FromPyObject derivation for structs and enums

    This is a draft addressing a bit of both topics in #301 and #1055.

    I have not written proc-macro code before, so this might be quite ugly. This is mostly proof-of-concept, but I think it'd be a great quality-of-life improvement if something like this was available.

    This produces the following behaviour:

    >>> union.foo(["Test"])
    ["Test"]
    >>> union.foo("Test")
    Test
    >>> union.foo(1)
    1
    >>> union.foo(None)
    TypeError: Can't convert NoneType to Union[Str, Int, StringList]
    

    This implementation covers both the cheap downcast variant and extraction of proper Rust types. Although I'm not sure whether relying on the variant holding a reference is the proper heuristic to apply here.


    Todos are marked in the code, a proper implementation should in my opinion:

    • make the reported names of the variants in the error configurable through an attribute on the variant, i.e.:
      #[union]
      pub enum MyUnion {
          #[rename(str)]
          SomeVariant(String)
      } 
      
    • support named fields & variants with multiple fields
    opened by sebpuetz 53
  • Add catch_unwind! macro to prevent panics crossing ffi boundaries

    Add catch_unwind! macro to prevent panics crossing ffi boundaries

    Fixes #492

    This is a first attempt at updating all the wrapping code to use catch_unwind to prevent unsoundness of panics inside Rust callback functions.

    I implemented this using a pyo3::catch_unwind macro, which takes a Python token and a body of code which should evaluate to a PyResult.

    I ran into complications around lifetimes and the callback converters so I ended up simplifying a lot of the callback conversion code. I think a lot of those macros are now in slightly better shape but there's probably more refactoring that could be done.

    Please give this a thorough review, I'm prepared to rework this as many times as needed to get it right.

    TODO:

    • [X] Custom Exception type from BaseException
      • [ ] Mechanism for users to import this Exception (will do later after #805)
      • [x] Resume unwind when pyO3 gets this Exception in a PyErr
    • [ ] Panic section for the book
    help-wanted 
    opened by davidhewitt 53
  • Get rid of specialization

    Get rid of specialization

    ~The last step for #5 is getting rid of specialization. I've remove most uses in 7c0379b1 and subsequent commits, except for the following ~17~ ~10~ ~9~ ~7~ 4:~ https://github.com/rust-lang/rust/pull/64564 changed the rules so there are again many default fn.

    src/class/descr.rs
    76:    default fn tp_descr_get() -> Option<ffi::descrgetfunc> {
    97:    default fn tp_descr_set() -> Option<ffi::descrsetfunc> {
    130:    default fn tp_as_descr(_type_object: &mut ffi::PyTypeObject) {}
    
    src/class/buffer.rs
    46:    default fn tp_as_buffer() -> Option<ffi::PyBufferProcs> {
    74:    default fn cb_bf_getbuffer() -> Option<ffi::getbufferproc> {
    110:    default fn cb_bf_releasebuffer() -> Option<ffi::releasebufferproc> {
    
    src/class/gc.rs
    27:    default fn update_type_object(_type_object: &mut ffi::PyTypeObject) {}
    72:    default fn tp_traverse() -> Option<ffi::traverseproc> {
    124:    default fn tp_clear() -> Option<ffi::inquiry> {
    
    src/class/iter.rs
    49:    default fn tp_as_iter(_typeob: &mut ffi::PyTypeObject) {}
    71:    default fn tp_iter() -> Option<ffi::getiterfunc> {
    94:    default fn tp_iternext() -> Option<ffi::iternextfunc> {
    
    src/class/basic.rs
    152:    default fn tp_as_object(_type_object: &mut ffi::PyTypeObject) {}
    153:    default fn nb_bool_fn() -> Option<ffi::inquiry> {
    183:    default fn tp_getattro() -> Option<ffi::binaryfunc> {
    252:        default fn set_attr() -> Option<ffi::setattrofunc> {
    274:        default fn del_attr() -> Option<ffi::setattrofunc> {
    296:        default fn set_del_attr() -> Option<ffi::setattrofunc> {
    324:    default fn tp_str() -> Option<ffi::unaryfunc> {
    344:    default fn tp_repr() -> Option<ffi::unaryfunc> {
    364:    default fn tp_hash() -> Option<ffi::hashfunc> {
    389:    default fn nb_bool() -> Option<ffi::inquiry> {
    409:    default fn tp_richcompare() -> Option<ffi::richcmpfunc> {
    
    src/class/mapping.rs
    83:    default fn tp_as_mapping() -> Option<ffi::PyMappingMethods> {
    116:    default fn mp_length() -> Option<ffi::lenfunc> {
    139:    default fn mp_subscript() -> Option<ffi::binaryfunc> {
    162:    default fn mp_ass_subscript() -> Option<ffi::objobjargproc> {
    187:    default fn mp_del_subscript() -> Option<ffi::objobjargproc> {
    201:    default fn det_set_dispatch() -> Option<ffi::objobjargproc> {
    
    src/class/pyasync.rs
    94:    default fn tp_as_async() -> Option<ffi::PyAsyncMethods> {
    121:    default fn am_await() -> Option<ffi::unaryfunc> {
    144:    default fn am_aiter() -> Option<ffi::unaryfunc> {
    167:    default fn am_anext() -> Option<ffi::unaryfunc> {
    
    src/conversion.rs
    107:    default fn with_borrowed_ptr<F, R>(&self, py: Python, f: F) -> R
    
    src/class/sequence.rs
    135:    default fn tp_as_sequence() -> Option<ffi::PySequenceMethods> {
    168:    default fn sq_length() -> Option<ffi::lenfunc> {
    190:    default fn sq_item() -> Option<ffi::ssizeargfunc> {
    239:        default fn set_item() -> Option<ffi::ssizeobjargproc> {
    285:        default fn del_item() -> Option<ffi::ssizeobjargproc> {
    328:        default fn del_set_item() -> Option<ffi::ssizeobjargproc> {
    372:    default fn sq_contains() -> Option<ffi::objobjproc> {
    394:    default fn sq_concat() -> Option<ffi::binaryfunc> {
    416:    default fn sq_repeat() -> Option<ffi::ssizeargfunc> {
    438:    default fn sq_inplace_concat() -> Option<ffi::binaryfunc> {
    465:    default fn sq_inplace_repeat() -> Option<ffi::ssizeargfunc> {
    
    src/class/number.rs
    625:    default fn tp_as_number() -> Option<ffi::PyNumberMethods> {
    692:    default fn nb_add() -> Option<ffi::binaryfunc> {
    714:    default fn nb_subtract() -> Option<ffi::binaryfunc> {
    736:    default fn nb_multiply() -> Option<ffi::binaryfunc> {
    758:    default fn nb_matrix_multiply() -> Option<ffi::binaryfunc> {
    780:    default fn nb_true_divide() -> Option<ffi::binaryfunc> {
    802:    default fn nb_floor_divide() -> Option<ffi::binaryfunc> {
    824:    default fn nb_remainder() -> Option<ffi::binaryfunc> {
    846:    default fn nb_divmod() -> Option<ffi::binaryfunc> {
    868:    default fn nb_power() -> Option<ffi::ternaryfunc> {
    890:    default fn nb_lshift() -> Option<ffi::binaryfunc> {
    912:    default fn nb_rshift() -> Option<ffi::binaryfunc> {
    934:    default fn nb_and() -> Option<ffi::binaryfunc> {
    956:    default fn nb_xor() -> Option<ffi::binaryfunc> {
    978:    default fn nb_or() -> Option<ffi::binaryfunc> {
    1000:    default fn nb_inplace_add() -> Option<ffi::binaryfunc> {
    1022:    default fn nb_inplace_subtract() -> Option<ffi::binaryfunc> {
    1044:    default fn nb_inplace_multiply() -> Option<ffi::binaryfunc> {
    1066:    default fn nb_inplace_matrix_multiply() -> Option<ffi::binaryfunc> {
    1088:    default fn nb_inplace_true_divide() -> Option<ffi::binaryfunc> {
    1110:    default fn nb_inplace_floor_divide() -> Option<ffi::binaryfunc> {
    1132:    default fn nb_inplace_remainder() -> Option<ffi::binaryfunc> {
    1154:    default fn nb_inplace_power() -> Option<ffi::ternaryfunc> {
    1176:    default fn nb_inplace_lshift() -> Option<ffi::binaryfunc> {
    1198:    default fn nb_inplace_rshift() -> Option<ffi::binaryfunc> {
    1220:    default fn nb_inplace_and() -> Option<ffi::binaryfunc> {
    1242:    default fn nb_inplace_xor() -> Option<ffi::binaryfunc> {
    1264:    default fn nb_inplace_or() -> Option<ffi::binaryfunc> {
    1287:    default fn nb_add_fallback() -> Option<ffi::binaryfunc> {
    1309:    default fn nb_sub_fallback() -> Option<ffi::binaryfunc> {
    1331:    default fn nb_mul_fallback() -> Option<ffi::binaryfunc> {
    1353:    default fn nb_matmul_fallback() -> Option<ffi::binaryfunc> {
    1375:    default fn nb_truediv_fallback() -> Option<ffi::binaryfunc> {
    1397:    default fn nb_floordiv_fallback() -> Option<ffi::binaryfunc> {
    1419:    default fn nb_mod_fallback() -> Option<ffi::binaryfunc> {
    1441:    default fn nb_divmod_fallback() -> Option<ffi::binaryfunc> {
    1463:    default fn nb_pow_fallback() -> Option<ffi::ternaryfunc> {
    1485:    default fn nb_lshift_fallback() -> Option<ffi::binaryfunc> {
    1507:    default fn nb_rshift_fallback() -> Option<ffi::binaryfunc> {
    1529:    default fn nb_and_fallback() -> Option<ffi::binaryfunc> {
    1551:    default fn nb_xor_fallback() -> Option<ffi::binaryfunc> {
    1573:    default fn nb_or_fallback() -> Option<ffi::binaryfunc> {
    1595:    default fn nb_negative() -> Option<ffi::unaryfunc> {
    1618:    default fn nb_positive() -> Option<ffi::unaryfunc> {
    1640:    default fn nb_absolute() -> Option<ffi::unaryfunc> {
    1662:    default fn nb_invert() -> Option<ffi::unaryfunc> {
    1684:    default fn nb_int() -> Option<ffi::unaryfunc> {
    1706:    default fn nb_float() -> Option<ffi::unaryfunc> {
    1728:    default fn nb_index() -> Option<ffi::unaryfunc> {
    
    src/instance.rs
    387:    default fn to_managed_py_ref<'p>(&self, py: Python<'p>) -> ManagedPyRef<'p, Self> {
    396:    default fn drop_impl(borrowed: &mut ManagedPyRef<Self>) {
    
    src/types/sequence.rs
    268:                default fn extract(obj: &'a PyAny) -> PyResult<Self> {
    307:    default fn extract(obj: &'a PyAny) -> PyResult<Self> {
    

    ~Once those are removed, we pyo3 can be stable on rust 1.30! I'm happy to take pull requests tackling one or more of those default fn uses~

    enhancement 
    opened by konstin 42
  • Questions about multithreading and asyncio

    Questions about multithreading and asyncio

    Hey, I've got a rather complicated program that involves mixing Rust async/await with Python asyncio. I know this problem isn't sorted out yet in pyo3 (although I think it's being worked on), but I figured I could get around those limitations by running two event loops, one for Rust and one for Python on its own dedicated thread.

    So here's what I'm doing:

    • Spin up an async Rust application with async-std runtime
    • Spawn a thread to act as the main thread for the asyncio event loop and call asyncio.set_event_loop(loop) and loop.run_forever() on that thread
    • Host a websocket server with async-tungstenite that occasionally interacts with the Python event loop via pyo3.
    • Interact with the Python event loop via call_soon_threadsafe and asyncio.run_coroutine_threadsafe to run my application
    • When the application exits, I call loop.call_soon_threadsafe(loop.stop) and join my thread

    My tests run exactly as expected with pyo3. The only problem is that when the test exits, I get one of three errors:

    If I'm lucky, my test exits successfully this warning:

    Exception ignored in: <module 'threading' from '/usr/lib/python3.8/threading.py'>
    Traceback (most recent call last):
      File "/usr/lib/python3.8/threading.py", line 1373, in _shutdown
        assert tlock.locked()
    AssertionError: 
    

    If I'm unlucky I either get this error:

    Fatal Python error: This thread state must be current when releasing
    
    Thread 0x00007f7dd1dd8b40 (most recent call first):
    
    Current thread 0x00007f7dcece9700 (most recent call first):
      File "/usr/lib/python3.6/asyncio/selector_events.py", line 148 in _write_to_self
      File "/usr/lib/python3.6/asyncio/base_events.py", line 643 in call_soon_threadsafe
    
    Thread 0x00007f7dcc4a0700 (most recent call first):
      File "/usr/lib/python3.6/selectors.py", line 445 in select
      File "/usr/lib/python3.6/asyncio/base_events.py", line 1415 in _run_once
      File "/usr/lib/python3.6/asyncio/base_events.py", line 438 in run_forever
    
    Thread 0x00007f7dce8e7700 (most recent call first):
    error: test failed
    

    or this one:

    FATAL: exception not rethrown
    error: test failed
    

    Things I've tried:

    • Spawning native rust threads with std::thread to run the event loop
    • Spawning python threads with the threading module to run my event loop
    • Calling pyo3::prepare_freethreaded_python() before the first call to Python::with_gil

    Sorry the problem description is so vague. I've tried simplifying my program to narrow the problem down, but it's frustratingly hard to replicate in a smaller codebase. Before I spend too much time on that, I just wanted to check with someone more experienced with embedding Python to see if I'm setting things up properly.

    Questions:

    • Am I setting up my threads correctly?
    • Do I need to call pyo3::prepare_freethreaded_python() before the first call to Python::with_gil in this situation?
    • Do I need to use a thread from the Python threading module or is it ok to use Rust's std::thread
    • Do I need to use some of the FFI calls to ensure that my Python thread state is correct?
    • Are there any other multi-threading limitations that I should know about with pyo3?

    🌍 Environment

    • OS: Ubuntu 20.04
    • Python version: 3.8.5 (installed with distro)
    • Rust version (rustc --version): rustc 1.47.0-nightly (6c8927b0c 2020-07-26)
    • PyO3 version: "0.12.3"
    • I have not tried pyo3 master yet (I think the problem is with my code)
    question 
    opened by awestlake87 41
  • Complete abi3 support

    Complete abi3 support

    For #1125. Continuation of #1132. What I did so far are

    • Prepared the proper feature flag. I named it all-apis but feel free to propose alternative.
      • EDITED: I renamed it unstable-api adviced by konstin.
    • Restructured method tables with cfg.

    TODO:

    • [x] Add CI setting for abi3
    • [x] Refactor tp_dict initialization codes (=classattr). It should be changed a lot.
    • [x] __text__signature__.
    • [x] Raise compile errors if some features (e.g., Buffer Protocol) is used without unstable-api feature.
    • [ ] Support additional feature flags with the minimum required Python version. See #1195 about the discussion.
    • [x] Documentation
      • Document about abi3 in the README and the guide.

    ... and so on.

    Sorry @alex for the conflict. I'm going to work on tp_dict problem for now and contributions for other areas are welcome.

    opened by kngwyu 41
  • Support rust extensions for PyPy via cpyext

    Support rust extensions for PyPy via cpyext

    Hi, Following the discussion on https://github.com/PyO3/pyo3/pull/370, I've taken some time to dust off my work on support for PyPy via cpyext.

    It runs nicely on my machine (osx) for the several test cases I've been using (raising errors, moving objects back and forth and so on).

    I've managed to run the word_count example successfully, altough for this usecase pypy beats rust in terms of performance (overhead is greater than benefit).

    ~~Unfortunely, it seems the rustapi_module example is still crashing, because of 2 outstanding issues:~~ EDIT: Both solved!

    ~~1. The datetime API wrapper.~~ ~~2. The way initialize_type behaves (causes SIGSEGV on attempt to register a new class) My guess is that there is still some outstanding difference in the implementation of the PyTypeObject struct that causes it to crash~~

    I'd like to outline some changes I've done, and why I think there is definitely some more work needed on this before it's ready for prime time.

    1. cpyext does not offer complete support for the entirety of the cpython API. So I've had (mostly by dumping symbols out of the pypy interpreter) manually add the parts of it that it implements with annotations like this: #[cfg_attr(PyPy, link_name = "PyPyErr_SetInterrupt")]

    I couldn't find any automatic way to set this up, since it's not for all the symbols, so it's quite verbose (I've done this with a regexes), but it makes it very easy to spot changes when the cpyext api layer changes and add them.

    1. I've never succeeded in getting this to work with python2, and since I've read in the PR that it's likely to be discontinued, I didn't invest more time with it.

    2. I've had to manually shim some functions that pypy expects to find when compiling against their version of Python.h, like PyUnicodeDecodeError_Create, these are mostly utility functions which they never bothered implementing, so it looks something like this: image

    3. I've modified the build.rs script originally in my implementation, since I needed it to detect and correctly emit configuration variables that are suitable for PyPy, it's a slight drift from the current model (single build.rs file), into a more structured internal crate that I've used. I didn't want to spend time porting all the changes back into the big build.rs file without hearing your thoughts on it first.

    4. As the title suggests, this only supports python -> rust, and not embedding pypy (due to pypy not behaving nicely with embedding), though I don't think embedding pypy is a significant use case, as it's mostly used to squeeze performance out of existing python codebases.

    EDIT: Functionallity that is still missing from cpyext that is used by PyO3 internally: These functions will need to be implemented in rust.

    • complex numbers _Py_c_* functions
    • PyDict_MergeFromSeq2
    • _PyLong_FromByteArray, _PyLong_AsByteArray
    • PySequence_Count
    • Py_CompileStringExFlags (EDIT: PyPy exports Py_CompileStringFlags, so this is actually OK)

    I'm still certain there are some semantics that I've missed, but i'd love to get another pair of eyes looking over some of this stuff.

    Would love to hear your thoughts :) Cheers 🐍

    P.S I'm away for next week, but I will be able to address suggestions the week after.

    opened by omerbenamram 39
  • #1064: Comparisons with __eq__ should not raise TypeError

    #1064: Comparisons with __eq__ should not raise TypeError

    RichComparisonToSelf expects to compare itself with objects of the same type, but then we shouldn't raise TypeError but return NotImplemented.

    Fixes #1064

    opened by mvaled 38
  • enum WIP

    enum WIP

    Thank you for contributing to pyo3!

    Here are some things you should check for submitting your pull request:

    • Run cargo fmt (This is checked by travis ci)
    • Run cargo clippy and check there are no hard errors (There are a bunch of existing warnings; This is also checked by travis)
    • If applicable, add an entry in the changelog.
    • If applicable, add documentation to all new items and extend the guide.
    • If applicable, add tests for all new or fixed functions
    • If you changed any python code, run black .. You can install black with pip install black)

    You might want to run tox (pip install tox) locally to check compatibility with all supported python versions. If you're using linux or mac you might find the Makefile helpful for testing.

    needs-implementer 
    opened by stillinbeta 38
  • Compile with stable rust

    Compile with stable rust

    PyO3 requires following unstable

    • [x] const_fn rust-lang/rust#24111
    • [x] specialization rust-lang/rust#31844
    • [x] use_extern_macros rust-lang/rust#35896
    • [x] proc_macro (proc_macro_attribute) rust-lang/rust#35896
    • [x] associated_consts
    • [x] ~~Optional, for wrap_function!: concat_idents (rust-lang/rust#29599)~~ Solved with mashup
    opened by fafhrd91 38
  • Undefined symbols for architecture arm64

    Undefined symbols for architecture arm64

    🐛 Bug Reports

    When reporting a bug, please provide the following information. If this is not a bug report you can just discard this template.

    🌍 Environment

    • Your operating system and version: MacOS 11.5 Big Sur, on M1 / arm64
    • Your python version: 3.9.5
    • How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: Pyenv, with virtualenv. Also tested with system python with virtualenv.
    • Your Rust version (rustc --version): rustc 1.54.0 (a178d0322 2021-07-26)
    • Your PyO3 version: 0.14.2
    • Have you tried using latest PyO3 main (replace version = "0.x.y" with git = "https://github.com/PyO3/pyo3")?: Yes

    💥 Reproducing

    $ cat Cargo.toml
    [package]
    name = "foo"
    version = "0.1.0"
    edition = "2018"
    
    [lib]
    name = "foo"
    crate-type = ["cdylib"]
    
    [dependencies.pyo3]
    version = "0.14.2"
    features = ["extension-module"]
    $ cat src/lib.rs
    use pyo3::prelude::*;
    
    #[pyfunction]
    fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
        Ok((a + b).to_string())
    }
    
    #[pymodule]
    fn foo(_py: Python, m: &PyModule) -> PyResult<()> {
        m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
        Ok(())
    }
    
    #[cfg(test)]
    mod tests {
        use super::*;
        #[test]
        fn test() {
            assert_ne!(sum_as_string(10, 20).unwrap(), "29");
            assert_eq!(sum_as_string(10, 20).unwrap(), "30");
        }
    }
    $ ./.venv/bin/python -m pip install --upgrade maturin
    Collecting maturin
      Using cached maturin-0.11.2-py3-none-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (10.5 MB)
    Collecting toml~=0.10.2
      Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
    Installing collected packages: toml, maturin
    Successfully installed maturin-0.11.2 toml-0.10.2
    $ maturin develop
    🔗 Found pyo3 bindings
    🐍 Found CPython 3.9 at python
       Compiling proc-macro2 v1.0.28
       Compiling unicode-xid v0.2.2
       Compiling proc-macro-hack v0.5.19
       Compiling syn v1.0.74
       Compiling pyo3-build-config v0.14.2 (https://github.com/PyO3/pyo3#336e87ec)
       Compiling once_cell v1.8.0
       Compiling libc v0.2.99
       Compiling unindent v0.1.7
       Compiling quote v1.0.9
       Compiling parking_lot_core v0.8.3
       Compiling parking_lot v0.11.1
       Compiling paste-impl v0.1.18
       Compiling pyo3 v0.14.2 (https://github.com/PyO3/pyo3#336e87ec)
       Compiling paste v0.1.18
       Compiling pyo3-macros-backend v0.14.2 (https://github.com/PyO3/pyo3#336e87ec)
       Compiling indoc-impl v0.3.6
       Compiling indoc v0.3.6
       Compiling pyo3-macros v0.14.2 (https://github.com/PyO3/pyo3#336e87ec)
       Compiling foo v0.1.0 (/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo)
        Finished dev [unoptimized + debuginfo] target(s) in 6.09s
    $ cargo test
       Compiling foo v0.1.0 (/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo)
    error: linking with `cc` failed: exit status: 1
      |
      = note: "cc" "-arch" "arm64" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.110fvvgslxwl7xjc.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.12kh3itm9hwj9eya.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.16g57alsqnv4g7f.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.1a0ghimskmekwiuk.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.1av1bln3bhfp9nj.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.1f7ufrshnyieunmq.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.1grj2h43vo039nxn.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.1w6b9uzfz2s5unib.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.23ga6qr0715pwz5y.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.24e8nkkcbbxnt787.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2a30e9rsdwwu0myc.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2bfqaxllo29w3ifh.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2crziipwfs2kz1rl.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2ehls50ircwkdmom.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2f1e1rxz18gsu2mj.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2fdsczpl1dm1fip.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2g4fb0qea2p4q6xm.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2guxm18uiabqkc4g.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2jufnvyjxohkzt7x.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2knf917igqi0yh23.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2p5zghtzgwcs84b3.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2ryebkgzo7xg8qnt.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2w9t2gyroz08k6r5.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2zqka4hc0y9fowwt.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2zxc1obih7i5hr7t.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.2zygjmkbd28bfn6s.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.31ff4xyicc3wnhmg.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.32mlaq0jdx1jm6es.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.386vuxnbh48s9wou.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.3cd74yt2oylhtzgz.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.3exvfe10ewcn52er.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.3g5mqpn1qg0wh1a9.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.3h0owt6q12tqs9k1.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.3lmvouff0a9ask7g.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.3m044sb2289k1a45.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.3nlmoppukou4iz9o.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.3sp1r43kboysk609.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.3z1hnn9k793ut1vv.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.404rd2jigivih456.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.41g3d6jr8g5e13dg.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.44bcgq52s2fm83bo.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.46y4g6gr7pgrmj5p.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4a8upifjlrchwi95.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4ibyiowjhkywlxz5.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4kb8d8qea0l2v9xb.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4u279idye23i71pz.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4vdhqlwluf10xra6.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4vw02eizd1tr7ojw.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4wsoqwueg83w4bj4.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4xwdfbwtcu6drnpn.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4zb71mjndf679w95.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.4zebmmyhpove6ig9.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.50i6cvni8o6e604x.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.50m5yqhkl1tj8tt7.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.51bwrx5e8skq6vdz.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.53gw0dkzu34phl59.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.54uz16ierizblqjb.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.58htorl2ddld3vy6.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.5efmic2dvfz0pyna.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.7vrbjuouu484zpk.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.d8kk98i9kjuql8j.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.gc7opmckjdvlw86.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.io5k9a5m56skmi7.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.m0svsb223gbvcdi.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.ov0hczhn18vqy4g.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.pp1aw1mhc4kqgka.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.q1smzop47y4w3d2.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.snh43turx8z4g5z.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.vfssu1qsbtfwmz3.rcgu.o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879.44z81xxn3zvsfpbf.rcgu.o" "-L" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps" "-L" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libtest-06ff11d645105e2b.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libterm-f05ecaeca66814e6.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgetopts-6ad0d034fc8e15c7.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunicode_width-c4beacdc35405cb8.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_std-fc0f4647def297a8.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libpyo3-194c7c6b45ae4486.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libparking_lot-3d6f176468cb3e55.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libparking_lot_core-1a66328f5320fcf8.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libsmallvec-b1d3333bf059e064.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/liblock_api-09ae7dbe1cd82c80.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libscopeguard-62e7a6d9289eec1f.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libinstant-4e05ab198a32da16.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libcfg_if-254a4227ea494762.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/liblibc-2d65d84e2d4e9c71.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libunindent-dda6001dcdbb754c.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libpaste-61320a4b949429d0.rlib" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/libindoc-758ee8d2665c486a.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-000cdec9267bfd7b.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-2669f3cbce8358f4.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-86461a1c60728ccb.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-f17574752cb5ddba.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-3bb606c936cc0d28.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-8139a4b0cda20184.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-ea0823eca3e9abf9.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-4f19e1259f6028e7.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-435daca85b8e10b5.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-d73085abefd284c6.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-e6a09ca0044b34e5.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-e07333f48f53c71e.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-2a49b0d9fbc7a459.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-b66dda66aafe36c9.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcore-34d0b58da984bf31.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-01275baa20724171.rlib" "-liconv" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879" "-Wl,-dead_strip" "-nodefaultlibs"
      = note: Undefined symbols for architecture arm64:
                "_PyExc_TypeError", referenced from:
                    _$LT$pyo3..exceptions..PyTypeError$u20$as$u20$pyo3..type_object..PyTypeInfo$GT$::type_object_raw::h3351d28f1f012e04 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.1.rcgu.o)
                "_PyExc_BaseException", referenced from:
                    _$LT$pyo3..exceptions..PyBaseException$u20$as$u20$pyo3..type_object..PyTypeInfo$GT$::type_object_raw::h3e7fd002a901032a in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.1.rcgu.o)
                "_PyBytes_AsString", referenced from:
                    pyo3::types::bytes::PyBytes::as_bytes::hcb83c311a32d8877 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.15.rcgu.o)
                "_PyErr_Print", referenced from:
                    pyo3::err::panic_after_error::h86153d46e74bf037 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.15.rcgu.o)
                "_PyBytes_Size", referenced from:
                    pyo3::types::bytes::PyBytes::as_bytes::hcb83c311a32d8877 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.15.rcgu.o)
                "_PyErr_NormalizeException", referenced from:
                    pyo3::err::PyErr::normalized::h9588758d1ccab3f5 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.15.rcgu.o)
                "_PyErr_PrintEx", referenced from:
                    pyo3::err::PyErr::print::h797d38ccce2cbbaf in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.15.rcgu.o)
                "_PyErr_NewException", referenced from:
                    pyo3::err::PyErr::new_type::hfb51b941f04ed644 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.15.rcgu.o)
                "_PyErr_Restore", referenced from:
                    pyo3::err::PyErr::restore::h1dbbf8b88416361d in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.15.rcgu.o)
                "_PyGILState_Release", referenced from:
                    _$LT$pyo3..gil..GILGuard$u20$as$u20$core..ops..drop..Drop$GT$::drop::hf2f8a6f21e2d4dfe in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.10.rcgu.o)
                "_PyErr_Fetch", referenced from:
                    pyo3::err::PyErr::fetch::h48d0241c5aeeaf12 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.15.rcgu.o)
                "_PyUnicode_AsUTF8AndSize", referenced from:
                    pyo3::types::string::PyString::to_str::h7219ef1704e18050 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.2.rcgu.o)
                "_PyGILState_Ensure", referenced from:
                    pyo3::gil::GILGuard::acquire_unchecked::hb63c5f585f222b6d in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.10.rcgu.o)
                "_PyEval_ThreadsInitialized", referenced from:
                    pyo3::gil::GILGuard::acquire::_$u7b$$u7b$closure$u7d$$u7d$::hb0a84271271719f2 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.10.rcgu.o)
                "_PyUnicode_AsEncodedString", referenced from:
                    pyo3::types::string::PyString::to_string_lossy::hdc219ee0b91d0d09 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.2.rcgu.o)
                "_Py_IsInitialized", referenced from:
                    pyo3::gil::GILGuard::acquire::_$u7b$$u7b$closure$u7d$$u7d$::hb0a84271271719f2 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.10.rcgu.o)
                "_PyExc_SystemError", referenced from:
                    _$LT$pyo3..exceptions..PySystemError$u20$as$u20$pyo3..type_object..PyTypeInfo$GT$::type_object_raw::h788c3eab76189041 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.1.rcgu.o)
                "_PyObject_Repr", referenced from:
                    pyo3::types::any::PyAny::repr::hb17f942fc5a28d64 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.11.rcgu.o)
                "__Py_Dealloc", referenced from:
                    pyo3::ffi::object::Py_DECREF::h99cb8d1a6d9fe0a4 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.5.rcgu.o)
                "_PyUnicode_FromStringAndSize", referenced from:
                    pyo3::types::string::PyString::new::h875d8b9d6e9e8e21 in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.2.rcgu.o)
                "_PyObject_GetAttr", referenced from:
                    pyo3::types::any::PyAny::getattr::_$u7b$$u7b$closure$u7d$$u7d$::h26dc56d4a757defc in libpyo3-194c7c6b45ae4486.rlib(pyo3-194c7c6b45ae4486.pyo3.960x274q-cgu.11.rcgu.o)
              ld: symbol(s) not found for architecture arm64
              clang: error: linker command failed with exit code 1 (use -v to see invocation)
    
    
    error: aborting due to previous error
    
    error: could not compile `foo`
    
    To learn more, run the command again with --verbose.
    

    Things I've tried with / without in various combinations:

    [target.aarch64-apple-darwin]
    rustflags = [
      "-C", "link-arg=-undefined",
      "-C", "link-arg=dynamic_lookup",
    ]
    

    This results in a different error:

    dyld: Symbol not found: _PyExc_BaseException
      Referenced from: /private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879
      Expected in: flat namespace
     in /private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879
    error: test failed, to rerun pass '--lib'
    
    Caused by:
      process didn't exit successfully: `/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.uLGa46f6aT/foo/target/debug/deps/foo-bf9e8350fad87879` (signal: 6, SIGABRT: process abort signal)
    
    • PYO3_PYTHON=/usr/bin/python3 cargo test no change
    • PYO3_PYTHON=/opt/homebrew/bin/python3 cargo test no change
    • PYO3_PYTHON=/usr/bin/python3 cargo test
    • PYO3_PYTHON=~/.pyenv/versions/3.9.6/bin/python cargo test no change
    • In or out of virtualenv no change

    Also no change with:

    [features]
    extension-module = ["pyo3/extension-module"]
    default = ["extension-module"]
    

    I've tried to cargo clean before cargo test with all of these, and tried with cargo test --no-default-features as well.

    Thanks for a cool library, I'm excited to start delving into it. FWIW this was building and testing fine on Windows yesterday :) It looks like there is some active work going on to make this easier on MacOS, so I hope this is more constructive than just adding to noise or confusion.

    Further reading:

    • https://github.com/PyO3/pyo3/issues/1719
    • https://github.com/PyO3/pyo3/issues/1741
    • https://github.com/PyO3/pyo3/issues/1330
    • https://pyo3.rs/latest/faq.html#i-cant-run-cargo-test-im-having-linker-issues-like-symbol-not-found-or-undefined-reference-to-_pyexc_systemerror
    documentation needs-implementer 
    opened by n8henrie 36
  • PySlice integer overflow on Windows

    PySlice integer overflow on Windows

    Bug Description

    In src/types/slice.rs, a PySlice is constructed like this:

    impl PySlice {
        /// Constructs a new slice with the given elements.
        pub fn new(py: Python<'_>, start: isizbe, stop: isize, step: isize) -> &PySlice {
            unsafe {
                let ptr = ffi::PySlice_New(
                    ffi::PyLong_FromLong(start as c_long),
                    ffi::PyLong_FromLong(stop as c_long),
                    ffi::PyLong_FromLong(step as c_long),
                );
                py.from_owned_ptr(ptr)
            }
        }
    ...
    

    The isize arguments are cast to std::os::raw::c_long. The issue is that, surprisingly, on Windows c_long is only 4 bytes. This causes slices to indices greater than i32::MAX to return incorrect results (see repro below).

    To be clear, slices above i32::MAX work in vanilla Python on Windows:

    import numpy
    array = numpy.arange(3_000_000_000)
    array[2_999_999_997:9_223_372_036_854_775_807]   # Slice goes to i64::MAX. This works in Python but fails in PyO3
    array([2999999997, 2999999998, 2999999999], dtype=int64)
    

    Steps to Reproduce

    use pyo3::{types::PySlice, Python};
    
    fn main() {
        Python::with_gil(|py| {
            let slice = PySlice::new(py, 0, isize::MAX, 1);
            println!("{}", slice.call_method0("__str__").unwrap().extract::<String>().unwrap());
            // "slice(0, 9223372036854775807, 1)" on Linux
            // "slice(0, -1, 1)" on Windows
        });
    }
    

    Backtrace

    No response

    Your operating system and version

    Windows

    Your Python version (python --version)

    3.10.5

    Your Rust version (rustc --version)

    rustc 1.65.0 (897e37553 2022-11-02)

    Your PyO3 version

    0.16.5

    How did you install python? Did you use a virtualenv?

    python.org installer

    Additional Info

    No response

    bug 
    opened by breckognize 1
  • Update logging.md for logging->rust

    Update logging.md for logging->rust

    Demonstrates how to register a rust log consumer with a pythong logging instance.

    Thank you for contributing to pyo3!

    Please consider adding the following to your pull request:

    • an entry for this PR in newsfragments - see [https://pyo3.rs/main/contributing.html#documenting-changes]
    • docs to all new functions and / or detail in the guide
    • tests for all new or changed functions

    PyO3's CI pipeline will check your pull request. To run its tests locally, you can run cargo xtask ci. See its documentation here.

    CI-skip-changelog 
    opened by dylanbstorey 5
  • Ensure fast linker is used by CI jobs

    Ensure fast linker is used by CI jobs

    Our CI jobs, especially the ones getting coverage, spent a lot of time building and running doc tests. Since each test is compiled separately, a lot of time is often spent linking those executables and we should make sure our CI jobs use a faster linker, e.g. LLD is often significantly faster and easily enabled by adding -Clink-arg=-fuse-ld=lld to RUSTFLAGS.

    Potentially, mold could provide further improvements but I have no personal experience using it.

    opened by adamreichold 1
  • Improving ergonomics of getting slices from a PyBuffer

    Improving ergonomics of getting slices from a PyBuffer

    I'm writing a function that accepts anything that can create a read-only buffer:

    fn validate_cbor(&self, py: Python<'_>, cbor: &PyAny) -> PyResult<()> {
            let buffer = PyBuffer::<u8>::get(cbor)?;
            if !buffer.readonly() {
                return Err(PyValueError::new_err("Must be read-only byte buffer"));
            }
            let slice = buffer
                .as_slice(py)
                .ok_or_else(|| PyTypeError::new_err("Must be a contiguous sequence of bytes"))?;
    
            // Safety: The slice is &[ReadOnlyCell<u8>]. A ReadOnlyCell has the same
            // memory representation as the underlying data; it's
            // #[repr(transparent)] newtype around UnsafeCell. And per Rust docs
            // "UnsafeCell<T> has the same in-memory representation as its inner
            // type T". So the main issue is whether the data is _really_ read-only.
            // We do the read-only check above, and yes a caller can probably somehow
            // lie, but if they do that, that's really their fault.
            let cbor: &[u8] = unsafe { std::mem::transmute(slice) };
            
            // ... custom code here
    }
    

    It's not clear to me why PyBytes::as_bytes() shouldn't return &[ReadOnlyCell<u8>] too. After all, there is nothing preventing a particularly malicious user from overwriting the data in a Python bytes object, just need to write a little bit of C. We need to trust callers to some extent in a world where they can write C.

    So my feeling is that as_slice() can just return Some(&[T]), unless there's some reason to expect buffer API implementations to lie about their readonly flag.

    As a follow-up, if that makes sense then maybe the pattern above could be simplified and built-in to PyO3.

    enhancement 
    opened by itamarst 4
  • 3.11.1 and 3.12 alphas need additional path configuration on Windows

    3.11.1 and 3.12 alphas need additional path configuration on Windows

    See https://github.com/PyO3/pyo3/actions/runs/3721944864/jobs/6312442577#step:11:1094 - we have CI failures on Windows along the lines of

    thread 'exceptions::asyncio::tests::TimeoutError' panicked at 'Can not import module asyncio: ModuleNotFoundError: No module named '_socket'
    

    Appears to be the following upstream issue: https://github.com/python/cpython/issues/100171

    We may need to do something smarter in auto-initialize depending on what the outcome of decisions upstream are. We have had several issues of interacting with virtualenvs in the past, e.g. #1741 #1896, maybe it's now time anyway to implement a new "default" in prepare_freethreaded_python which covers what users are most likely to want?

    opened by davidhewitt 0
Releases(v0.17.3)
  • v0.17.3(Nov 1, 2022)

    This release confirms support for Python 3.11 in PyO3.

    (Previous versions of PyO3 0.17 had been tested against release candidates of Python 3.11 and no Python 3.11 fixes have been added since 0.17.0, so all PyO3 0.17 versions should in practice be fine to use with Python 3.11.)

    In addition, some other small additions and fixes have been added on top of PyO3 0.17.2.

    Thank you to the following users for the improvements:

    @adamreichold @ctb @datapythonista @davidhewitt @messense @saethlin

    Source code(tar.gz)
    Source code(zip)
  • v0.17.2(Oct 4, 2022)

    This release contains non-breaking improvements and bugfixes over PyO3 0.17.1.

    A new chrono feature has been added to support converting from types in chrono to types in the Python datetime module. The num-bigint feature has been expanded to add support to PyPy.

    There has also been fixes for a couple of regressions observed in PyO3 0.17.

    Thank you to the following users for the improvements:

    @adamreichold @AdilZouitine @davidhewitt @messense @mrob95 @Oppen @prehner @Psykopear @ryanrussell @smheidrich @SquidDev

    Source code(tar.gz)
    Source code(zip)
  • v0.17.1(Aug 28, 2022)

    This release contains some minor bug fixes for PyO3 0.17.0. In particular the new PyDictItems, PyDictKeys and PyDictValues types are actually accessible!

    Thanks to @davidhewitt, @messense and @PrettyWood for the fixes.

    Source code(tar.gz)
    Source code(zip)
  • v0.17.0(Aug 23, 2022)

    This release contains a focus on quality improvements over the PyO3 0.16 releases.

    There have been new API types added such as PyDictKeys, PyDictValues, PyDictItems, PyCode, PyFrame, and PySuper. The PyMapping and PySequence types have changed so they are more directly compatible with the corresponding Python Mapping and Sequence base classes in the collections.abc module (this is a breaking change).

    A new #[pyclass(frozen)] option has been added to opt-out of runtime borrow checking by removing the ability to access &mut self for objects owned by Python.

    There have been a number of soundness fixes, both to the PyCapsule type (see the CHANGELOG for more details) and to a number of FFI bindings which had fallen out of sync with newer Python and PyPy releases.

    There have been numerous other smaller improvements, changes and fixes. For full details see the CHANGELOG.

    Please consult the migration guide for help upgrading.

    Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @acshi @aganders3 @alex @birkenfeld @cjermain @Cryptex-github @cuishuang @davidhewitt @drewkett @dswij @herquan @hoodmane @ikrivosheev @indygreg @jeertmans @jinlow @jonaspleyer @kngwyu @mejrs @messense @n8henrie @PigeonF @PWhiddy @ravenexp @savente93 @yankun1992 @yodaldevoid

    Source code(tar.gz)
    Source code(zip)
  • v0.16.6(Aug 23, 2022)

    This release is a tactical set of soundness fixes identified for the PyCapsule bindings released in PyO3 0.16. To avoid breaking API changes capsules created with PyCapsule::new and PyCapsule::new_with_destructor will now leak their contents (and not call the destructor) if released on a thread other than the one they were created.

    PyO3 0.17 will be released shortly with breaking API changes which resolve the PyCapsule issues with better design (e.g. the destructor has a Send bound added). Users are encouraged to upgrade at their earliest convenience.

    Thanks to @saethlin for reporting the issue, and to @adamreichold and @davidhewitt for implementing the resolution.

    Source code(tar.gz)
    Source code(zip)
  • v0.16.5(May 15, 2022)

    This release contains an FFI definition correction to resolve crashes for PyOxidizer on Python 3.10, and a new generate-import-lib feature to allow easier cross-compiling to Windows.

    Thank you to the following users for the improvements:

    @cjermain @davidhewitt @indygreg @messense

    Source code(tar.gz)
    Source code(zip)
  • v0.16.4(Apr 14, 2022)

    This release fixes a regression introduced in 0.16.3 leading to build failures on Windows, enables crates depending on PyO3 to collect code coverage using LLVM instrumentation on stable Rust version 1.60 or later, and enables safe access to time zone information attached to Python's time and datetime objects. There are also some adjustments to PyO3's build-time interpreter detection to make it easier to cross-compile abi3 Python extensions.

    Thank you to the following users for the improvements:

    @adamreichold @davidhewitt @mejrs @messense @pickfire @ravenexp @RicoHageman

    Source code(tar.gz)
    Source code(zip)
  • v0.15.2(Apr 14, 2022)

    This release is a backport of PyO3 0.16's support for PyPy 3.9.

    Thanks to @mejrs and @messense for the implementation work, and to @alex for testing it to build the cryptography package.

    Source code(tar.gz)
    Source code(zip)
  • v0.16.3(Apr 5, 2022)

    This release contains a number of non-breaking additions and fixes since PyO3 0.16.2.

    They are mostly centered on improving support for various build configurations. There are also usability tweaks to the #[pyclass] macro and a new intern! macro to create statically-backed PyString objects as an optimization technique.

    Thank you to the following users for the improvements:

    @adamreichold @aganders3 @alex @kmp1 @mejrs @messense @mityax
    @momirza @ravenexp @zh-jq

    Source code(tar.gz)
    Source code(zip)
  • v0.16.2(Mar 15, 2022)

    This release contains support for the latest PyPy 7.3.8 release, a fix for a regression from PyO3 0.16.0 which would cause flaky build failures, and corrections to documentation.

    Thank you for the following users for the improvements:

    @adamreichold @alex @birkenfeld @davidhewitt @messense

    Source code(tar.gz)
    Source code(zip)
  • v0.16.1(Mar 5, 2022)

    This release contains a couple of fixes for regressions in PyO3 0.16.0, as well as documentation fixes. The hashbrown optional dependency range has been extended to include its new 0.12 version.

    Thank you for the following users for the improvements:

    @adamreichold @alex @birkenfeld @davidhewitt @JerzySpendel @mejrs @messense @PanQL

    Source code(tar.gz)
    Source code(zip)
  • v0.16.0(Feb 27, 2022)

    This release contains substantial changes and improvements over PyO3 0.15.1. To support these improvements, support has been dropped for the end-of-life Python 3.6 and Rust versions older than 1.48.

    The pyo3::ffi submodule has been split out into a separate pyo3-ffi crate, so that users who want to use PyO3's Python bindings without any of the safe APIs or macros are able to do so.

    #[pyclass] can now be used on simple "C-like" enums to create Python enums.

    The #[pyproto] macro has been deprecated, and can be disabled by disabling the optional #[pyproto] feature. The "magic methods" such as __repr__ which previously were implemented by #[pyproto] gained support in #[pymethods] in 0.15, and now in PyO3 0.16 #[pymethods] is intended to be the only attribute macro needed to write class method implementations.

    There are numerous other reworks, improvements, and bugfixes.

    For full details of all changes, see the CHANGELOG.

    Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @adamreichold @aganders3 @Amanieu @aviramha @birkenfeld @b05902132 @CarlKCarlK @cmpute @danielhenrymantilla @davidhewitt @DSPOM2 @ghuls @Gobot1234 @kevinheavey @konstin @mejrs @messense @milesgranger @mrl5 @parsons20 @RicoHageman @saidvandeklundert @Tom1380 @vxgmichel

    Source code(tar.gz)
    Source code(zip)
  • v0.15.1(Nov 19, 2021)

    This release is a set of bug fixes for some minor issues reported since PyO3 0.15's release. There are also some small additions for those storing PyIterator, PySequence, and PyMapping in Py smart pointers, and a PyTraceback type to ease interacting with Python tracebacks from Rust.

    For full details of all changes, see the CHANGELOG.

    Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @dansvo @davidhewitt @KRunchPL @mejrs @messense @moriyoshi @saidvandeklundert @taiki-e

    Source code(tar.gz)
    Source code(zip)
  • v0.15.0(Nov 3, 2021)

    This release of PyO3 brings support for Python 3.10 and PyPy 3.8. In addition, new optional dependencies on anyhow and eyre have been added for easy integration of the popular error-handling libraries with Python code.

    A number of consistency improvements have been made to PyList, PyTuple and PySequence APIs. They now all exclusively use usize- based indexing, and now also support Rust's indexing operator.

    In this release #[pymethods] are now able to implement many magic methods such as __str__ and __repr__, removing the need for #[pyproto] macro implementations. For the 0.15 release series both #[pymethods] and #[pyproto] will be supported; #[pyproto] is expected to be deprecated in the future.

    For full details of all changes, see the CHANGELOG. For help with upgrading, see the migration guide.

    Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback.

    Source code(tar.gz)
    Source code(zip)
  • v0.14.5(Sep 5, 2021)

    This release fixes a compile regression of PyO3 0.14.4 where not all APIs related to PyStringData were conditionally disabled correctly on big-endian platforms.

    In addition, a few public APIs have been added to the pyo3_build_config crate to support needs of the PyOxidizer project.

    Thanks to @decathorpe and @indygreg for further reports, discussions, and resolution.

    Source code(tar.gz)
    Source code(zip)
  • v0.14.4(Aug 29, 2021)

    This release resolves issues with the PyString::data API added in 0.14.3. After release it was found to be unsound on certain platforms, so has been disabled on those platforms and marked unsafe with the safety expectation that users will ensure it works appropriately on their platform as part of usage.

    Thanks to @decathorpe and @indygreg for the report, discussion, and resolution.

    Source code(tar.gz)
    Source code(zip)
  • v0.14.3(Aug 22, 2021)

    This release is a small set of bugfixes added on top of the PyO3 0.14.2 release.

    For full details of all changes, see the CHANGELOG.

    Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @awestlake87 @davidhewitt @indygreg @mejrs @messense @mtreinish @pschafhalter @Ptrskay3 @sansyrox @tiran

    Source code(tar.gz)
    Source code(zip)
  • v0.14.2(Aug 9, 2021)

    This release is a small quality-of-life update for the PyO3 0.14 release series. Optional support for the indexmap crate has been added. In addition, there have been a number of documentation improvements, and bugfixes for regressions and incorrect FFI definitions.

    Users who removed macOS cargo configuration from their setup after updating to PyO3 0.14 will unfortunately have to once again add configuration to their compilation after updating to this release. This is because PyO3 was using functionality which Cargo had erroneously allowed.

    The recommended way to restore configuration for macOS is to add a build script which invokes pyo3_build_config::add_extension_module_link_args(). The cargo configuration previously recommended is also still an option.

    For full details of all changes, see the CHANGELOG.

    Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @alex @awestlake87 @batconjurer @birkenfeld @davidhewitt @deantvv @Eric-Arellano @indygreg
    @IvanIsCoding @mejrs @messense @nihaals @tommilligan

    Source code(tar.gz)
    Source code(zip)
  • v0.14.1(Jul 4, 2021)

    This release addresses some incorrect FFI definitions in PyO3 0.14.0 which caused crashes when targeting PyPy. In addition, IntoPy<PyObject> has been implemented for &PathBuf and &OsString for additional convenience when working with the new conversions.

    Thank you to @alex and @jameshilliard for the quick reports, and @birkenfeld and @messense for the quick fixes!

    Source code(tar.gz)
    Source code(zip)
  • v0.14.0(Jul 3, 2021)

    This release contains a significant number of improvements and optimizations since the 0.13 releases. Conversion support for more Rust standard library types has been added: [T; N], Path, PathBuf, OsStr, and OsString. In addition, the #[pyo3(...)] attribute options already used in #[pyclass] and #[derive(FromPyObject)] have been expanded to all macros to cover more options such as #[pyo3(name = "...")].

    This release also includes some notable performance optimizations to PyO3's Python function generation, and users should observe reductions in overhead when calling PyO3-wrapped functions from Python. The documentation has also benefited from extensive work from several contributors.

    Finally, there are a number of breaking changes to this release:

    • To reduce the default dependency set, a new feature multiple-pymethods has been added. If this feature is not enabled, each #[pyclass] may only have a single #[pymethods] block. If this feature is enabled then the previous behavior (and dependency on inventory) is restored.
    • The pyo3::ffi module has received many updates to keep in sync with CPython's definitions.
    • A number of APIs and particular usages of PyO3 macros have been marked deprecated where better alternatives now exist.
    • A number of previously-deprecated APIs have been removed.

    For full details of all changes, see the CHANGELOG. For help with upgrading, see the migration guide.

    Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @alex @alonblade @aviramha @birkenfeld @c410-f3r @Daggy1234 @daniil-konovalenko @davidhewitt @erikjohnston @GabrielMajeri @gilescope @indygreg @isosphere @jameshilliard @kangalioo @kngwyu @konstin @m-ou-se @mejrs @messense @nw0 @ohadravid @ravenexp @scalexm @simonrainerson @TheGhostHuCodes @winxton @1tgr

    Source code(tar.gz)
    Source code(zip)
  • v0.13.2(Feb 12, 2021)

    This release contains small improvements over PyO3 0.13.1. The minimum supported Rust version has been lowered to Rust 1.41. A serde feature has been added which provides implementations of Serialize and Deserialize for types inside the Py<T> smart pointer.

    Of note is that this release disables automatic finalization of the Python interpreter at program exit for Rust binaries embedding the Python interpreter. This was found to cause a number of issues with C extensions such as Tensorflow and Scipy which were sensitive to the deinitialization order. For the vast majority of users this should cause no impact or fix a number of problem cases. For the minority who did depend on finalization, the unsafe method with_embedded_python_interpreter has been added to manually control the interpreter lifecycle.

    For full details of all changes, see the CHANGELOG.

    Thank you to everyone who contributed code, documentation fixes, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @alex @awestlake87 @birkenfeld @cfbolz @daniil-konovalenko @davidhewitt @decathorpe @ijl @kangalioo @kngwyu @lazka @nw0 @tdamsma

    Source code(tar.gz)
    Source code(zip)
  • v0.13.1(Jan 10, 2021)

    This release is a set of quality-of-life improvements and bugfixes for PyO3 0.13.0. All new functionality adds flexibility to pyo3 in rarer cases, such as fixing #[pyclass(dict)] support for the abi3 feature (on compatible Python versions). In addition, a long-overdue initiative to reconcile PyO3's ffi definitions with the current state of the upstream Python API has started. This resulted in a number of new deprecations where mismatches were found.

    For full details of all changes, see the CHANGELOG.

    Thank you to everyone who contributed code, documentation fixes, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @cecini @dalcde @daniil-konovalenko @davidhewitt @kngwyu @konstin @nw0 @OrangeTux

    Source code(tar.gz)
    Source code(zip)
  • v0.13.0(Dec 22, 2020)

    The major feature for this PyO3 release is support for the CPython limited API. This is an opt-in feature when building extension modules which makes it possible for the built wheel to be installable on multiple Python versions.

    As well as this, a number of other refinements, fixes, and performance improvements have been merged into this release. In particular, note that the internal "macros" crates have been renamed to pyo3-macros and pyo3-macros-backend for clarity and consistency (previously pyo3cls and pyo3-derive-backend).

    For full details of all changes, see the CHANGELOG.

    Thank you to everyone who contributed code, documentation fixes, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @AleksaC @alex @Askaholic @aviramha @birkenfeld @Daggy1234 @davidhewitt @dvermd @kngwyu @konstin @Palladinium @roblabla @scalexm @Stranger6667

    Source code(tar.gz)
    Source code(zip)
  • v0.12.4(Nov 28, 2020)

    This release fixes a reference count issue discovered in PyO3 0.12.3, present in the implementation of From<Py<T>> for PyObject. Usage of this implementation would lead to an incorrect Python reference count decrease, likely leading to use-after-free.

    This regression was only present in the PyO3 0.12 series. The offending implementation was used only in one location inside the PyO3 codebase itself, though of course is likely used in downstream code. As a result, previous PyO3 versions in the 0.12 series will be yanked.

    Sincerest apologies for the inconvenience.

    Fixed

    • Fix reference count bug in implementation of From<Py<T>> for PyObject, a regression introduced in PyO3 0.12. #1297
    Source code(tar.gz)
    Source code(zip)
  • v0.12.3(Oct 12, 2020)

    This release fixes support for Rust versions 1.39 through Rust 1.44, which was erroneously broken in PyO3 0.12.2. Thanks to @mtreinish for the quick bug report!

    Source code(tar.gz)
    Source code(zip)
  • v0.12.2(Oct 12, 2020)

    This release adds the ability to specify keyword-only arguments in #[pyfunction] definitions, a new function Python::check_signals, and a couple of smaller refinements improving upon PyO3 0.12.1.

    Thank you to everyone who contributed code, documentation fixes, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @birkenfeld @davidhewitt @dvermd
    @kngwyu @sebpuetz

    Source code(tar.gz)
    Source code(zip)
  • v0.12.1(Sep 16, 2020)

    This release contains a couple of bugfixes to resolve complation issues with 0.12.0 on specific targets. Thank you to @alex and @mtreinish for the bug reports and subsequent patches.

    Fixed

    • Fix building for a 32-bit Python on 64-bit Windows with a 64-bit Rust toolchain. #1179
    • Fix building on platforms where c_char is u8. #1182
    Source code(tar.gz)
    Source code(zip)
  • v0.12.0(Sep 12, 2020)

    This release includes a few careful revisions to the PyO3 API with the intention to make it easier to learn and use. The PyErr type is reworked to implement std::error::Error. The FromPy trait is removed. Finally, the PyObject struct is now just a type alias to Py<PyAny>. While we do not take pleasure in making breaking changes for users' code, we have done so with careful thought to make migration as simple as possible while also allowing the PyO3 API to mature. Please see the migration guide for help upgrading if any of these changes affect you.

    Also added is a new #[derive(FromPyObject)] macro, which enables a convenient way to accept arguments of the Python "type" Union. (See the guide entry on this new feature.)

    There have been many other improvements and bugfixes too numerous to go into detail here. For the full list, see the CHANGELOG.

    Thank you to everyone who contributed code, documentation fixes, design ideas, bug reports, and feedback on the way to PyO3 0.12. The full list of users who commited at least one contribution to PyO3 0.12 is below:

    @alex @birkenfeld @cathay4t @davidhewitt @Hakuyume @Hywan @kngwyu @konstin @marioortizmanero @MoritzLangenstein @mtreinish @mvaled @nagisa @noam93k @Progdrasil @programmerjake @rob-thatcher @sebpuetz @vorner @vthriller

    Source code(tar.gz)
    Source code(zip)
  • v0.11.1(Jul 3, 2020)

  • v0.11.0(Jun 28, 2020)

    This is our first version that supports the stable Rust toolchain. The minimum required version is 1.39.0. Thank you @davidhewitt @scalexm @tamuhey @konstin @Nateckert @Alexander-N @m-ou-se and all issue reporters :slightly_smiling_face:

    Maybe the most influential change is now #[pyclass] requires Send. Please see the migration guide for more.

    Added

    • Support stable versions of Rust (>=1.39). #969
    • Add FFI definition PyObject_AsFileDescriptor. #938
    • Add PyByteArray::data, PyByteArray::as_bytes, and PyByteArray::as_bytes_mut. #967
    • Add GILOnceCell to use in situations where lazy_static or once_cell can deadlock. #975
    • Add Py::borrow, Py::borrow_mut, Py::try_borrow, and Py::try_borrow_mut for accessing #[pyclass] values. #976
    • Add IterNextOutput and IterANextOutput for returning from __next__ / __anext__. #997

    Changed

    • Simplify internals of #[pyo3(get)] attribute. (Remove the hidden API GetPropertyValue.) #934
    • Call Py_Finalize at exit to flush buffers, etc. #943
    • Add type parameter to PyBuffer. #951
    • Require Send bound for #[pyclass]. #966
    • Add Python argument to most methods on PyObject and Py<T> to ensure GIL safety. #970
    • Change signature of PyTypeObject::type_object() - now takes Python argument and returns &PyType. #970
    • Change return type of PyTuple::slice() and PyTuple::split_from() from Py<PyTuple> to &PyTuple. #970
    • Change return type of PyTuple::as_slice to &[&PyAny]. #971
    • Rename PyTypeInfo::type_object to type_object_raw, and add Python argument. #975
    • Update num-complex optional dependendency from 0.2 to 0.3. #977
    • Update num-bigint optional dependendency from 0.2 to 0.3. #978
    • #[pyproto] is re-implemented without specialization. #961
    • PyClassAlloc::alloc is renamed to PyClassAlloc::new. #990
    • #[pyproto] methods can now have return value T or PyResult<T> (previously only PyResult<T> was supported). #996
    • #[pyproto] methods can now skip annotating the return type if it is (). #998

    Removed

    • Remove ManagedPyRef (unused, and needs specialization) #930

    Fixed

    • Fix passing explicit None to Option<T> argument #[pyfunction] with a default value. #936
    • Fix PyClass.__new__'s not respecting subclasses when inherited by a Python class. #990
    • Fix returning Option<T> from #[pyproto] methods. #996
    • Fix accepting PyRef<Self> and PyRefMut<Self> to #[getter] and #[setter] methods. #999
    Source code(tar.gz)
    Source code(zip)
Owner
PyO3
Pythonium Trioxide
PyO3
RustPython - A Python Interpreter written in Rust

RustPython A Python-3 (CPython >= 3.9.0) Interpreter written in Rust ?? ?? ?? . Usage Check out our online demo running on WebAssembly. RustPython req

null 13.3k Jan 2, 2023
Rust bindings for the Wasm spec interpreter.

wasm-spec-interpreter This project shows how to use ocaml-interop to call into the Wasm spec interpreter. There are several steps to making this work:

Bytecode Alliance 9 Aug 23, 2022
Create a Python project automatically with rust (like create-react-app but for python)

create-python-project Create a Python project automatically with rust (like create-react-app but for python) Installation cargo install create-python-

Dhravya Shah 2 Mar 12, 2022
Create, open, manage your Python projects with ease, a project aimed to make python development experience a little better

Create, open, manage your Python projects with ease, a project aimed to make python development experience a little better

Dhravya Shah 7 Nov 18, 2022
Rust <-> Python bindings

rust-cpython Rust bindings for the python interpreter. Documentation Cargo package: cpython Copyright (c) 2015-2020 Daniel Grunwald. Rust-cpython is l

Daniel Grunwald 1.7k Dec 29, 2022
Implementation of Monte Carlo PI approximation algorithm in Rust Python bindings

rusty_pi Implementation of Monte Carlo PI approximation algorithm in Rust Python bindings. Time of 100M iterations approximation on Core i7 10th gen:

Aleksey Popov 1 Jul 6, 2022
Very experimental Python bindings for the Rust biscuit-auth library

Overview This is a very experimental take on Python bindings for the biscuit_auth Rust library. It is very much a work in progress (limited testing, m

Josh Wright 5 Sep 14, 2022
Python bindings for heck, the Rust case conversion library

pyheck PyHeck is a case conversion library (for converting strings to snake_case, camelCase etc). It is a thin wrapper around the Rust library heck. R

Kevin Heavey 35 Nov 7, 2022
lavalink-rs bindings for Python

lavasnek_rs Dev Docs: Main Site | Fallback: GitHub Pages GitHub repo GitLab repo Using the library The library is available on PyPi, and you can insta

Victoria Casasampere Fernandez 39 Dec 27, 2022
The polyglot bindings generator for your library (C#, C, Python, …) 🐙

Interoptopus ?? The polyglot bindings generator for your library. Interoptopus allows you to deliver high-quality system libraries to your users, and

Ralf Biedert 155 Jan 3, 2023
Python bindings for akinator-rs using pyo3

Akinator-py python bindings for akinator-rs using pyo3 Installation Prebuilt wheels are uploaded onto pypi, if you platform is supported, you can inst

Tom-the-Bomb 4 Nov 17, 2022
A simple Pascal interpreter written in rust.

rascal A simple Pascal interpreter written in rust. Usage Download the latest rascal executable from the release page. Run the executable. rascal.exe

null 47 Dec 7, 2022
A simple interpreter language written in Rust.

Glang Backstory Hello and welcome to the Glang git repository. Game Slang or in short Glang is a super simple interpreted language written in Rust wit

null 6 Nov 12, 2022
A memory safe Lua interpreter

Hematita Da Lua Hematita Da Lua is an interpreter for the scripting language Lua, written entirely in 100% safe Rust. Hematita is the portugese word f

Daniel 149 Dec 29, 2022
An interpreter for the esoteric programming language, Brainf*ck.

Brainf*ck Interpreter This is just a normal Brainf*ck interpreter written in Rust. If you don't know what Brainf*ck is, you can check out the wikipedi

Callum Irving 0 Dec 23, 2021
WebAssembly (Wasm) interpreter.

Continuous Integration Test Coverage Documentation Crates.io wasmi- WebAssembly (Wasm) Interpreter wasmi was conceived as a component of parity-ethere

Parity Technologies 1k Jan 4, 2023
Lisp interpreter that might be fast someday maybe?

ehlisp Pronunciation I'm not really sure. Maybe like an incorrect pronunciation of "ellipse", like "ellisp"? Also maybe like "a lisp". I named it this

Eddie Hatfield 3 Oct 6, 2022
A script language like Python or Lua written in Rust, with exactly the same syntax as Go's.

A script language like Python or Lua written in Rust, with exactly the same syntax as Go's.

null 1.4k Jan 1, 2023
Rust Python modules for interacting with Metaplex's NFT standard.

Simple Metaplex Metadata Decoder Install the correct Python wheel for your Python version with pip: pip install metaplex_decoder-0.1.0-cp39-cp39-manyl

Samuel Vanderwaal 11 Mar 31, 2022