Standalone python3.dll import library generator

PyO3

Last update: Oct 9, 2022

Related tags

Foreign Function Interface python3-dll-a

Overview

Standalone `python3.dll` import library generator

Generates import libraries for the Stable ABI Python DLL for MinGW-w64 and MSVC (cross-)compile targets.

See https://docs.python.org/3/c-api/stable.html for details.

This crate does not require Python 3 distribution files to be present on the (cross-)compile host system.

Note: MSVC (cross-)compile targets require LLVM binutils to be available on the host system. More specifically, python3-dll-a requires llvm-dlltool executable to be present in PATH when targeting *-pc-windows-msvc.

PyO3 integration

Since version 0.16.4, the pyo3 crate implements support for the Stable ABI Python DLL import library generation via its new generate-abi3-import-lib feature.

In this configuration, python3-dll-a becomes a pyo3 crate dependency and is automatically invoked by its build script in both native and cross compilation scenarios.

Example `Cargo.toml` usage for a PyO3 extension module

[dependencies]
pyo3 = { version = "0.16.4", features = ["extension-module", "abi3-py37", "generate-abi3-import-lib"] }

Standalone build script usage

If an older pyo3 crate version is used, or a different Python bindings library is required, python3-dll-a can be used directly from the crate build script.

The examples below assume using an older version of PyO3.

Example `build.rs` script

The following cargo build script can be used to cross-compile Stable ABI PyO3 extension modules for Windows (64/32-bit x86 or 64-bit ARM) using either MinGW-w64 or MSVC target environment ABI:

fn main() {
    if std::env::var("CARGO_CFG_TARGET_OS").unwrap() == "windows" {
        let cross_lib_dir = std::env::var_os("PYO3_CROSS_LIB_DIR")
            .expect("PYO3_CROSS_LIB_DIR is not set when cross-compiling");
        let arch = std::env::var("CARGO_CFG_TARGET_ARCH").unwrap();
        let env = std::env::var("CARGO_CFG_TARGET_ENV").unwrap();

        let libdir = std::path::Path::new(&cross_lib_dir);
        python3_dll_a::generate_implib_for_target(libdir, &arch, &env)
            .expect("python3.dll import library generator failed");
    }
}

A compatible python3.dll import library file named python3.dll.a or python3.lib will be automatically created in the directory pointed by the PYO3_CROSS_LIB_DIR environment variable.

Example `cargo build` invocation

PYO3_CROSS_LIB_DIR=target/python3-dll cargo build --target x86_64-pc-windows-gnu

Maintenance

This crate embeds the stable_abi.txt definitions file from CPython in the Misc subdirectory.

The upstream version of this file is located in the CPython project repository under the same path. This file should be updated for every subsequent CPython release (e.g. for CPython 3.12).

Comments

Move to PyO3 organization
[x] Move repository

[x] Keep @ravenexp as a collaborator of this repository

[x] Add pyo3/admins to the owners of this crate on crates.io

[x] Update related links, for example repository in Cargo.toml
opened by messense 14
RFC: Include python3.def itself in the Rust source

All parsing and processing is done in the CI so that the crate itself includes the python3.def file itself instead of creating it on the fly.

Based on the idea voiced in https://github.com/PyO3/python3-dll-a/pull/9#issuecomment-1113238455

I did try to keep the existing semantics in place and just moved things around to a Python script running in the CI. I just chose Python because I assume that the ubuntu-latest image contains a reasonably up-to-date version. (The updated CI workflow is untested, but I used the script locally the produce the definition file committed here.)

One could try using the upstream script to do the parsing and generating, but I think that should be evaluated separately.

opened by adamreichold 13
Support generating non-abi3 pythonXY.dll?
In theory we can also generate a pythonXY.dll from a .def file as long as we can obtain it somehow. Unlike stable API that has a stable_api.toml file, ~~the obvious way to obtain a .def file for pythonXY.dll is using dumpbin.exe, for example dumpbin /exports python310.dll.~~

~~Unfortunately, there seems no easy way to differentiate between function and data in the output of dumpbin /exports, it treats them all as function.~~

[x] Collect DLLs and generate DEF file on CI

[x] Use DEF files to generate pythonXY.dll import library
opened by messense 11
Customize the MinGW-w64 `dlltool` program name
Currently, the dlltool program from MinGW-w64 is set to a hard-coded constant (namely, x86_64-w64-mingw32-dlltool for the 64-bit version). Would it be feasible to allow a custom name, such as by checking for an environment variable?

The motivation here is that Fedora 37 will be adding support for the MinGW-w64 UCRT toolchain. Thus, the new ucrt64-binutils package has /usr/bin/x86_64-w64-mingw32ucrt-dlltool.

As a quick hack to check that it works, I replaced the constant and was able to successfully build a .pyd using the UCRT:

sed -i -e 's/x86_64-w64-mingw32-dlltool/x86_64-w64-mingw32ucrt-dlltool/' $HOME/.cargo/registry/src/github.com-*/python3-dll-a-0.2.5/src/lib.rs

(My first approach was to create a link from /usr/bin/x86_64-w64-mingw32ucrt-dlltool to /usr/bin/x86_64-w64-mingw32-dlltool, but for some reason it doesn't work.)
opened by richli 4
Automate `stable_abi.txt` updates
We can create a cron workflow on GitHub Actions that updates stable_abi.txt via pull request automatically.

Resources:

https://github.com/marketplace/actions/create-pull-request
opened by messense 2
Generate import library without external tools

I've take some time to port llvm-dlltool to Rust, implib-rs is a crate implements generating Windows import library from module definition file.

~~Unfortunately rust-ar doesn't support writing symbol table, so the currently it requires to run ranlib after generation to be useful. Thus it's not ready to be integrated in python3-dll-a right now. Hopefully someday we can find a solution to it.~~

I wonder how hard it is to port the mingw64 version, it'd be nice to support them both.

Update: implib crate now produces the exact same .lib file like llvm-dlltool and it works for linking with lld with the this python3-dll-a patch:

opened by messense 5

Owner

PyO3

Pythonium Trioxide

GitHub

High-level memory-safe binding generator for Flutter/Dart <-> Rust

flutter_rust_bridge: High-level memory-safe binding generator for Flutter/Dart <-> Rust Want to combine the best between Flutter, a cross-platform hot

2.1k Dec 31, 2022

Rust library for build scripts to compile C/C++ code into a Rust library

A library to compile C/C++/assembly into a Rust library/application.

1.3k Dec 21, 2022

Rust Attribute-Based Encryption library rabe's C FFI binding , support CP-ABE and KP-ABE encrypt and decrypt, submodule of Rabe.Core c# library.

Rabe-ffi Rust Attribute-Based Encryption library rabe's C FFI binding , support CP-ABE and KP-ABE encrypt and decrypt, submodule of Rabe.Core c# libra