PyOxidizer is a utility for producing binaries that embed Python

Overview

PyOxidizer

Build Status

PyOxidizer is a utility for producing binaries that embed Python. The over-arching goal of PyOxidizer is to make complex packaging and distribution problems simple so application maintainers can focus on building applications instead of toiling with build systems and packaging tools.

PyOxidizer is capable of producing a single file executable - with a copy of Python and all its dependencies statically linked and all resources (like .pyc files) embedded in the executable. You can copy a single executable file to another machine and run a Python application contained within. It just works.

PyOxidizer exposes its lower level functionality for embedding self-contained Python interpreters as a tool and software library. So if you don't want to ship executables that only consist of a Python application, you can still use PyOxidizer to e.g. produce a library containing Python suitable for linking in any application or use PyOxidizer's embedding library directly for embedding Python in a larger application.

The Oxidizer part of the name comes from Rust: executables produced by PyOxidizer are compiled from Rust and Rust code is responsible for managing the embedded Python interpreter and all its operations. If you don't know Rust, that's OK: PyOxidizer tries to make the existence of Rust nearly invisible to end-users.

While solving packaging and distribution problems is the primary goal of PyOxidizer, a side-effect of solving that problem with Rust is that PyOxidizer can serve as a bridge between these two languages. PyOxidizer can be used to easily add a Python interpreter to any Rust project. But the opposite is also true: PyOxidizer can also be used to add Rust to Python. Using PyOxidizer, you could bootstrap a new Rust project which contains an embedded version of Python and your application. Initially, your project is a few lines of Rust that instantiates a Python interpreter and runs Python code. Over time, functionality could be (re)written in Rust and your previously Python-only project could leverage Rust and its diverse ecosystem. Since PyOxidizer abstracts the Python interpreter away, this could all be invisible to end-users: you could rewrite an application from Python to Rust and people may not even know because they never see a libpython, .py files, etc.

Project Info

๐Ÿ  The official home of the PyOxidizer project is https://github.com/indygreg/PyOxidizer.

๐Ÿ“” Documentation (generated from the docs/ directory) is available at https://gregoryszorc.com/docs/pyoxidizer/main/.

๐Ÿ’ฌ The pyoxidizer-users mailing list is a forum for users to discuss all things PyOxidizer.

๐Ÿ’ฐ If you want to financially contribute to PyOxidizer, do so via GitHub Sponsors or on Patreon.

Comments
  • Build failure with 0.11.0 release on MacOS

    Build failure with 0.11.0 release on MacOS

    It seems that latest release broke building targets on MacOS.

    Context:

    • pyoxidizer 0.11.0
    • rust 1.50.0
    • macOS 10.14.6

    The build target succeed with release 0.10.3.

    Output (truncated):

    pyoxidizer build --release --target-triple=x86_64-apple-darwin
    resolving 1 targets
    resolving target install
    resolving target exe
    resolving target dist
    resolving Python distribution Url { url: "https://github.com/indygreg/python-build-standalone/releases/download/20210303/cpython-3.8.8-x86_64-apple-darwin-pgo-20210303T0937.tar.zst", sha256: "b87aba6f6c6abed1365cee5b73e120fc1beca35414b535ffe944136943bdc8a3" }
    Python distribution available at /Users/giovanni/Development/ProntoPro/cd-tool/./build/python_distributions/cpython-3.8.8-x86_64-apple-darwin-pgo-20210303T0937.tar.zst
    reading data from Python distribution...
    
    [...]
    
    building with Rust 1.50.0
        Updating crates.io index
       Compiling memchr v2.3.4
       Compiling once_cell v1.7.2
       Compiling regex-syntax v0.6.22
       Compiling cc v1.0.67
       Compiling libc v0.2.87
       Compiling fs_extra v1.2.0
       Compiling byteorder v1.4.2
       Compiling anyhow v1.0.38
       Compiling encoding_rs v0.8.28
       Compiling proc-macro-hack v0.5.19
       Compiling cfg-if v1.0.0
       Compiling autocfg v1.0.1
       Compiling lazy_static v1.4.0
       Compiling smallvec v1.6.1
       Compiling either v1.6.1
       Compiling same-file v1.0.6
       Compiling quoted_printable v0.4.2
       Compiling base64 v0.12.3
       Compiling pyembed v0.11.0
       Compiling cpython v0.5.2
       Compiling tugger-file-manifest v0.1.0
       Compiling dunce v1.0.1
       Compiling thread_local v1.1.3
       Compiling base64 v0.10.1
       Compiling walkdir v2.3.1
       Compiling itertools v0.10.0
       Compiling embed-resource v1.6.1
       Compiling num-traits v0.2.14
       Compiling aho-corasick v0.7.15
       Compiling python-packed-resources v0.5.0
       Compiling jemalloc-sys v0.3.2
       Compiling alfred v0.1.0 (/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/alfred)
       Compiling memmap v0.7.0
       Compiling paste-impl v0.1.18
       Compiling charset v0.1.2
       Compiling mailparse v0.13.2
       Compiling regex v1.4.3
       Compiling paste v0.1.18
       Compiling spdx v0.3.6
       Compiling tugger-licensing v0.1.0
       Compiling python-packaging v0.4.0
       Compiling python3-sys v0.5.2
       Compiling jemallocator v0.3.2
    error: linking with `cc` failed: exit code: 1
      |
      = note: "cc" "-m64" "-arch" "x86_64" "-L" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.0.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.1.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.10.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.11.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.2.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.3.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.4.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.5.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.6.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.7.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.8.rcgu.o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.alfred.86kylpbd-cgu.9.rcgu.o" "-o" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/alfred.4vn6ezcmupjx8p50.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps" "-L" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/release/deps" "-L" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/build/jemalloc-sys-cc97a8e57aa21db6/out/build/lib" "-L" "/Library/Developer/CommandLineTools/usr/lib/clang/10.0.1/lib/darwin" "-L" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizer-build-exe-packaging1w160a" "-L" "/Users/giovanni/Development/ProntoPro/cd-tool/./build/python_distributions/python.b87aba6f6c6a/python/build/lib" "-L" "/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/artifacts" "-L" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libjemallocator-f37943904cb00a9d.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libpyembed-b9a6fb91cb9a3657.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libmemmap-c8c285284c6a0932.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libjemalloc_sys-d9a3757db5f656db.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libdunce-c5e9b78ee82be9f9.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libcpython-e765973583055114.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libpaste-d38ce5b5784be175.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libnum_traits-cf66d0025fde9c0d.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libpython_packaging-1f0435f87ba8cf06.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libitertools-a57eec996a8cc945.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libeither-7639319a24c1ce6b.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libwalkdir-3274d4d3dec30549.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libsame_file-312b7c3f4db734a3.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libpython_packed_resources-77130fe057d6663e.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libmailparse-d81cbb87b0580c26.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libquoted_printable-76fd619fe8e7b795.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libcharset-7b45e18ee5f18914.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libencoding_rs-3eb6f7d3e915f5c3.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libcfg_if-2c09d33489315c6b.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libbase64-b1a9e17536c5510e.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libbase64-fdf9c4e974d9166a.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libtugger_licensing-402ca77557399c61.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libspdx-19c36274f2b04fa3.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libregex-3dea78d0190721fd.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libthread_local-f0ed3e184ee41f51.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libonce_cell-2677ce9b9e668579.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libregex_syntax-8cbaaf8fb94140ae.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libaho_corasick-57a5daf663859cea.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libmemchr-512039b4256b8b78.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/liblazy_static-be02bd543b1a672a.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libsmallvec-bf2bbad9b17c20a6.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libtugger_file_manifest-e421c134bef57653.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libbyteorder-79457a5cbc751dc5.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libanyhow-19c0706a53df082e.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/libpython3_sys-c0c3d622228c777c.rlib" "/private/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizervKSqLu/build/target/x86_64-apple-darwin/release/deps/liblibc-6a12c3f69f3b87c2.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libstd-1356eb4a7f983e16.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-8ae047be503d7619.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libobject-ab8fafc452749532.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libaddr2line-f75e34ad235373a3.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libgimli-ed4b8c38c97ddcd6.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-9ca625a2899d3719.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-64c7f86b25c47576.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-4d1eb83250258253.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libunwind-46e0e02619331401.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-71dbab27ef745144.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/liblibc-8425a559f49d7df9.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/liballoc-3b6462f38288b0eb.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-820a42893aa0c85f.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libcore-cfb9ec37e32acc9c.rlib" "/usr/local/Cellar/rust/1.50.0/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-36c5611c149fc184.rlib" "-framework" "AppKit" "-framework" "ApplicationServices" "-framework" "Carbon" "-framework" "CoreFoundation" "-framework" "CoreGraphics" "-framework" "CoreServices" "-framework" "IOKit" "-framework" "SystemConfiguration" "-ldl" "-lncurses" "-lclang_rt.osx" "-lpthread" "-liconv" "-lSystem" "-lresolv" "-lc" "-lm" "-rdynamic"
      = note: ld: warning: directory not found for option '-L/var/folders/k9/_w6k8j8170zggr5jpq9g4bl00000gn/T/pyoxidizer-build-exe-packaging1w160a'
              Undefined symbols for architecture x86_64:
                "___darwin_check_fd_set_overflow", referenced from:
                    _seq2set in libpyembed-b9a6fb91cb9a3657.rlib(selectmodule.o)
                    _set2list in libpyembed-b9a6fb91cb9a3657.rlib(selectmodule.o)
                    _QueueFileEvents in libpyembed-b9a6fb91cb9a3657.rlib(tclMacOSXNotify.o)
                    _Tcl_CreateFileHandler in libpyembed-b9a6fb91cb9a3657.rlib(tclMacOSXNotify.o)
                    _Tcl_DeleteFileHandler in libpyembed-b9a6fb91cb9a3657.rlib(tclMacOSXNotify.o)
                    _TclUnixWaitForFile in libpyembed-b9a6fb91cb9a3657.rlib(tclMacOSXNotify.o)
                    _NotifierThreadProc in libpyembed-b9a6fb91cb9a3657.rlib(tclMacOSXNotify.o)
                    ...
              ld: symbol(s) not found for architecture x86_64
              clang: error: linker command failed with exit code 1 (use -v to see invocation)
    
    
    error: aborting due to previous error
    

    I'm not able to test this on MacOS 11, it might be working. Also, same target on linux works.

    opened by gi0baro 34
  • Addition of Mimalloc & Snmalloc as an allocators

    Addition of Mimalloc & Snmalloc as an allocators

    I added Mimalloc as an optional feature on Windows systems since jemalloc is not really an option. Since Mimalloc was designed as a drop-in for malloc I decided the easiest route would be to make it the rust global allocator when the feature was enabled.

    opened by ryancinsight 29
  • Switch to PyO3

    Switch to PyO3

    I'm opening this PR so we have a better place to look at the code diff.

    Current state is I'm able to build a working binary with PyO3 on Linux. macOS probably works as well. Windows will need more work (and likely some changes to PyO3's build system).

    There's run-time crash with release builds on Linux. I think we were tickling an existing bug, however, as the crash is due to memory safety related to argv handling during Python interpreter initialization. It is a heisenbug and goes away if I sprinkle print statements around the suspect code (which I was doing because the stack trace was nearly worthless since a lot of functions were inlined). I may have to break out rr to debug it.

    Anyway, the build system integration is the piece of this PR that still needs the most work. The Rust code speaking to PyO3 in the pyembed crate should be pretty solid and ready for review. All tests pass against the PyO3 branch I'm running. At present time, that branch only has commits for which there are open PRs in the PyO3 project.

    CC @davidhewitt if you feel like looking at how I'm using PyO3. The main think I know I'm not doing right is I haven't implemented a GC visitor for types that contain references to other Python types. But that was a preexisting bug I think.

    opened by indygreg 17
  • Tkinter module missing in build

    Tkinter module missing in build

    Very happy to see a tool like this coming along. I'm having an issue with a small GUI application. It compiles fine. But it neither run with pyoxidizer run or as built binary.

    Traceback (most recent call last):
      File "importlib.util", line 94, in find_spec
      File "gui", line 2, in <module>
      File "tkinter", line 36, in <module>
    ModuleNotFoundError: No module named '_tkinter'
    Segmentation fault: 11
    

    This is on a Mac so I'm building x86_64-apple-darwin. I know my local python 2 seems to lack tkinter, my local python 3 does not. But from what I understand you pull other python builds. I'm guessing they don't have tkinter for some reason. Maybe not the most used extension, but quite useful to generate simple GUI.

    Let me know if I can do anything to assist with figuring out the issue.

    enhancement python-build-standalone 
    opened by lawik 16
  • Fix #337: Add __path__ filtering to OxidizedFinder's pkgutil.iter_modules support

    Fix #337: Add __path__ filtering to OxidizedFinder's pkgutil.iter_modules support

    This PR is a rewrite of and supersedes #340, and it fixes #337.

    Summary

    To make pkgutil.iter_modules(pkg.__path__) list only modules in pkg, I've added the OxidizedFinder.path_hook method, and, based on existing settings in OxidizedPythonInterpreterConfig, the OxidizedFinder instance in sys.meta_path has its bound path_hook method added to sys.path_hooks. When pkgutil.iter_modules passes pkg.__path__ to path_hook, path_hook returns an importlib.abc.PathEntryFinder object whose iter_modules method searches the OxidizedFinder instance for embedded resources whose names identify them as being in pkg.

    Review

    I think this is ready to go as long as you agree with me that none of the known bugs are blockers. All tests pass on my Mac except tests that were failing before in main at 2b4e66406f4fafdd7e09bc469878b239dcd55880. I'll run tests on Windows this weekend or early next week.

    Known bugs

    Multiple OxidizedFinder instances' path_hooks being in sys.path_hooks would interfere with each other. The first one in sys.path_hooks wins control over all paths under sys.executable.

    The PathEntryFinder that OxidizedFinder.path_hook returns does not (yet) implement legacy methods find_loader and find_module, which were deprecated in Python 3.4. They wouldn't be hard to write, but I'm having trouble convincing myself there's any point. I have confirmed that nothing in the standard library relies on those methods. If we really want those methods, I think they can be added in future PRs, and we can crib Python's implementations.

    OxidizedFinder.path_hook panics when a path contains an unpaired surrogate after the sys.executable part of the path because of dgrunwald/rust-cpython#246. I added a unittest.expectedFailure test case for a UnicodeDecodeError being raised instead.

    My algorithm for separating the sys.executable part of the path from the pkg/mod part of the path has some trash strewn about it. The worst bit is where I make a couple of allocations (.collect<PathBuf>) where really none are needed. Open to suggestions about how to tighten that up.

    UPDATE (12/22/20): Backwards incompatible change to achieve compatibility with CPython standard library

    After some further testing I noticed that pkgutil.iter_modules() (i.e., with no path argument) was behaving differently when an OxidizedFinder was present on sys.meta_path. The standard library (vanilla CPython) yields only top-level modules (e.g., importlib but not importlib.abc). Moreover, the standard library only yields the last component of modules' __name__s:

    >>> import importlib, pkgutil
    >>> [name for finder, name, ispkg in pkgutil.iter_modules(importlib.__path__)]
    ['_bootstrap', '_bootstrap_external', 'abc', 'machinery', 'metadata', 'resources', 'util']
    

    02b9c2504960b3e53ea52eb2cf4224958f818d92 modifies OxidizedFinder.iter_modules and OxidizedFinder().path_hook(path).iter_modules to conform to the standard library's behavior. The documentation says almost nothing about a finder's iter_modules method, so the standard library is our best source of expected behavior.

    opened by wkschwartz 14
  • apple-codesign: Can't disable main-binary executable segment for multiple binaries

    apple-codesign: Can't disable main-binary executable segment for multiple binaries

    Hi, thank you so much for your work on Notary support, this is super interesting. I tried it out recently from master.

    I have a xar pkg that fails notarization. The rcodesign diff-signatures option was super helpful in researching my problem. I suspect my problem is caused because my xar pkg has 6 main binaries detected out of a total 14 Mach-O files. The notarization complaint only lists The signature of the binary is invalid. for the 6 binaries that were all detected as main binary.

    I saw it's possible to control main-binary detection via the --executable-segment-flags option to rcodesign.

    • How can you specify multiple --executable-segment-flags ? I tried with multiple ordered arguments but the clap library parsing is complicated. Should it have .multiple_occurrences(true) in main.rs ?
    • How can you specify no flags at all for a SettingsScope? The FromStr for ExecutableSegmentFlags function doesn't allow empty-string or something to clear the flag. I could try with jit as a benign alternative perhaps.
    bug apple-codesign 
    opened by mappu 13
  • `cargo fetch` can't pull down deps for v0.8.0

    `cargo fetch` can't pull down deps for v0.8.0

    I'm pretty sure I'm missing something obvious, but this works at main and not at v0.8.0 (but also worked at 9b2631ce84817d177b8b0cbf16aeb2a7dab34e3f). We use cargo fetch to slurp down deps so we can satisfy requirements around non-use of the network during builds. Output:

    augie% git checkout v0.8.0 > /dev/null
    HEAD is now at 270a69e0 docs: document 0.8.0 release and link to blog post
    augie% cargo fetch
        Updating crates.io index
    error: failed to select a version for `pyembed`.
        ... required by package `oxidized-importer v0.8.0 (/Users/augie/Programming/pyoxidizer/oxidized-importer)`
    versions that meet the requirements `^0.8.0` are: 0.8.0
    
    the package `pyembed` links to the native library `pythonXY`, but it conflicts with a previous package which links to `pythonXY` as well:
    package `pyembed v0.8.0 (/Users/augie/Programming/pyoxidizer/pyembed)`
    
    failed to select a version for `pyembed` which could resolve this conflict
    

    I'm confused because the pythonXY string doesn't show up many places and there don't seem to be any materially interesting changes around that between the tag and main. I'm happy to work on this, but I need some pointers to the right docs (either in pyoxidizer's code or in some Cargo docs) to get un-stuck. Thanks!

    bug 
    opened by durin42 12
  • Cannot build app

    Cannot build app

    Build: geadda2d

    ...
    PyOxidizer does not set __file__ and this may create problems at run-time
    See https://github.com/indygreg/PyOxidizer/issues/69 for more
    package README.txt does not exist; excluding resources: ["README.txt"]
    package bin does not exist; excluding resources: ["chardetect.exe"]
    deriving custom importlib modules to support in-memory importing
    building with Rust 1.42.0-nightly
        Updating crates.io index
    error: no matching package named `pyembed` found
    location searched: registry `https://github.com/rust-lang/crates.io-index`
    ...
    
    bug 
    opened by BiatuAutMiahn 12
  • multiphase initialisation suport of pyoxidizer

    multiphase initialisation suport of pyoxidizer

    Hi,

    This is a reduced example to the issue https://github.com/indygreg/PyOxidizer/issues/489

    Thanks to @rgommers to point me to the right direction

    I made a minimal example showing how cython modules can break pyoxidizer single file executables on windows.

    https://github.com/franzhaas/pyoxidizer_cython_example

    when importing fib, the repl crashes.

    versions used are documented in pipenv.lock.

    I am new to pyoxidizer, and would apreciate if I could get some guidnace how to tackle this. Is this a regression? Has that been working with older versions?

    I do have access to working pyd files, which have been build out of hand crafted c code and will have a look at the difference between this and the cython c code, but this is likely not to work out....

    Thanks for the grate tool!

    bug 
    opened by franzhaas 9
  • Issue with Python distribution on Windows 10 with default init pyapp.

    Issue with Python distribution on Windows 10 with default init pyapp.

    Hi

    Firstly, thanks for this great tool - it's exactly what I've wanted to join the worlds of Rust and Python and I can't wait to get it going!

    pyoxidizer itself builds fine, both via crate and the github master build, but I'm having an issue with the stand-alone python aspect on Windows 10 it sems. Here's the logs trying to build the default in init pyapp example.

    ...pyapp>pyoxidizer build
    no existing PyOxidizer artifacts found
    processing config file ...pyapp\pyoxidizer.toml
    resolving Python distribution...
    downloading https://github.com/indygreg/python-build-standalone/releases/download/20190617/cpython-3.7.3-windows-amd64-20190618T0516.tar.zst
    Python distribution available at ...pyapp\build\target\x86_64-pc-windows-msvc\debug\pyoxidizer\cpython-3.7.3-windows-amd64-20190618T0516.tar.zst
    reading data from Python distribution...
    thread 'main' panicked at 'unable to extract tar archive: Custom { kind: NotFound, error: TarError { desc: "failed to unpack `...pyapp\\build\\target\\x86_64-pc-windows-msvc\\debug\\pyoxidizer\\python.fd43554b5654\\python\\install\\Lib\\site-packages\\pip-19.1.1-py3.7.egg\\pip\\_vendor\\urllib3\\contrib\\_securetransport\\__pycache__\\low_level.cpython-37.pyc`", io: Custom { kind: NotFound, error: TarError { desc: "failed to unpack `python/install/Lib/site-packages/pip-19.1.1-py3.7.egg/pip/_vendor/urllib3/contrib/_securetransport/_` into `...pyapp\\build\\target\\x86_64-pc-windows-msvc\\debug\\pyoxidizer\\python.fd43554b5654\\python\\install\\Lib\\site-packages\\pip-19.1.1-py3.7.egg\\pip\\_vendor\\urllib3\\contrib\\_securetransport\\__pycache__\\low_level.cpython-37.pyc`", io: Os { code: 3, kind: NotFound, message: "The system cannot find the path specified." } } } } }', src\libcore\result.rs:997:5
    note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
    
    

    Backtrace enabled Version:

    pyoxidizer build
    no existing PyOxidizer artifacts found
    processing config file ...pyapp\pyoxidizer.toml
    resolving Python distribution...
    Python distribution available at ...pyapp\build\target\x86_64-pc-windows-msvc\debug\pyoxidizer\cpython-3.7.3-windows-amd64-20190618T0516.tar.zst
    reading data from Python distribution...
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "unable to read Python license"', src\libcore\result.rs:997:5
    stack backtrace:
       0: git_packbuilder_object_count
       1: git_packbuilder_object_count
       2: git_packbuilder_object_count
       3: git_packbuilder_object_count
       4: git_packbuilder_object_count
       5: git_packbuilder_object_count
       6: git_packbuilder_object_count
       7: <unknown>
       8: <unknown>
       9: <unknown>
      10: <unknown>
      11: <unknown>
      12: <unknown>
      13: <unknown>
      14: <unknown>
      15: git_packbuilder_object_count
      16: git_packbuilder_object_count
      17: git_packbuilder_object_count
      18: <unknown>
      19: git_libgit2_version
      20: BaseThreadInitThunk
      21: RtlUserThreadStart
    

    Any tips or pointers on this would be appreciated. Thanks!

    opened by McSpidey 9
  • Panic signing a particular executable file

    Panic signing a particular executable file

    It seems like this particular binary causes apple-codesign to break for some reason:

    signing cross-x86_64-centos-7-0.9.0-macos-aarch64/libexec/gcc/x86_64-linux-gnu/11.2.0/cc1plus in place
    signing cross-x86_64-centos-7-0.9.0-macos-aarch64/libexec/gcc/x86_64-linux-gnu/11.2.0/cc1plus as a Mach-O binary
    inferring default signing settings from Mach-O binary
    preserving existing binary identifier in Mach-O
    preserving code signature flags in existing Mach-O signature
    setting binary identifier to cc1plus
    parsing Mach-O
    signing Mach-O binary at index 0
    thread 'main' panicked at 'assertion failed: segment.fileoff == 0 || segment.fileoff == cursor.position()', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/apple-codesign-0.16.0/src/macho_signing.rs:166:9
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    

    Here's the offending binary in a tarball: cc1plus.tar.gz

    bug apple-codesign 
    opened by sfackler 8
  • Unable build project with aiohttp and asyncio

    Unable build project with aiohttp and asyncio

    I tried to build the same project on Windows and Linux, without any success. I tried to build with pyoxidizer from pip and pyoxidizer build from cargo(Linux). I got the same error.

    Bazzel config was generated by pyoxidizer init-config-file ProjectA and I uncommented only exe.add_python_resources(exe.pip_install(["-r", "requirements.txt"]))

    /home/m.krawczuk/.cargo/bin/pyoxidizer run
    resolving 1 targets
    resolving target install
    resolving target exe
    target Python distribution for x86_64-unknown-linux-gnu resolves to: https://github.com/indygreg/python-build-standalone/releases/download/20220528/cpython-3.10.4%2B20220528-x86_64-unknown-linux-gnu-pgo-full.tar.zst (sha256=ab27c27d034da39e999efce6bfa416be645918f27c13e89be9794dea0ef2f116)
    reusing target Python distribution for host execution
    installing modified distutils to /tmp/pyoxidizer-pip-installSEd957/packages
    modifying distutils/_msvccompiler.py for oxidation
    modifying distutils/command/build_ext.py for oxidation
    modifying distutils/unixccompiler.py for oxidation
    pip installing to /tmp/pyoxidizer-pip-installSEd957/install
    Collecting qasync~=0.23.0
      Using cached qasync-0.23.0-py3-none-any.whl (13 kB)
    Collecting PyQT6~=6.3.0
      Using cached PyQt6-6.3.1-cp37-abi3-manylinux1_x86_64.whl (7.8 MB)
    Collecting asyncio~=3.4.3
      Using cached asyncio-3.4.3-py3-none-any.whl (101 kB)
    Collecting aiohttp
      Using cached aiohttp-3.8.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.2 MB)
    Collecting httpx
      Using cached httpx-0.23.0-py3-none-any.whl (84 kB)
    Collecting uplink~=0.9.7
      Using cached uplink-0.9.7-py2.py3-none-any.whl (69 kB)
    Collecting uplink-httpx
      Using cached uplink_httpx-2.0-py3-none-any.whl (4.5 kB)
    Collecting numpy
      Using cached numpy-1.23.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
    Collecting pandas~=1.4.2
      Using cached pandas-1.4.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
    Collecting openpyxl
      Using cached openpyxl-3.0.10-py2.py3-none-any.whl (242 kB)
    Collecting pip>=22.1.2
      Using cached pip-22.2.2-py3-none-any.whl (2.0 MB)
    Collecting requests~=2.27.1
      Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
    Collecting keyring~=23.6.0
      Using cached keyring-23.6.0-py3-none-any.whl (34 kB)
    Collecting PyQt6-Qt6>=6.3.0
      Using cached PyQt6_Qt6-6.3.1-py3-none-manylinux_2_28_x86_64.whl (52.4 MB)
    Collecting PyQt6-sip<14,>=13.4
      Using cached PyQt6_sip-13.4.0-cp310-cp310-manylinux1_x86_64.whl (310 kB)
    Collecting frozenlist>=1.1.1
      Using cached frozenlist-1.3.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB)
    Collecting async-timeout<5.0,>=4.0.0a3
      Using cached async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
    Collecting charset-normalizer<3.0,>=2.0
      Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
    Collecting multidict<7.0,>=4.5
      Using cached multidict-6.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
    Collecting aiosignal>=1.1.2
      Using cached aiosignal-1.2.0-py3-none-any.whl (8.2 kB)
    Collecting yarl<2.0,>=1.0
      Using cached yarl-1.8.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (263 kB)
    Collecting attrs>=17.3.0
      Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
    Collecting certifi
      Using cached certifi-2022.9.14-py3-none-any.whl (162 kB)
    Collecting rfc3986[idna2008]<2,>=1.3
      Using cached rfc3986-1.5.0-py2.py3-none-any.whl (31 kB)
    Collecting sniffio
      Using cached sniffio-1.3.0-py3-none-any.whl (10 kB)
    Collecting httpcore<0.16.0,>=0.15.0
      Using cached httpcore-0.15.0-py3-none-any.whl (68 kB)
    Collecting uritemplate>=3.0.0
      Using cached uritemplate-4.1.1-py2.py3-none-any.whl (10 kB)
    Collecting six>=1.13.0
      Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
    Collecting httpx
      Using cached httpx-1.0.0b0-py3-none-any.whl (82 kB)
    Collecting pytz>=2020.1
      Using cached pytz-2022.2.1-py2.py3-none-any.whl (500 kB)
    Collecting python-dateutil>=2.8.1
      Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
    Collecting et-xmlfile
      Using cached et_xmlfile-1.1.0-py3-none-any.whl (4.7 kB)
    Collecting urllib3<1.27,>=1.21.1
      Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
    Collecting charset-normalizer<3.0,>=2.0
      Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
    Collecting idna<4,>=2.5
      Using cached idna-3.4-py3-none-any.whl (61 kB)
    Collecting jeepney>=0.4.2
      Using cached jeepney-0.8.0-py3-none-any.whl (48 kB)
    Collecting SecretStorage>=3.2
      Using cached SecretStorage-3.3.3-py3-none-any.whl (15 kB)
    Collecting h11<0.13,>=0.11
      Using cached h11-0.12.0-py3-none-any.whl (54 kB)
    Collecting anyio==3.*
      Using cached anyio-3.6.1-py3-none-any.whl (80 kB)
    Collecting cryptography>=2.0
      Using cached cryptography-38.0.1-cp36-abi3-manylinux_2_28_x86_64.whl (4.2 MB)
    Collecting cffi>=1.12
      Using cached cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
    Collecting pycparser
      Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
    Installing collected packages: rfc3986, pytz, PyQt6-Qt6, asyncio, urllib3, uritemplate, sniffio, six, qasync, PyQt6-sip, pycparser, pip, numpy, multidict, jeepney, idna, h11, frozenlist, et-xmlfile, charset-normalizer, certifi, attrs, async-timeout, yarl, requests, python-dateutil, PyQT6, openpyxl, cffi, anyio, aiosignal, uplink, pandas, httpcore, cryptography, aiohttp, SecretStorage, httpx, uplink-httpx, keyring
    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    xdis 6.0.4 requires click, which is not installed.
    requests-oauthlib 1.3.1 requires oauthlib>=3.0.0, which is not installed.
    pynsist 2.8 requires jinja2, which is not installed.
    jira 3.3.1 requires packaging, which is not installed.
    decompyle3 3.9.0 requires click, which is not installed.
    Successfully installed PyQT6-6.3.1 PyQt6-Qt6-6.3.1 PyQt6-sip-13.4.0 SecretStorage-3.3.3 aiohttp-3.8.1 aiosignal-1.2.0 anyio-3.6.1 async-timeout-4.0.2 asyncio-3.4.3 attrs-22.1.0 certifi-2022.9.14 cffi-1.15.1 charset-normalizer-2.0.12 cryptography-38.0.1 et-xmlfile-1.1.0 frozenlist-1.3.1 h11-0.12.0 httpcore-0.15.0 httpx-0.23.0 idna-3.4 jeepney-0.8.0 keyring-23.6.0 multidict-6.0.2 numpy-1.23.3 openpyxl-3.0.10 pandas-1.4.4 pip-22.2.2 pycparser-2.21 python-dateutil-2.8.2 pytz-2022.2.1 qasync-0.23.0 requests-2.27.1 rfc3986-1.5.0 six-1.16.0 sniffio-1.3.0 uplink-0.9.7 uplink-httpx-2.0 uritemplate-4.1.1 urllib3-1.26.12 yarl-1.8.1
    adding Python executable ProjectA to .
    ensuring Rust toolchain 1.61.0 is available
    rustc for x86_64-unknown-linux-gnu in /home/m.krawczuk/.cache/pyoxidizer/rust/1.61.0-x86_64-unknown-linux-gnu is up-to-date
    cargo for x86_64-unknown-linux-gnu in /home/m.krawczuk/.cache/pyoxidizer/rust/1.61.0-x86_64-unknown-linux-gnu is up-to-date
    rust-std for x86_64-unknown-linux-gnu in /home/m.krawczuk/.cache/pyoxidizer/rust/1.61.0-x86_64-unknown-linux-gnu is up-to-date
         Created binary (application) package
    writing /tmp/pyoxidizer3QmZay/ProjectA/.cargo/config
    writing /tmp/pyoxidizer3QmZay/ProjectA/Cargo.lock
    writing /tmp/pyoxidizer3QmZay/ProjectA/build.rs
    writing /tmp/pyoxidizer3QmZay/ProjectA/src/main.rs
    writing /tmp/pyoxidizer3QmZay/ProjectA/pyoxidizer.bzl
    writing /tmp/pyoxidizer3QmZay/ProjectA/ProjectA.exe.manifest
    writing /tmp/pyoxidizer3QmZay/ProjectA/ProjectA-manifest.rc
    error[PYOXIDIZER_PYTHON_EXECUTABLE]: adding PythonExecutable to FileManifest
        
        Caused by:
            0: building Python executable
            1: building executable with Rust project
            2: obtaining embedded python context
            3: converting asyncio.base_events to resource
            4: compiling relative path module bytecode
            5: compiling error: invalid syntax (asyncio.base_events, line 296)
           --> ./pyoxidizer.bzl:283:5
            |
        283 |     files.add_python_resource(".", exe)
            |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PythonExecutable.to_file_manifest()
        
        
    error: adding PythonExecutable to FileManifest
    
    Caused by:
        0: building Python executable
        1: building executable with Rust project
        2: obtaining embedded python context
        3: converting asyncio.base_events to resource
        4: compiling relative path module bytecode
        5: compiling error: invalid syntax (asyncio.base_events, line 296)
    
    opened by MateuszKrawczuk 0
  • Memory loading issue possible workaround

    Memory loading issue possible workaround

    Hi, i've recently came across the memory loading issue like in the issue (438). I am also using pydantic(1.10.2). After I tried around for some time I tried using pydantic from github instead of Pypi. In that case, the error extension module pydantic cannot be loaded from memory but memory loading required won't come anymore.

    Now the main question is, why there is a difference between using the pre-build wheels from Pypi and using the github repo and building it itself. The problem is, that we want to use pyoxidizer to build independent executables without the requirement to install additional libs (at least for linux). I've made a demo repo to easily reproduce this error. There's also a little more documentation on how to reproduce this error.

    I hope this is the right place for this issue because I figured, this has to be an issue with how pip is used to installs the wheel, but that's just guess.

    opened by henningWoehr 0
  • init_fs_encoding fails when include_distribution_sources = False

    init_fs_encoding fails when include_distribution_sources = False

    Hey :-) When I set include_distribution_sources = False, the interpreter fails to start:

    $ pyoxidizer run --release --target-triple i686-pc-windows-msvc
    [ ... ]
    Python path configuration:
      PYTHONHOME = 'C:\w\build\i686-pc-windows-msvc\release\install\foo'
      PYTHONPATH = (not set)
      program name = 'C:\w\build\i686-pc-windows-msvc\release\install\foo\foo.exe'
      isolated = 1
      environment = 0
      user site = 0
      import site = 1
      sys._base_executable = 'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo\\foo.exe'
      sys.base_prefix = 'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo'
      sys.base_exec_prefix = 'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo'
      sys.platlibdir = 'lib'
      sys.executable = 'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo\\foo.exe'
      sys.prefix = 'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo'
      sys.exec_prefix = 'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo'
      sys.path = [
        'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo\\python310.zip',
        'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo\\DLLs',
        'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo\\lib',
        'C:\\w\\build\\i686-pc-windows-msvc\\release\\install\\foo',
      ]
    error instantiating embedded Python interpreter: during initializing Python main: init_fs_encoding: failed to get the Python codec of the filesystem encoding
    error: cargo run failed
    

    my pyoxidizer.bzl is:

    def make_exe():
        dist = default_python_distribution()
        policy = dist.make_python_packaging_policy()
        policy.include_distribution_sources = False
        python_config = dist.make_python_interpreter_config()
        exe = dist.to_python_executable(
            name="foo",
            config=python_config,
            packaging_policy=policy,
        )
        return exe
    
    def make_embedded_resources(exe):
        return exe.to_embedded_resources()
    
    def make_install(exe):
        files = FileManifest()
        files.add_python_resource("foo", exe)
        return files
    
    register_target("exe", make_exe)
    register_target("resources", make_embedded_resources, depends=["exe"], default_build_script=True)
    register_target("install", make_install, depends=["exe"], default=True)
    resolve_targets()
    

    At first I thought it was due to some other specifics of my bzl (static build, py3.8 for win7 support, windows_runtime_dlls_mode=never) but it still happens after simplifying it to this.

    My assumption was that it would compile the source modules to bytecode and include those into the exe instead; maybe I'm mistaken? Seeing the PYTHONHOME in the output, is the intention to provide your own copy of the stdlib, similar to the zipfile in the official cpython windows-embedded-packages?

    opened by 9001 0
  • unknown option --multiprocessing-fork

    unknown option --multiprocessing-fork

    unknown option --multiprocessing-fork Try `python -h' for more information. error instantiating embedded Python interpreter: during initializing Python core: could not format PyStatus

    What usually causes this error? The same build and configuration works fine on Linux and Mac.

    opened by LyronFoster 0
  • Add support for MSI start menu shortcut

    Add support for MSI start menu shortcut

    WiX Toolset has the capabilities of adding a shortcut to the Start Menu on Windows, but I don't see any option to include this during building.

    This would add a lot of value to installations where the user is trying to find the program they just installed on Windows.

    WiX Toolset - How To: Create a Shortcut on the Start Menu

    opened by AceofSpades5757 0
  • init_fs_encoding on google/cloud-sdk:slim docker image

    init_fs_encoding on google/cloud-sdk:slim docker image

    I have managed to get my fairly complicated python application to build even though it contains pandas and numpy. However I noticed that whenever I try and run it on google/cloud-sdk:slim docker image it fails. I traced it down to even hello world application failing without any error other than below

    error instantiating embedded Python interpreter: during initializing Python main: init_fs_encoding: failed to get the Python codec of the filesystem encoding
    

    Dockerfile.txt Here is a docker file that builds a helloworld application using python 3.8 on the manylinux build docker and then runs it inside google/cloud-sdk:slim as a minimal repeatable error. I have not tried other python versions yet or other build instances nor using musl as I cannot make standalones with numpy

    opened by whinis 0
Releases(apple-codesign/0.17.0)
Owner
Gregory Szorc
Gregory Szorc
My solutions for the Advent of Code 2021 in Scala, Python, Haskell and Rust.

Advent of Code 2021 These are my Advent of Code 2021 solutions written in Scala 3, Haskell, Python and Rust. Day Title L1 L2 L3 L4 01 Sonar Sweep Scal

Axel Suรกrez 2 Aug 9, 2022
Fast DNA manipulation for Python, written in Rust.

quickdna Quickdna is a simple, fast library for working with DNA sequences. It is up to 100x faster than Biopython for some translation tasks, in part

Secure DNA 14 Sep 16, 2022
Songbird bindings for python

Songbird-Py Songbird bindings for python. The goal is to provide an easy to use alternitive to Lavalink. Its written with rust-bindings to Songbird. S

null 0 Jul 23, 2022
A toy example showing how to run Rust code in Python for speed and progress.

PoC: Integrating Rust in Python A toy example showing how to run Rust code in Python for speed and progress. Requirements Python 3.6+ Rust 1.44+ Cargo

Emil Thorenfeldt 2 Feb 7, 2022
This is a simple command line application to convert bibtex to json written in Rust and Python

bibtex-to-json This is a simple command line application to convert bibtex to json written in Rust and Python. Why? To enable you to convert very big

null 3 Mar 23, 2022
Rust implementation of Python command line progress bar tool tqdm.

tqdm Rust implementation of Python command line progress bar tool tqdm. From original documentation: tqdm derives from the Arabic word taqaddum (ุชู‚ุฏู‘ู…

Yiheng Du 6 Sep 12, 2022
Python/Rust implementations and notes from Proofs Arguments and Zero Knowledge study group

What is this? This is where I'll be collecting resources related to the Study Group on Dr. Justin Thaler's Proofs Arguments And Zero Knowledge Book. T

Thor 57 Sep 21, 2022
๐ŸŽ A fast implementation of the Aho-Corasick algorithm using the compact double-array data structure. (Python wrapper for daachorse)

python-daachorse daachorse is a fast implementation of the Aho-Corasick algorithm using the compact double-array data structure. This is a Python wrap

Koichi Akabe 3 Jun 5, 2022
A Command-line tool to create, manage and deploy your python projects

PPM A Command-line tool to create, manage and deploy your python projects Table of Contents PPM Main Features Create a Project project.ini file Projec

FUSEN 6 Aug 30, 2022
Fuzzy Index for Python, written in Rust. Works like error-tolerant dict, keyed by a human input.

FuzzDex FuzzDex is a fast Python library, written in Rust. It implements an in-memory fuzzy index that works like an error-tolerant dictionary keyed b

Tomasz bla Fortuna 8 Sep 15, 2022
A fast python geohash library created by wrapping rust.

Pygeohash-Fast A Fast geohasher for python. Created by wrapping the rust geohash crate with pyo3. Huge shout out to the georust community :) Currently

Zach Paden 3 Aug 18, 2022
Python PEP-440 Version Parsing

PyVer (WIP) Python PEP-440 Version Parsing This package allows for parsing Python PEP-440 version numbers and comparisons between PEP-440 Versions Usa

Allstreamer 3 Sep 6, 2022
A CLI utility to secretly copy secrets to your clipboard. ๐Ÿฆ€

seclip ?? ?? A CLI utility to secretly copy secrets to your clipboard. ?? Table of Contents Features Installation Usage Build From Source Contribution

Mufeed VH 32 May 22, 2022
mdBook is a utility to create modern online books from Markdown files.

Create book from markdown files. Like Gitbook but implemented in Rust

The Rust Programming Language 10.7k Sep 19, 2022
A CLI utility installed as "ansi" to quickly get ANSI escape sequences. Supports the most basic ones, like colors and styles as bold or italic.

'ansi' - a CLI utility to quickly get ANSI escape codes This Rust project called ansi-escape-sequences-cli provides an executable called ansi which ca

Philipp Schuster 5 Jul 28, 2022
fixred is a command line utility to fix outdated links in files with redirect URLs.

fixred fixred is a command line utility to fix outdated links in files with redirect URLs. Installation fixred is installed via cargo package manager.

Linda_pp 35 Aug 6, 2022
Rust command line utility to quickly display useful secrets in a Kubernetes namespace

kube-secrets This is a command line utility for quickly looking at secrets in a Kubernetes namespace that are typically looked at by humans. It specif

Frank Wiles 8 Feb 10, 2022
A command-line utility that creates project structure.

petridish A command-line utility that creates project structure. If you have heard of the cookiecutter project, petridish is a rust implementation of

null 11 Jun 26, 2022
Command line utility for controlling LIFX smart lights

lifxc is a command line utility for controlling LIFX smart lights. Currently, communication over the LIFX LAN protocol is supported.

Harrison Rigg 1 Nov 17, 2021