Thanks for the excellent bindings, we have had great success using them. Do you know when tensorflow 2 saved models will be supported?

I have inquired in other channels about the stability of the tensorflow 2.0 C API but not gotten a response.

In my own project, I am using Rust to write TFRecords full of TFExamples to disk.

The code for doing this is something that might be useful to many projects that import the tensorflow crate, and I'd like to submit a PR for these utils.

However, it would be nice to have an agreement from package maintainers on the specifics of the design before I code up the CL. To give an example of outstanding questions:

1. Do you even want RecordWriter and/or ExampleWriter in this crate?
2. Namespace? By analogy to the Python API, what about tensorflow::io?
3. File structure? src/io.rs?

Here is the output

$ cargo build -j 1
    Updating registry `https://github.com/rust-lang/crates.io-index`
   Compiling libc v0.2.17
   Compiling pkg-config v0.3.8
   Compiling tensorflow-sys v0.5.0 (file:///mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys)
error: failed to run custom build command for `tensorflow-sys v0.5.0 (file:///mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys)`
process didn't exit successfully: `/mnt/c/Users/tyoc/test/nada/rust/target/debug/build/tensorflow-sys-5a768d9e18f318a5/build-script-build` (exit code: 101)
--- stdout
libtensorflow-sys/build.rs:31: output = "/mnt/c/Users/tyoc/test/nada/rust/target/debug/build/tensorflow-sys-5a768d9e18f318a5/out"
libtensorflow-sys/build.rs:33: source = "/mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys/target/source-0.10.0"
libtensorflow-sys/build.rs:35: lib_dir = "/mnt/c/Users/tyoc/test/nada/rust/target/debug/build/tensorflow-sys-5a768d9e18f318a5/out/lib-0.10.0"
libtensorflow-sys/build.rs:39: Creating directory "/mnt/c/Users/tyoc/test/nada/rust/target/debug/build/tensorflow-sys-5a768d9e18f318a5/out/lib-0.10.0"
libtensorflow-sys/build.rs:43: library_path = "/mnt/c/Users/tyoc/test/nada/rust/target/debug/build/tensorflow-sys-5a768d9e18f318a5/out/lib-0.10.0/libtensorflow_c.so"
libtensorflow-sys/build.rs:48: target_path = "tensorflow/libtensorflow_c.so"
libtensorflow-sys/build.rs:102: Executing "git" "clone" "--branch=v0.10.0" "--recursive" "https://github.com/tensorflow/tensorflow.git" "/mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys/target/source-0.10.0"
libtensorflow-sys/build.rs:106: Command "git" "clone" "--branch=v0.10.0" "--recursive" "https://github.com/tensorflow/tensorflow.git" "/mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys/target/source-0.10.0" finished successfully
libtensorflow-sys/build.rs:77: Checking build file "/mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys/target/source-0.10.0/tensorflow/BUILD"
libtensorflow-sys/build.rs:84: Patching build file "/mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys/target/source-0.10.0/tensorflow/BUILD"
libtensorflow-sys/build.rs:102: Executing "./configure"
No Google Cloud Platform support will be enabled for TensorFlow
No GPU support will be enabled for TensorFlow
Configuration finished
libtensorflow-sys/build.rs:106: Command "./configure" finished successfully
libtensorflow-sys/build.rs:102: Executing "bazel" "build" "--jobs=1" "--compilation_mode=opt" "tensorflow:libtensorflow_c.so"

--- stderr
Clonar en «/mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys/target/source-0.10.0»...
Note: checking out 'c715c3102df1556fc0ce88fc987440a3c80e5380'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

Checking out files: 100% (4084/4084), done.
Can't do inplace edit on tensorflow/core/platform/default/build_config.bzl: No existe el archivo o el directorio.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os { code: 2, message: "No such file or directory" } }', ../src/libcore/result.rs:788
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Here is the backtrace (even that is pretty clear)

--- stderr
Can't open tensorflow/core/platform/default/build_config.bzl: No existe el archivo o el directorio.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os { code: 2, message: "No such file or directory" } }', ../src/libcore/result.rs:788
stack backtrace:
   1:     0x7fe71b839739 - std::sys::backtrace::tracing::imp::write::hd4b54a4a2078cb15
   2:     0x7fe71b8403ec - std::panicking::default_hook::_{{closure}}::h51a5ee7ba6a9fcef
   3:     0x7fe71b83f6e9 - std::panicking::default_hook::hf823fce261e27590
   4:     0x7fe71b83fd28 - std::panicking::rust_panic_with_hook::h8d486474663979b9
   5:     0x7fe71b83fb82 - std::panicking::begin_panic::h72862f004a4942ab
   6:     0x7fe71b83faf0 - std::panicking::begin_panic_fmt::hdc424a357d9142e1
   7:     0x7fe71b83fa71 - rust_begin_unwind
   8:     0x7fe71b8762cf - core::panicking::panic_fmt::h6b06f78ae7f9dd57
   9:     0x7fe71b80fb9c - core::result::unwrap_failed::hca88b4a09ab2a5f1
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libcore/result.rs:29
  10:     0x7fe71b80c005 - _<core..result..Result<T, E>>::unwrap::h59f218bad74f813b
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libcore/result.rs:726
  11:     0x7fe71b816545 - build_script_build::run::h1436e6466b486231
                        at /mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys/build.rs:17
  12:     0x7fe71b815249 - build_script_build::main::h84db52058401dfc4
                        at /mnt/c/Users/tyoc/test/nada/rust/tensorflow-sys/build.rs:58
  13:     0x7fe71b847ed6 - __rust_maybe_catch_panic
  14:     0x7fe71b83ee62 - std::rt::lang_start::hca48e539ce72a288
  15:     0x7fe71b817603 - main
  16:     0x7fe71aa01f44 - __libc_start_main
  17:     0x7fe71b8072d8 - <unknown>
  18:                0x0 - <unknown>

Also would prefer the name of the repo be some like tensorflow-rust so when you clone with guthub app doesnt overwrite rust repo :).

This issue was automatically created by Allstar.

Security Policy Violation Project is out of compliance with Binary Artifacts policy: binaries present in source code

Rule Description Binary Artifacts are an increased security risk in your repository. Binary artifacts cannot be reviewed, allowing the introduction of possibly obsolete or maliciously subverted executables. For more information see the Security Scorecards Documentation for Binary Artifacts.

Remediation Steps To remediate, remove the generated executable artifacts from the repository.

Artifacts Found

• test_resources/library/linux/test_op.so
• test_resources/library/macos/test_op.so

Additional Information This policy is drawn from Security Scorecards, which is a tool that scores a project's adherence to security best practices. You may wish to run a Scorecards scan directly on this repository for more details.

Allstar has been installed on all Google managed GitHub orgs. Policies are gradually being rolled out and enforced by the GOSST and OSPO teams. Learn more at http://go/allstar

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Running cargo test in the tensorflow-sys directory fails (but tests pass for the main crate).

Here's the output:

 -> cd ~/tensorflow_rust.git/tensorflow-sys
 -> cargo test
    Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
     Running target/debug/deps/tensorflow-9f5d1dac7e952430

running 12 tests
test buffer::tests::basic ... ok
test session::tests::smoke ... ok
test session::tests::test_close ... ok
test graph::tests::smoke ... ok
test tests::smoke ... ok
test tests::test_close ... ok
test tests::test_extend_graph ... ok
test session::tests::test_run ... ok
test tests::test_set_config ... ok
test tests::test_set_target ... ok
test tests::test_tensor ... ok
test tests::test_run ... ok

test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured

   Doc-tests tensorflow

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured

 -> cd tensorflow-sys/
 -> cargo test --verbose
       Fresh lazy_static v0.2.4
       Fresh regex-syntax v0.3.9
       Fresh pkg-config v0.3.9
       Fresh utf8-ranges v0.1.3
       Fresh libc v0.2.21
       Fresh winapi v0.2.8
       Fresh winapi-build v0.1.1
       Fresh memchr v0.1.11
       Fresh aho-corasick v0.5.3
       Fresh kernel32-sys v0.2.2
       Fresh thread-id v2.0.0
       Fresh thread_local v0.2.7
       Fresh regex v0.1.80
       Fresh semver-parser v0.6.2
       Fresh semver v0.5.1
       Fresh tensorflow-sys v0.7.0 (file:///home/nbigaouette/tensorflow_rust.git/tensorflow-sys)
    Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `/home/nbigaouette/tensorflow_rust.git/target/debug/deps/lib-a517463cab98ea9f`

running 1 test
test linkage ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured

     Running `/home/nbigaouette/tensorflow_rust.git/target/debug/deps/tensorflow_sys-b18d6c19e08d67bd`
terminate called without an active exception
error: process didn't exit successfully: `/home/nbigaouette/tensorflow_rust.git/target/debug/deps/tensorflow_sys-b18d6c19e08d67bd` (signal: 6, SIGABRT: process abort signal)

Caused by:
  process didn't exit successfully: `/home/nbigaouette/tensorflow_rust.git/target/debug/deps/tensorflow_sys-b18d6c19e08d67bd` (signal: 6, SIGABRT: process abort signal)

Here's a backtrace from running /home/nbigaouette/tensorflow_rust.git/target/debug/deps/tensorflow_sys-b18d6c19e08d67bd through gdb:

Program received signal SIGABRT, Aborted.
0x00007ffff32f904f in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff32f904f in raise () from /usr/lib/libc.so.6
#1  0x00007ffff32fa47a in abort () from /usr/lib/libc.so.6
#2  0x00007ffff2ccb4ed in __gnu_cxx::__verbose_terminate_handler () at /build/gcc/src/gcc/libstdc++-v3/libsupc++/vterminate.cc:95
#3  0x00007ffff2cc92a6 in __cxxabiv1::__terminate (handler=<optimized out>) at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:47
#4  0x00007ffff2cc92f1 in std::terminate () at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:57
#5  0x00007ffff2cc8062 in __cxxabiv1::__cxa_allocate_exception (thrown_size=136, thrown_size@entry=8) at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_alloc.cc:278
#6  0x00007ffff2cc9a98 in operator new (sz=32) at /build/gcc/src/gcc/libstdc++-v3/libsupc++/new_op.cc:54
#7  0x00007ffff57ab4c8 in tensorflow::monitoring::Counter<2>* tensorflow::monitoring::Counter<2>::New<char const (&) [46], char const (&) [58], char const (&) [11], char const (&) [7]>(char const (&) [46], char c
onst (&) [58], char const (&) [11], char const (&) [7]) () from /usr/lib/libtensorflow.so
#8  0x00007ffff571a50b in ?? () from /usr/lib/libtensorflow.so
#9  0x00007ffff7de94fa in call_init.part () from /lib64/ld-linux-x86-64.so.2
#10 0x00007ffff7de960b in _dl_init () from /lib64/ld-linux-x86-64.so.2
#11 0x00007ffff7ddadaa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#12 0x0000000000000001 in ?? ()
#13 0x00007fffffffe846 in ?? ()
#14 0x0000000000000000 in ?? ()

The library should be able to generate computation graphs itself, rather than relying on Tensorflow's Python library. We need a way to generate version 3 protobufs, though. The obvious solution, https://github.com/stepancheg/rust-protobuf, does not support proto3.

When compiling on Windows while using a gnu toolchain, the tensorflow-sys build file will try to download a file for Windows but with Unix compression. It happens here, when choosing the file extension:

https://github.com/tensorflow/rust/blob/c1a1f6bb0cea568ac301d7c58acfab797df18d63/tensorflow-sys/build.rs#L160-L163

These line assume that msvc will be Windows, and gnu Unix, however having a gnu toolchain activated on a Windows machine causes the build script to download the file https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-gpu-windows-x86_64-1.15.0.tar.gz (notice the "windows" associated with ".tar.gz") which does not exist because the Windows files are zip files.

When looking at the actual files available from tensorflow it seems that the distinction is Windows / unix rather than msvc / gnu.

I have solved this locally by patching the extension lines as follows:

let ext = match os {
    "windows" => ".zip",
    _ => ".tar.gz",
};

and replacing all target_env = "msvc" checks with target_os = "windows" (the not conditions apply correctly to that new boolean).

Am I missing something that would make this break everything on other machines? Otherwise I believe that could be an improvement to the build script, as after this I can compile with both msvc and gnu, whereas before gnu will fail.

Added set and get to the Tensor, closes #222.

You should consider to use usize instead of u64 for the Tensor dims, so we don't need to convert u64 to usize.

Expose Output types for each Operation as mentioned in issue #358 and create an object we can add features to such as Input types for runtime (graph build time) access of Operation properties.

This is a naive port of the current test suite over to GitHub Actions (GH Actions), and completes in 7-10 minutes. It is just a boilerplate runner that then executes the test-all script. Note that this GH Action does not replicate documentation generation and push. If that's a desired feature, please let me know and I'll update the PR to include it.

Fixes #301.

I guess I left the work unfinished when I submitted RecordWriter a few years ago. I was hoping some upstream issues would be resolved (e.g., GATs for efficient iteration, and I was expecting the Read trait to change to deal with the uninitialized buffers problem).

I think this is the best that can be done for right now, and it's better to have something than nothing.

Several changes and fixes have been implemented in the previous months, can we get a new release please? Some projects don't allow for pulling from github and require a published crate.

Thanks in advance.

The examples, for instance, examples/regression.rs include code to load a graph from a .pb file

    let mut graph = Graph::new();
    let mut proto = Vec::new();
    File::open(filename)?.read_to_end(&mut proto)?;
    graph.import_graph_def(&proto, &ImportGraphDefOptions::new())?;

The code works with a .pb file generated with Tensorflow 1.x for Python.

import os
import tensorflow as tf

x = tf.placeholder(tf.float32, name='x')
y = tf.placeholder(tf.float32, name='y')

w = tf.Variable(tf.random_uniform([1], -1.0, 1.0), name='w')
b = tf.Variable(tf.zeros([1]), name='b')
y_hat = w * x + b

loss = tf.reduce_mean(tf.square(y_hat - y))
optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss, name='train')

init = tf.variables_initializer(tf.global_variables(), name='init')

definition = tf.Session().graph_def
directory = 'examples/regression'
tf.train.write_graph(definition, directory, 'model.pb', as_text=False)

But if we run this Python file under Tensorflow 2.x after making some small changes for compatibility

import os
import tensorflow as tf
import tensorflow.compat.v1 as tf1
tf1.disable_eager_execution()

x = tf1.placeholder(tf.float32, name='x')
y = tf1.placeholder(tf.float32, name='y')

w = tf.Variable(tf.random_uniform_initializer(minval=-1.0, maxval=1.0)(shape=[1]), name='w')
b = tf.Variable(tf.zeros([1]), name='b')
y_hat = w * x + b

loss = tf.reduce_mean(tf.square(y_hat - y))
optimizer = tf1.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss, name='train')

init = tf1.variables_initializer(tf1.global_variables(), name='init')

definition = tf1.Session().graph_def
directory = 'examples/regression'
tf.io.write_graph(definition, directory, 'model.pb', as_text=False)

But when trying to run the example, after initializing Tensorflow session, the following error appears Error: {inner:0x55665fcf34b0, InvalidArgument: Requested tensor type does not match actual tensor type: Resource vs Float}

This is support for pluggable device using experimental TensorFlow feature. Motivated by the desire to use the M1 Chip GPU cores. Thanks to gists of @lnshi, @wangjia184 and my own trial and error.

Suppose that one, for instance, restores a graph from a file. Then one needs to add some elements. But one needs a Scope as an argument for building operations.

Tensorflow-java allows one to build a Scope from a Graph: see the example in the documentation. Why not in tensorflow-rust?

Best regards.

Im compiling using 'cargo build' with '--target=aarch64-unknown-linux-gnu', and the compilation fails as it doesn't detect bazel. I have installed bazel and added it to path on windows. As the script ./configure seems to run from 'bash' i have installed bazel in the bash too, using './bazel-5.3.1-installer-linux-x86_64.sh'.

Also the ./configure file had to be changed using dos2unix, as it was run from bash.

Full log:

cargo build --target=aarch64-unknown-linux-gnu Compiling tensorflow-sys v0.21.0 error: failed to run custom build command for tensorflow-sys v0.21.0

Caused by: process didn't exit successfully: C:\code\rl\target\debug\build\tensorflow-sys-a87e5a4a076eaf92\build-script-build (exit code: 101) --- stdout cargo:rerun-if-env-changed=TENSORFLOW_NO_PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_aarch64-unknown-linux-gnu cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_aarch64_unknown_linux_gnu cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_ALLOW_CROSS cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS cargo:rerun-if-env-changed=PKG_CONFIG_aarch64-unknown-linux-gnu cargo:rerun-if-env-changed=PKG_CONFIG_aarch64_unknown_linux_gnu cargo:rerun-if-env-changed=TARGET_PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_aarch64-unknown-linux-gnu cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_aarch64_unknown_linux_gnu cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_SYSROOT_DIR cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR tensorflow-sys/build.rs:303: output = "C:\code\rl\target\aarch64-unknown-linux-gnu\debug\build\tensorflow-sys-08f60dab431ef5c2\out" tensorflow-sys/build.rs:305: source = "C:\Users\ugav\.cargo\registry\src\github.com-1ecc6299db9ec823\tensorflow-sys-0.21.0\target/source-v2.8.0" tensorflow-sys/build.rs:307: lib_dir = "C:\code\rl\target\aarch64-unknown-linux-gnu\debug\build\tensorflow-sys-08f60dab431ef5c2\out\lib-v2.8.0" tensorflow-sys/build.rs:309: Directory "C:\code\rl\target\aarch64-unknown-linux-gnu\debug\build\tensorflow-sys-08f60dab431ef5c2\out\lib-v2.8.0" already exists tensorflow-sys/build.rs:316: framework_library_path = "C:\code\rl\target\aarch64-unknown-linux-gnu\debug\build\tensorflow-sys-08f60dab431ef5c2\out\lib-v2.8.0\libtensorflow_framework.so.2" tensorflow-sys/build.rs:319: library_path = "C:\code\rl\target\aarch64-unknown-linux-gnu\debug\build\tensorflow-sys-08f60dab431ef5c2\out\lib-v2.8.0\libtensorflow.so.2" tensorflow-sys/build.rs:436: Executing "bazel" "version" tensorflow-sys/build.rs:438: Command "bazel" "version" finished successfully tensorflow-sys/build.rs:335: framework_target_path = "tensorflow/libtensorflow_framework.so.2" tensorflow-sys/build.rs:337: target_path = "tensorflow/libtensorflow.so" tensorflow-sys/build.rs:420: Executing "bash" "-c" "yes ''|./configure" Cannot find bazel. Please install bazel.

--- stderr thread 'main' panicked at 'failed to execute "bash" "-c" "yes ''|./configure"', C:\Users\ugav.cargo\registry\src\github.com-1ecc6299db9ec823\tensorflow-sys-0.21.0\build.rs:422:9 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

Although TF_LoadPluggableDeviceLibrary is an experimental API, it is necessary to use GPU on platform like Mac Arm.

I tried to include this API in code generating but it caused other errors. Now I always add it manually.

extern "C" {
    pub fn TF_LoadPluggableDeviceLibrary(
        library_filename: *const ::std::os::raw::c_char,
        status: *mut TF_Status,
    ) -> *mut TF_Library;
}

This API had better be included.