Wrapper around Microsoft CNTK library

Vlado Boza

Last update: Nov 30, 2021

Related tags

Text processing cntk-rs

Overview

Bindings for CNTK library

Simple low level bindings for CNTK library from Microsoft.

API Documentation

Status

Currently exploring ways how to interact with C++ API nicely from Rust. Expect a lot of breaking changes.

Build scripts are not ready yet, might not work outside of 64bit linux.

Goals

Ability to train all reasonable architectures
No memory leaks
Reasonable ease of use

Nongoals

Idiomatic Rust everywhere
Super good ergonomics
Avoiding all unnecessary copies

Building and installing

You need to have CNTK-2.3 installed and paths to includes and library files in relevant enviroment variables (cntk activate scripts does this well). You also need g++-4.8 installed (because CNTK uses it to compile things).

Example usage

See examples folder.

Other limitations

Only works with single precision (f32 in Rust, float in C++) types. Only works with dense representations of vectors/matrices/tensors. Only works with ASCII strings for variable names and filenames.

What works

Passing data in and out of computation.
Backpropagation.
Training fully connected feedforward, convolutional and recurrent network.
Saving and loading the model.
Code for most operations. - Almost all, except couple of helpers.
Demo of seq2seq model training.
Turning c++ exception into rust panics (most of the time).
Interop with NDArray library.

Planned in future

GPU.
Better docs.
Finish all operations.
Better build scripts.
Figure out whether we want NDArrayView or go directly from Rust data to Value and back.
Builder pattern where appropriate (Variable?).

A morphological analysis library.

Lindera A Japanese morphological analysis library in Rust. This project fork from fulmicoton's kuromoji-rs. Lindera aims to build a library which is e

240 Dec 27, 2022

A fast, low-resource Natural Language Processing and Text Correction library written in Rust.

nlprule A fast, low-resource Natural Language Processing and Error Correction library written in Rust. nlprule implements a rule- and lookup-based app

496 Jan 8, 2023

A Rust library containing an offline version of webster's dictionary.

webster-rs A Rust library containing an offline version of webster's dictionary. Add to Cargo.toml webster = 0.3.0 Simple example: fn main() { le

12 Sep 27, 2022

Rust-nlp is a library to use Natural Language Processing algorithm with RUST

nlp Rust-nlp Implemented algorithm Distance Levenshtein (Explanation) Jaro / Jaro-Winkler (Explanation) Phonetics Soundex (Explanation) Metaphone (Exp

34 Dec 20, 2022

A lightweight library with vehicle tuning utilities.

A lightweight library with vehicle tuning utilities. This includes utilities for communicating with OBD-II services, firmware downloading/flashing, and table modifications.

6 Oct 3, 2022

lingua-rs Python binding. An accurate natural language detection library, suitable for long and short text alike.

lingua-py lingua-rs Python binding. An accurate natural language detection library, suitable for long and short text alike. Installation pip install l

7 Dec 30, 2022

A small random number generator hacked on top of Rust's standard library. An exercise in pointlessness.

attorand from 'atto', meaning smaller than small, and 'rand', short for random. A small random number generator hacked on top of Rust's standard libra

1 Nov 24, 2021

Library to calculate TF-IDF (Term Frequency - Inverse Document Frequency) for generic documents.

Library to calculate TF-IDF (Term Frequency - Inverse Document Frequency) for generic documents. The library provides strategies to act on objects that implement certain document traits (NaiveDocument, ProcessedDocument, ExpandableDocument).

13 Oct 31, 2022

A simple and fast linear algebra library for games and graphics

glam A simple and fast 3D math library for games and graphics. Development status glam is in beta stage. Base functionality has been implemented and t

953 Jan 3, 2023

Comments

error when building on ubuntu machines

Getting the error when building on an ubuntu machine

cargo:warning=/usr/include/c++/5/bits/cxxabi_forced.h:36:37: error: expected declaration before end of line
exit code: 1

opened by infinite-Joy9l 1

not able to build on ubuntu

I have been trying to build this library on ubuntu machines without success. First discovered that the files such as these needs to be copied manually to the system. Currently this is the error that is showing up.

   Compiling cntk v0.2.1 (/home/saionee/opensource/programming-languages/rust-lang/cntk-rs)
error: recursion limit reached while expanding the macro `__cpp_internal`
   --> src/ops.rs:134:23
    |
134 |           let payload = cpp!([xpayload as "Variable", adata_ptr as "Axis*", len as "size_t", bdata_ptr as "int*", edata_ptr as "int*", mut error_p as "char*"] -> FunctionInner as "FunctionPtr" {
    |  _______________________^
135 | |             try {
136 | |                 return Slice(xpayload,
137 | |                              vector<Axis>(adata_ptr, adata_ptr + len),
...   |
145 | |             }
146 | |         });
    | |__________^
    |
    = help: consider adding a `#![recursion_limit="128"]` attribute to your crate
    = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: aborting due to previous error

Are there better build instruction as I am not able to figure this thing out.

opened by infinite-Joy9l 1

Owner

Vlado Boza

GitHub

Wrapper around Microsoft CNTK library

Related tags

Overview

Bindings for CNTK library

Status

Goals

Nongoals

Building and installing

Example usage

Other limitations

What works

Planned in future

You might also like...

A morphological analysis library.

A fast, low-resource Natural Language Processing and Text Correction library written in Rust.

A Rust library containing an offline version of webster's dictionary.

Rust-nlp is a library to use Natural Language Processing algorithm with RUST

A lightweight library with vehicle tuning utilities.

lingua-rs Python binding. An accurate natural language detection library, suitable for long and short text alike.

A small random number generator hacked on top of Rust's standard library. An exercise in pointlessness.

Library to calculate TF-IDF (Term Frequency - Inverse Document Frequency) for generic documents.

A simple and fast linear algebra library for games and graphics

Comments

error when building on ubuntu machines

not able to build on ubuntu

Owner

Vlado Boza

nombytes is a library that provides a wrapper for the bytes::Bytes byte container for use with nom.

A Rust wrapper for the Text synthesization service TextSynth API

🛥 Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto.

Viterbi-based accelerated tokenizer (Python wrapper)

A command-line tool and library for generating regular expressions from user-provided test cases

An efficient and powerful Rust library for word wrapping text.

⏮ ⏯ ⏭ A Rust library to easily read forwards, backwards or randomly through the lines of huge files.

Natural language detection library for Rust. Try demo online: https://www.greyblake.com/whatlang/

A Rust library for generically joining iterables with a separator

👄 The most accurate natural language detection library in the Rust ecosystem, suitable for long and short text alike