libhdfs binding and wrapper APIs for Rust

Related tags

Distributed systems hdfs-rs
Overview

hdfs-rs

libhdfs binding library and rust APIs which safely wraps libhdfs binding APIs

Current Status

  • Alpha Status (Rust wrapping APIs can be changed)
  • All libhdfs FFI APIs are ported.
  • Safe Rust wrapping APIs to cover some of the libhdfs APIs.

Documentation

Requirements

Usage

Add this to your Cargo.toml:

[dependencies]
hdfs = "0.0.4"

and this to your crate root:

extern crate hdfs;

hdfs-rs uses libhdfs, which is JNI native implementation. JNI native implementation requires the proper CLASSPATH. exec.sh included in the source code root plays a role to execute your program with the proper CLASSPATH. exec.sh requires HADOOP_HOME. So, you firstly set HADOOP_HOME shell environment variable as follows:

export HADOOP_HOME=<hadoop install dir>
```

Then, you can execute your program as follows:

```bash
./exec your_program arg1 arg2
```
You might also like...
Easy c̵̰͠r̵̛̠ö̴̪s̶̩̒s̵̭̀-t̶̲͝h̶̯̚r̵̺͐e̷̖̽ḁ̴̍d̶̖̔ ȓ̵͙ė̶͎ḟ̴͙e̸̖͛r̶̖͗ë̶̱́ṉ̵̒ĉ̷̥e̷͚̍ s̷̹͌h̷̲̉a̵̭͋r̷̫̊ḭ̵̊n̷̬͂g̵̦̃ f̶̻̊ơ̵̜ṟ̸̈́ R̵̞̋ù̵̺s̷̖̅ţ̸͗!̸̼͋

Rust S̵̓i̸̓n̵̉ I̴n̴f̶e̸r̵n̷a̴l mutability! Howdy, friendly Rust developer! Ever had a value get m̵̯̅ð̶͊v̴̮̾ê̴̼͘d away right under your nose just when

null 294 Dec 23, 2022
A safe and idiomatic wrapper over shared memory APIs in rust with proper cleanups.

shmem-bind A safe and idiomatic wrapper over shared memory APIs in rust with proper cleanups. Quick start: check the message-passing example for bette

ArshiA Akhavan 3 Apr 6, 2024
🎳 Rust binding and wrapper over NVIDIA PhysX 🦀
🎳 Rust binding and wrapper over NVIDIA PhysX 🦀

🎳 physx-rs Rust binding and wrapper over NVIDIA PhysX, a popular and mature physics engine particularly well-suited for games. Created and maintained

Embark 489 Dec 29, 2022
REC2 (Rusty External Command and Control) is client and server tool allowing auditor to execute command from VirusTotal and Mastodon APIs written in Rust. 🦀
REC2 (Rusty External Command and Control) is client and server tool allowing auditor to execute command from VirusTotal and Mastodon APIs written in Rust. 🦀

Information: REC2 is an old personal project (early 2023) that I didn't continue development on. It's part of a list of projects that helped me to lea

Quentin Texier (g0h4n) 104 Oct 7, 2023
Bolt is a desktop application that is designed to make the process of developing and testing APIs easier and more efficient.

Bolt ⚡ Bolt is a desktop application that is designed to make the process of developing and testing APIs easier and more efficient. Quick start 👩‍💻

0xHiro 6 Mar 26, 2023
A html document syntax and operation library written in Rust, use APIs similar to jQuery.

Visdom A server-side html document syntax and operation library written in Rust, it uses apis similar to jQuery, left off the parts thoes only worked

轩子 80 Dec 21, 2022
Rust client to Opensea's APIs and Ethereum smart contracts

opensea.rs Rust bindings & CLI to the Opensea API and Contracts CLI Usage Run cargo r -- --help to get the top level help menu: opensea-cli 0.1.0 Choo

Georgios Konstantopoulos 226 Dec 27, 2022
Safe, idiomatic bindings to cFE and OSAL APIs for Rust
Safe, idiomatic bindings to cFE and OSAL APIs for Rust

n2o4 The n2o4 crate provides safe, idiomatic Rust bindings to the APIs of cFE and OSAL, the libraries of the Core Flight System (cFS). IMPORTANT NOTE

null 3 Aug 29, 2022
Rust client to Seaport's APIs and Ethereum smart contracts(WIP)

Seaport-rs WIP - WORK IN PROGRESS. REFER TO https://github.com/Alcibiades-Capital/quay FOR PRODUCTION READY CODE I'm new to Rust and seek to be legend

Perelyn 21 Nov 29, 2022
A set of Rust libraries to interact with apple's private APIs and servers.

apple-private-apis A set of Rust libraries to interact with apple's private APIs and servers, made for use in SideInstaller. Library Description omnis

SideStore Team 4 Jan 25, 2023
A rust library for interacting with multiple Discord.com-compatible APIs and Gateways at the same time.
A rust library for interacting with multiple Discord.com-compatible APIs and Gateways at the same time.

Chorus A rust library for interacting with (multiple) Spacebar-compatible APIs and Gateways (at the same time). Explore the docs » Report Bug · Reques

polyphony 4 Apr 30, 2023
Fork of the Official Python3 API connector for Bybit's HTTP (bybit) and WebSockets APIs to rust

Fork of pybit python libary in Rust For the rust lovers and creators for bybit exchange version 5 Official Python3 API connector for Bybit's HTTP and

Omambia Dauglous 4 Aug 29, 2023
Rust library for integrating local LLMs (with llama.cpp) and external LLM APIs.

Table of Contents About The Project Getting Started Roadmap Contributing License Contact A rust interface for the OpenAI API and Llama.cpp ./server AP

Shelby Jenkins 4 Dec 18, 2023
Rust crate providing a variety of automotive related libraries, such as communicating with CAN interfaces and diagnostic APIs

The Automotive Crate Welcome to the automotive crate documentation. The purpose of this crate is to help you with all things automotive related. Most

I CAN Hack 29 Mar 11, 2024
Rust Attribute-Based Encryption library rabe's C FFI binding , support CP-ABE and KP-ABE encrypt and decrypt, submodule of Rabe.Core c# library.

Rabe-ffi Rust Attribute-Based Encryption library rabe's C FFI binding , support CP-ABE and KP-ABE encrypt and decrypt, submodule of Rabe.Core c# libra

Aya0wind 2 Oct 10, 2022
Utilities and tools based around Amazon S3 to provide convenience APIs in a CLI

s3-utils Utilities and tools based around Amazon S3 to provide convenience APIs in a CLI. This tool contains a small set of command line utilities for

Isaac Whitfield 47 Dec 15, 2022
Easy to use cryptographic framework for data protection: secure messaging with forward secrecy and secure data storage. Has unified APIs across 14 platforms.
Easy to use cryptographic framework for data protection: secure messaging with forward secrecy and secure data storage. Has unified APIs across 14 platforms.

Themis provides strong, usable cryptography for busy people General purpose cryptographic library for storage and messaging for iOS (Swift, Obj-C), An

Cossack Labs 1.6k Dec 30, 2022
hy-rs, pronounced high rise, provides a unified and portable to the hypervisor APIs provided by various platforms.

Introduction The hy-rs crate, pronounced as high rise, provides a unified and portable interface to the hypervisor APIs provided by various platforms.

S.J.R. van Schaik 12 Nov 1, 2022
PyQIR is a set of APIs for generating, parsing, and evaluating Quantum Intermediate Representation (QIR).

PyQIR PyQIR is a set of APIs for generating, parsing, and evaluating Quantum Intermediate Representation (QIR). It consists of the following component

QIR Alliance 37 Dec 31, 2022
Comments
  • Rust wrapping for hdfs::binding::hdfsGetPathInfo()

    Rust wrapping for hdfs::binding::hdfsGetPathInfo()

    Hi,

    could you please add Rust wrapping for hdfs::binding::hdfsGetPathInfo()? If i understand it correctly, it's the only way to get actual block size for specific file. Other functions are returning only defualt block size (global or for some path). Thanks.

    enhancement 
    opened by dorny 5
  • Enable platform-dependent include path.

    Enable platform-dependent include path.

    Hi, I added some codes to enable platform-dependent include path. I'm a newbie of Rust, so my patch may not follow the conventional Rust code style. Welcome any comments.

    Review on Reviewable

    opened by jihoonson 2
  • The function `str_to_chars` in utils.rs returns the invalid address

    The function `str_to_chars` in utils.rs returns the invalid address

    util.rs

    pub fn str_to_chars(s: &str) -> *const c_char {
  CString::new(s.as_bytes()).unwrap().as_ptr()
}

    I think s doesn't live long enough. When this function returns, s has been dropped, so the address it returned leads an expected value.

    For example, I have to do this:

    original

      pub fn exist(&self, path: &str) -> bool {
    if unsafe {hdfsExists(self.raw, str_to_chars(path))} == 0 {
      true
    } else {
      false
    }
}

    new

      pub fn exist(&self, path: &str) -> bool {
    let path = CString::new(path.as_bytes()).unwrap();
    if unsafe {hdfsExists(self.raw, path.as_ptr())} == 0 {
      true
    } else {
      false
    }
  }
    opened by starccy 0
  • `exec.sh cargo test` failed

    `exec.sh cargo test` failed

    I'm trying to use this Hadoop 2.7.2. Compiled the native lib on Mac. cargo build works fine, but when exec.sh cargo test I got this:

    ➜  hdfs-rs (master) ✗ ./exec.sh cargo test
   Compiling hdfs v0.0.4 (file:///Users/chao/git/hdfs-rs)
warning: lint raw_pointer_derive has been removed: using derive with raw pointers is ok
   --> src/dfs.rs:273:9
    |
273 | #[allow(raw_pointer_derive)]
    |         ^^^^^^^^^^^^^^^^^^
    |
    = note: #[warn(renamed_and_removed_lints)] on by default

warning: lint raw_pointer_derive has been removed: using derive with raw pointers is ok
   --> src/dfs.rs:273:9
    |
273 | #[allow(raw_pointer_derive)]
    |         ^^^^^^^^^^^^^^^^^^

warning: unused import: `itertools::Itertools`
   --> src/dfs.rs:787:7
    |
787 |   use itertools::Itertools;
    |       ^^^^^^^^^^^^^^^^^^^^
    |
    = note: #[warn(unused_imports)] on by default

error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-m64" "-L" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.158wz1xuz2s5vbhj.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.15ocno9e0gz60klj.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.16u6js6g0l3k1ic6.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.17erb31xn70viq4b.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.181cuta0v63atwcm.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.1e698527iprj2v8f.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.1ha8q1iwal888w5t.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.1im38lueib99jsk0.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.1jnlhyxj59jycbjv.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.1qko0s7o0yku8wi7.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.1y16o1qfye96o7m0.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.1zwd8n7bcl3vhvvh.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.22hy4jf5cexj8ofe.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.23ce453q2vda0cq6.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.23tqyymcb18u96mb.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.2bmdyctmqwyrczo2.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.2cf3hj3i603y6nxm.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.2jqywn86b2gsqohu.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.2lyh15q6cjwzy18c.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.2oiwwqohid6mabhc.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.2w6xsa2hbi53xhak.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.2xnvmuhjbhd7vxcm.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.2xs38k3nu4oe9uo6.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.2xwdgiv55wx7wpkg.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3ayaeypdcro9d6yk.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3cutt3lg8vmui89m.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3g94tobrpgum5dk6.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3ik0x0hz6l66cx38.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3nb415nst569tn21.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3oqznv5oym7l3m4e.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3rngp6bm2u2q5z0y.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3thsdxsfmyg8gwej.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3vck0z67htzlxjio.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.3wta9ctgdrpkmlpr.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.41fhlsd7519xuco5.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.436dotimmrgzkwfa.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.43v6g0y2xsxoggnt.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.44bsbddupzfao2om.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.45pc7c65foh9i35f.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.48721dc4k5qxei0u.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.49a7n47po4ttqjl7.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.49lx1q7cxvpykyv0.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.49qmpuk8w1opmi5q.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4brwxiorhmgsb6oc.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4ezmh1vbs95c5ack.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4g47d2sx6m33abop.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4oc10dk278mpk1vy.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4rzh6npf1tuwcxx9.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4sw9hz7i7fiueihz.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4v3lwapkm7lhovox.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4xq48u46a1pwiqn7.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4xuhwz5yy6q4gvgg.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4yh8x2b62dcih00t.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4ypvbwho0bu5tnww.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.4z8gk4dt64uq1bs.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.51s1w397y42gpez1.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.56dly8q07ws8ucdq.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.572bgozz9kcwe4zq.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.57k06xfugllsc526.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.5beqqbexwb143v6a.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.5by9xggibmun5kn9.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.5gf6du7k58s78kob.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.63mmpc50mjcb0lb.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.8xzrsc1ux72v29j.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.98g0d9x8aw3akpe.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.9elsx31vb4it187.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.c6lbtaiefvx3wya.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.d6p2df93qm1h3gm.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.e19kx9t7xhzajfl.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.mb501chi8fnqgmm.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.oa3rad818d8sgn4.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.y08g5q2x813c4wx.rcgu.o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.z9ox7biyn1otfln.rcgu.o" "-o" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55" "/Users/chao/git/hdfs-rs/target/debug/deps/hdfs-2b714980d01b0f55.crate.allocator.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/Users/chao/git/hdfs-rs/target/debug/deps" "-L" "/opt/hadoop/lib/native" "-L" "/Users/chao/git/hdfs-rs/target/debug/build/hdfs-6b27ce521f6b3b8b/out" "-L" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "-l" "hdfs" "-l" "hdfs" "-Wl,-force_load,/Users/chao/git/hdfs-rs/target/debug/build/hdfs-6b27ce521f6b3b8b/out/libminidfs.a" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libtest-0d97595764cf9b64.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libterm-0f60ec9e394e0248.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libgetopts-6f2ac69fc328f830.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/liburl-1012528998a3d996.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/libmatches-d634ed9077d128bf.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/libuuid-2cdedf5331d88c7d.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/librand-67af2eb6503806d0.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/librand-c95782c5d16a1bc5.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/liblibc-0819fa82d85a83bd.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/librustc_serialize-a4d600dec7edbf71.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/liblog-5e9334cd63f74e97.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/liblog-41e198eccb985521.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/libcfg_if-2bd44b22855dd866.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/liblibc-017538ef75185d0f.rlib" "/Users/chao/git/hdfs-rs/target/debug/deps/libitertools-a36391e000121329.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-81ad5fb7ec87f0fb.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-2e56e130d8e4c4c5.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-65f3bd01304e8cb7.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libunwind-63ece1d61d0caa09.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_system-1f539c0f734c73b4.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liblibc-72ea87627c504b3f.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc-f664b1dcf152920f.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcore-39fcd66b77381104.rlib" "/Users/chao/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-c6bbc2262c448bc9.rlib" "-l" "c" "-l" "m" "-l" "System" "-l" "resolv" "-l" "pthread" "-l" "c" "-l" "m"
  = note: Undefined symbols for architecture x86_64:
            "_hadoopConfSetStr", referenced from:
                _nmdConfigureShortCircuit in libminidfs.a(native_mini_dfs.o)
            "_hdfsDisableDomainSocketSecurity", referenced from:
                _nmdConfigureShortCircuit in libminidfs.a(native_mini_dfs.o)
            "_newRuntimeError", referenced from:
                _nmdConfigureShortCircuit in libminidfs.a(native_mini_dfs.o)
            "_getJNIEnv", referenced from:
                _nmdCreate in libminidfs.a(native_mini_dfs.o)
                _nmdFree in libminidfs.a(native_mini_dfs.o)
                _nmdShutdown in libminidfs.a(native_mini_dfs.o)
                _nmdGetNameNodePort in libminidfs.a(native_mini_dfs.o)
            "_newJavaStr", referenced from:
                _nmdCreate in libminidfs.a(native_mini_dfs.o)
            "_constructNewObjectOfClass", referenced from:
                _nmdCreate in libminidfs.a(native_mini_dfs.o)
            "_invokeMethod", referenced from:
                _nmdCreate in libminidfs.a(native_mini_dfs.o)
                _nmdShutdown in libminidfs.a(native_mini_dfs.o)
                _nmdGetNameNodePort in libminidfs.a(native_mini_dfs.o)
            "_printExceptionAndFree", referenced from:
                _nmdCreate in libminidfs.a(native_mini_dfs.o)
                _nmdShutdown in libminidfs.a(native_mini_dfs.o)
                _nmdGetNameNodePort in libminidfs.a(native_mini_dfs.o)
            "_printPendingExceptionAndFree", referenced from:
                _nmdCreate in libminidfs.a(native_mini_dfs.o)
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)


error: aborting due to previous error

error: Could not compile `hdfs`.

To learn more, run the command again with --verbose.

    is version 2.7.2 supported by this library? do I need to have a running Hadoop environment on my Mac in order to run the tests? Thanks.

    opened by sunchao 3
Releases(release-0.0.3)

  • release-0.0.3(Oct 7, 2015)

    This release is considered alpha quality. We are still adding more wrapping APIs in order to fully cover the original native libhdfs APIs. Also, some wrapping APIs are still unstable, and their signature can be changed later.

    Major changes

    • All libhdfs APIs are ported.
    • Implemented safe and more idomatic rust APIs: HdfsFsCache, HdfsFile, HdfsUtil
    • HdfsFsCache, HdfsFile, HdfsUtil were reexported into the root module.
    • Fixed a bug caused by missed field in hdfsFileInfo
    Source code(tar.gz)
    Source code(zip)
Owner
Hyunsik Choi
Hyunsik Choi
GitHub
Easy Hadoop Streaming and MapReduce interfaces in Rust

Efflux Efflux is a set of Rust interfaces for MapReduce and Hadoop Streaming. It enables Rust developers to run batch jobs on Hadoop infrastructure wh

Isaac Whitfield 31 Nov 22, 2022
Twitch data consumer and broadcaster

NeoTwitch Arch Network broadcaster Chat (message buffer) If the message buffer is full then shut down Channel point events If the message buffer is fu

Togglebit 3 Dec 3, 2021
A fully asynchronous, futures-based Kafka client library for Rust based on librdkafka

rust-rdkafka A fully asynchronous, futures-enabled Apache Kafka client library for Rust based on librdkafka. The library rust-rdkafka provides a safe

Federico Giraud 1.1k Jan 8, 2023
Rust client for Apache Kafka

Kafka Rust Client Project Status This project is starting to be maintained by John Ward, the current status is that I am bringing the project up to da

Yousuf Fauzan 902 Jan 2, 2023
Easy-to-use beanstalkd client for Rust (IronMQ compatible)

rust-beanstalkd Easy-to-use beanstalkd client for Rust (IronMQ compatible) Install Add this dependency to your Cargo.toml beanstalkd = "*" Documentati

Johannes Schickling 44 Oct 4, 2022
Fluvio is a high-performance distributed streaming platform that's written in Rust

Fluvio is a high-performance distributed streaming platform that's written in Rust, built to make it easy to develop real-time applications.

InfinyOn 1.6k Dec 30, 2022
Magical Automatic Deterministic Simulator for distributed systems in Rust.

MadSim Magical Automatic Deterministic Simulator for distributed systems. Deterministic simulation MadSim is a Rust async runtime similar to tokio, bu

MadSys Research Group 249 Dec 28, 2022
The Raft algorithm implement by Rust.

Raft The Raft algorithm implement by Rust. This project refers to Eli Bendersky's website, the link as follows: https://eli.thegreenplace.net/2020/imp

Qiang Zhao 1 Oct 23, 2021
Raft distributed consensus for WebAssembly in Rust

WRaft: Raft in WebAssembly What is this? A toy implementation of the Raft Consensus Algorithm for WebAssembly, written in Rust. Basically, it synchron

Emanuel Evans 60 Oct 22, 2022
Paxakos is a pure Rust implementation of a distributed consensus algorithm

Paxakos is a pure Rust implementation of a distributed consensus algorithm based on Leslie Lamport's Paxos. It enables distributed systems to consistently modify shared state across their network, even in the presence of failures.

Pavan Ananth Sharma 2 Jul 5, 2022