Easy to use rust driver for arangoDB

fMeow

Last update: Jan 1, 2023

Related tags

Database rust nosql rust-library arangodb arangodb-client

Overview

arangors

arangors is an intuitive rust client for ArangoDB, inspired by pyArango.

arangors enables you to connect with ArangoDB server, access to database, execute AQL query, manage ArangoDB in an easy and intuitive way, both async and plain synchronous code with any HTTP ecosystem you love.

Philosophy of arangors

arangors is targeted at ergonomic, intuitive and OOP-like API for ArangoDB, both top level and low level API for users' choice.

Overall architecture of ArangoDB:

databases -> collections -> documents/edges

In fact, the design of arangors just mimic this architecture, with a slight difference that in the top level, there is a connection object on top of databases, containing a HTTP client with authentication information in HTTP headers.

Hierarchy of arangors:

connection -> databases -> collections -> documents/edges

Features

By now, the available features of arangors are:

make connection to ArangoDB
get list of databases and collections
fetch database and collection info
create and delete database or collections
full featured AQL query
support both async and sync

TODO

(Done) Milestone 0.1.x

Synchronous connection based on reqwest and full featured AQL query.
(X) Milestone 0.2.x

Fill the unimplemented API in Connection, Database, Collection and Document.

~~In this stage, all operations available for database, collection and document should be implemented.~~

Well, I am too lazy to fill all API, as the AQL syntax suffices in most cases. Maybe fulfill this goal in 0.4.x .
(Done) Milestone 0.3.x

Implement both sync and async client. Also, offers a way to use custom HTTP client ecosystem.
(WIP) Milestone 1.0.x

Provides the API related to:
- (X) Graph Management
- (X) Index Management
- ( ) User Management
In this stage, all operations available for database, collection and document should be implemented.

Glance

Use Different HTTP Ecosystem, Regardless of Async or Sync

You can switch to different HTTP ecosystem with a feature gate, or implement the Client yourself (see examples).

Currently out-of-box supported ecosystem are:

reqwest_async
reqwest_blocking
surf_async

By default, arangors use reqwest_async as underling HTTP Client to connect with ArangoDB. You can switch other ecosystem in feature gate:

[dependencies]
arangors = { version = "0.4", features = ["surf_async"], default-features = false }

Or if you want to stick with other ecosystem that are not listed in the feature gate, you can get vanilla arangors without any HTTP client dependency:

[dependencies]
## This one is async
arangors = { version = "0.4", default-features = false }
## This one is synchronous
arangors = { version = "0.4", features = ["blocking"], default-features = false }

Thanks to maybe_async, arangors can unify sync and async API and toggle with a feature gate. Arangors adopts async first policy.

Connection

There is three way to establish connections:

jwt
basic auth
no authentication

So are the arangors API.

Example:

With authentication

use arangors::Connection;

// (Recommended) Handy functions
let conn = Connection::establish_jwt("http://localhost:8529", "username", "password")
    .await
    .unwrap();
let conn = Connection::establish_basic_auth("http://localhost:8529", "username", "password")
    .await
    .unwrap();

Without authentication, only use in evaluation setting

let conn = Connection::establish_without_auth("http://localhost:8529").await.unwrap();

Database && Collection

use arangors::Connection;

let db = conn.db("test_db").await.unwrap();
let collection = db.collection("test_collection").await.unwrap();

AQL Query

All AQL query related functions are associated with database, as AQL query is performed at database level.

There are several way to execute AQL query, and can be categorized into two classes:

batch query with cursor
- aql_query_batch
- aql_next_batch
query to fetch all results
- aql_str
- aql_bind_vars
- aql_query

This later ones provide a convenient high level API, whereas batch queries offer more control.

Typed or Not Typed

Note that results from ArangoDB server, e.x. fetched documents, can be strong typed given deserializable struct, or arbitrary JSON object with serde::Value.

#[derive(Deserialize, Debug)]
struct User {
    pub username: String,
    pub password: String,
}

// Typed
let resp: Vec<User> = db
    .aql_str("FOR u IN test_collection RETURN u")
    .await
    .unwrap();
// Not typed: Arbitrary JSON objects
let resp: Vec<serde_json::Value> = db
    .aql_str("FOR u IN test_collection RETURN u")
    .await
    .unwrap();

Batch query

arangors offers a way to manually handle batch query.

Use aql_query_batch to get a cursor, and use aql_next_batch to fetch next batch and update cursor with the cursor.

let aql = AqlQuery::builder()
    .query("FOR u IN @@collection LIMIT 3 RETURN u")
    .bind_var("@collection", "test_collection")
    .batch_size(1)
    .count(true)
    .build();

// fetch the first cursor
let mut cursor = db.aql_query_batch(aql).await.unwrap();
// see metadata in cursor
println!("count: {:?}", cursor.count);
println!("cached: {}", cursor.cached);
let mut results: Vec<serde_json::Value> = Vec::new();
loop {
    if cursor.more {
        let id = cursor.id.unwrap().clone();
        // save data
        results.extend(cursor.result.into_iter());
        // update cursor
        cursor = db.aql_next_batch(id.as_str()).await.unwrap();
    } else {
        break;
    }
}
println!("{:?}", results);

Fetch All Results

There are three functions for AQL query that fetch all results from ArangoDB. These functions internally fetch batch results one after another to get all results.

The functions for fetching all results are listed as bellow:

`aql_str`

This function only accept a AQL query string.

Here is an example of strong typed query result with aql_str:

#[derive(Deserialize, Debug)]
struct User {
    pub username: String,
    pub password: String,
}

let result: Vec<User> = db
    .aql_str(r#"FOR i in test_collection FILTER i.username=="test2" return i"#)
    .await
    .unwrap();

`aql_bind_vars`

This function can be used to start a AQL query with bind variables.

use arangors::{Connection, Document};

#[derive(Serialize, Deserialize, Debug)]
struct User {
    pub username: String,
    pub password: String,
}


let mut vars = HashMap::new();
let user = User {
    username: "test".to_string(),
    password: "test_pwd".to_string(),
};
vars.insert("user", serde_json::value::to_value(&user).unwrap());
let result: Vec<Document<User>> = db
    .aql_bind_vars(r#"FOR i in test_collection FILTER i==@user return i"#, vars)
    .await
    .unwrap();

`aql_query`

This function offers all the options available to tweak a AQL query. Users have to construct a AqlQuery object first. And AqlQuery offer all the options needed to tweak AQL query. You can set batch size, add bind vars, limit memory, and all others options available.

use arangors::{AqlQuery, Connection, Cursor, Database};
use serde_json::value::Value;


let aql = AqlQuery::builder()
    .query("FOR u IN @@collection LIMIT 3 RETURN u")
    .bind_var("@collection", "test_collection")
    .batch_size(1)
    .count(true)
    .build();

let resp: Vec<Value> = db.aql_query(aql).await.unwrap();
println!("{:?}", resp);

Contributing

Contributions and feed back are welcome following Github workflow.

License

arangors is provided under the MIT license. See LICENSE. An ergonomic ArangoDB client for rust.

Comments

Add Working with a document
Hey ! Let's implement from the Arangodb documentation -> Working with a Document

[x] read a single document

[x] read document header

[x] create document

[x] replace document

[x] update document

[x] removes a document

From our work on documents we saw the need for updating some of the existing code regarding request and response. Therefore the todolist for this PR gets also updated

[x] update how we do request to the server to have more flexibility for adding headers and other options on a request

[x] update how we handle response from the server

Same as previously, each item is implemented with code and unit test :)

I see that I got some previous commits. No worries. I ll hard push on my branch later when stuff will be ready.

Let's go !
opened by arn-the-long-beard 79
Add collection modifying
Hello ! Let's implement from theArangodb documentation -> Modifying a Collection

NB : In this todo list, the unit test goes together with the implementation. So an implementation is considered done only if the unit test is also done :).

[x] load collection

[x] unload collection

[x] load indexes into memory

[x] change properties of a collection

[x] rename collection

[x] rotate journal

[x] recalculate count of a collection

Let's go !
opened by arn-the-long-beard 21
Feature/collection-properties
Hey, First pull request ever on Github. Every feedback are welcomed !!! Here is what I going to do :

[x] update base_url for collection and add doc to it

[x] implement collection::properties

[x] add unit test for properties

Is feature/properties good or feature/collection-properties better maybe for my local branch name?

Also I have some commit on readme, I guess it is because I had derived my branch from master in the first place and dev does not have the commit. Is it dirty to have it there ?

I tested the code and it works good. I will continue my work on the collection.rs. Maybe I should improve the unit test also and check for the properties instead of just no error.
enhancement
opened by arn-the-long-beard 18
How to use arangors with a web server
Hello ! First, I am huge fan of ArangoDB, and I am starting rust serisouly only now. So big thank you for having made this package ! I am very fresh on rust, like a newbie, nice contrast from my js/ts world lol

Your package is the most advanced regarding Arango driver for Rust. But it seems you might need extra help maybe.

But first, I need some help. I am trying to use Arangodb on a actix-web server that I am making. But I am getting this error message

Here is the message I am having

error[E0277]: the trait bound arangors::database::Database<'_>: std::clone::Clone is not satisfied in [closure@src/main.rs:130:38: 161:6 db:arangors::database::Database<'_>]

And here is the code that make it happen

let conn = Connection::establish_jwt("http://localhost:8529", "USER", "BEST PASSWORDEVER").unwrap(); let db= conn.db("test_db").unwrap() ; let mut server = HttpServer::new(|| { App::new() .data(db) .wrap(Logger::default()) .service( web::scope("/api") .service( web::resource("/auth") .route(web::post().to(login)) ), ) });

I tried to implement it with :

impl Clone for Database { fn clone(&self) -> Self { unimplemented!() } }

But then I got

| 104 | impl Clone for Database { | ^^^^^^^^^^^^^^^-------- | | | | | arangors::database::Database is not defined in the current crate | impl doesn't use only types from inside the current crate | = note: define and implement a trait or new type instead

Do you know how could I handle arangors inside my web server ? I do not want to write backend in Foxx Microservice, I really need to write rust code for my stuff. Also, if you need contributors, maybe I could be one if you want. But never done it before. I just have much time available right now.

Best regards, a man with a very long beard because the barbershop is closed due to you know lol
opened by arn-the-long-beard 14

arangors 0.5.0 phantom compile errors

I'm using rustc 1.57.0-nightly (f03eb6bef 2021-10-02). Getting compile errors from 0.5.0 crate:

    Checking arangors v0.5.0
error[E0560]: struct `AqlQueryBuilder<'_, (__query, (HashMap<&str, Value>,), __count, __batch_size, __cache, __memory_limit, __ttl, __options)>` has no field named `_phantom`
   --> /Users/evan/.cargo/registry/src/github.com-1ecc6299db9ec823/arangors-0.5.0/src/aql.rs:153:13
    |
153 |             _phantom: self._phantom,
    |             ^^^^^^^^ help: a field with a similar name exists: `phantom`

error[E0609]: no field `_phantom` on type `AqlQueryBuilder<'a, (__query, (), __count, __batch_size, __cache, __memory_limit, __ttl, __options)>`
   --> /Users/evan/.cargo/registry/src/github.com-1ecc6299db9ec823/arangors-0.5.0/src/aql.rs:153:28
    |
153 |             _phantom: self._phantom,
    |                            ^^^^^^^^ help: a field with a similar name exists: `phantom`

Some errors have detailed explanations: E0560, E0609.
For more information about an error, try `rustc --explain E0560`.

opened by velvia 11

reqwest + rustls

reqwest has the rustls-tls feature that will use rustls instead of the native system openssl library.

Would be great to have an extra feature to toggle this.

opened by theduke 6

AqlQuery with bind_vars

How is one supposed to be using bind_vars? Using the web-query-builder from ArangoDB, I'm used to being able to create queries as follows: INSERT @user INTO @@collection with user: { "name": "Test" } and @collection: "users".

Assuming I tried the same with the query builder, I tried:

const COLLECTION: &str = "users";

#[derive(Serialize)]
pub struct User {
    name: String,
}

fn main() {
    let user = User { name: "Tester".to_string() };
    let conn = Connection::establish_jwt("http://localhost:8529", "root", "Password")
        .unwrap();

    let db = conn.db("Test").unwrap();

    let json = serde_json::to_string(&user).unwrap();
    let aql = AqlQuery::builder()
        .query("INSERT @user INTO @@collection LET result = NEW RETURN result")
        .bind_var("@collection", COLLECTION)
        .bind_var("user", json)
        .build();

    let result: Vec<Value> = db.aql_query(aql).expect("Query failed");
}

However, that fails, as the query generated escapes the double quotes in the JSON that I generated. Resulting in this: {"query":"INSERT @client INTO @@collection LET result = NEW RETURN result","bindVars":{"client":"{\"name\":\"Tester\"}","@collection":"fs_clients" }

instead of this {"query":"INSERT @client INTO @@collection LET result = NEW RETURN result","bindVars":{"client":"{"name":"Tester"}","@collection":"fs_clients" }

If I tried assigning the user struct directly however:

const COLLECTION: &str = "users";

#[derive(Serialize)]
pub struct User {
    name: String,
}

fn main() {
    let user = User { name: "Tester".to_string() };
    let conn = Connection::establish_jwt("http://localhost:8529", "root", "Password")
        .unwrap();

    let db = conn.db("Test").unwrap();

    let aql = AqlQuery::builder()
        .query("INSERT @user INTO @@collection LET result = NEW RETURN result")
        .bind_var("@collection", COLLECTION)
        .bind_var("user", user)
        .build();

    let result: Vec<Value> = db.aql_query(aql).expect("Query failed");
}

I cannot compile, as: ^^^^^^^^ the trait 'std::convert::From<User>' is not implemented for 'serde_json::value::Value'

opened by inzanez 6

Add collection information
Well, I guess that we need to implement all the methods we find on Arangodb documentation for collection or maybe not.

Here is the todo list

[x] implement get_collection -> done previously in database::collection

[x] update base_url for collection and add doc to it -> done in previous PR

[x] implement collection::properties -> done in previous PR

[x] add unit test for properties -> done in previous PR

[x] add or update structures that match the information from the doc Should we add Option<T> on CollectionDetails or having new structures?

[x] implement document count

[x] add unit test for get count

[x] implement statistics

[x] add unit test for statistics

Should we do something about the shard stuff there and there ?

[x] implement revision id

Question : the response in Arangodb 3.7 and 3.6 are different for revision. What is the best way to handle that ?

[x] add unit test for revision id

[x] implement checksum for collection

[x] add unit test for checksum for collection

Should we add read all collections ?
opened by arn-the-long-beard 6

Improve error message when credentials for connection are wrong

Hello guys ! It is a long time :smile:

I am still continuing my journey on Rust :stuck_out_tongue:

Well I saw today that if the credentials for connection are wrong we get this :

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("missing field `jwt`"

The error comes from there :


    #[maybe_async]
    async fn jwt_login<T: Into<String>>(
        arango_url: &Url,
        username: T,
        password: T,
    ) -> Result<String, ClientError> {
        #[derive(Deserialize)]
        struct JWT {
            pub jwt: String,
        }
        let url = arango_url.join("/_open/auth").unwrap();

        let mut map = HashMap::new();
        map.insert("username", username.into());
        map.insert("password", password.into());

        let jwt: JWT = serde_json::from_str(
            C::new(None)?
                .post(url, &serde_json::to_string(&map)?)
                .await?
                .body(),
        )
        .unwrap();
        Ok(jwt.jwt)
    }

Maybe we could improve the error message maybe and give an error like " Unauthorized" or something .

What do you think ?

I can make it if you want :wink:

opened by arn-the-long-beard 5

Streaming Transactions API Support?

Any plans on adding support for this: https://www.arangodb.com/2019/08/rc7-of-arangodb-3-5-streaming-transactions-api/

Also, definitely keep up the good work here! I am planning on cutting a system over to ArangoDB which has previously been based on MongoDB. It would be excellent to see support for this stuff as well as async support based on the new futures & async/await features landing in Rust.

Also, I built the Wither ODM a while back, which is great, but the MongoDB team has left the underlying driver in a pretty stagnant state. Do you think there is value in building a data modelling ODM-like system for Arango as well?

opened by thedodd 5
Reconsider futures-based interface

I really need to have a asynchronous interface to Arango.

reqwest already provides the reqwest::r#async::Client async client that can be used easily. It would need an additional refactor and major version bump when async/await hits stable, but that will only happen in late September.

Let me know if you'll accept a PR, otherwise I'll just roll my own.

opened by theduke 5
build(deps): bump uclient from 0.1.3 to 0.2.3
Bumps uclient from 0.1.3 to 0.2.3.

Changelog

Sourced from uclient's changelog.

0.2.3 (2021-05-30)

Bug Fixes

add multipart header after content disposition (54e263c)

0.2.2 (2021-05-29)

Features

add FormStream (cc5135d)

use Form Stream (78a2765)

0.2.1 (2021-05-29)

Features

add method to turn into header (04ba0bc)

count bytes count of multipart Read object (787a5f9)

guess mime for file (1f4583e)

use crate::Error for multipart_to_read() (e695847)

Bug Fixes

test (5714e44)

0.2.0 (2021-05-28)

Features

multipart support, send read object (323611e)

Bug Fixes

add uclient::Client (7ec95ef)

0.1.4 (2021-05-27)

Features

request_bytes (ff530e2)

Commits

e5454a5 chore(release): 0.2.3

54e263c fix: add multipart header after content disposition

61bf8de test: use into_form_stream for form test

2f043e0 chore(release): 0.2.2

78a2765 feat: use Form Stream

cc5135d feat: add FormStream

13011c3 chore(release): 0.2.1

ab8aff0 chore: remove surf/middleware-logger

5714e44 fix: test

787a5f9 feat: count bytes count of multipart Read object

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies
opened by dependabot[bot] 0
Unit test failing on AqlQueryBuilder

Hey @fMeow , how are you doing ? :smile:

I am still playing with Rust :laughing:

But I got an issue today and I am not sure why. I did updated arangor to 0.5.0 since I am using the latest actix-web now.

Here is the error when I want to use arangors 0.5.0 or unit test it :

Did somebody sae that 0.5.0 tests were failing ?

opened by arn-the-long-beard 5
Make Document and Header derive Clone

Hello, it's a simple request, but is there any reason why these two structures don't derive Clone. I'm trying to return by value and I'm struggling a bit how to get an owned Document out of the local results vector

opened by blagovestb2c2 2
QueryExtra
@fMeow Did you have any plans on how to implement retrieval of QueyExtra data? I would like to implement that asap, as I need it.

I was just scanning through the code and think that extending the existing functions:

aql_fetch_all

aql_query

might be hard. But it could still be convenient to have something integrated. How about adding new functions:

aql_fetch_all_extra

aql_query_extra

which would return QueryExtra data in addition to the results.
opened by inzanez 4
[Question] Writing bare-metal (from scratch container) applications.

Does arangors require any linked Linux user-land components?

I'm looking for a way of creating single binaries that use arrangodb servers that can be instanced as a single file in a "from scratch" container.

In particular does it require any linked security components for SSL connections? or can it perform that with just the binary and a cert?

opened by thawkins 3