Basiliq

Exposing a Postgres database via a REST API that follows the JSON:API specs.
All in all, a tasty API.

What is Basiliq
Quickstart
- Ready to use example
- Running locally
Understanding the API
- How to query
- Example requests
The configuration
Testing

What is Basiliq

Basiliq is a very alpha REST API that replaces the need to write CRUD methods by exposing a standardized API to interact with a Postgres database

It follows the established JSON:API specifications. Written in Rust, it tries to conciliate performance with stability.

Quickstart

Ready to use example

You could try out the API already deployed on Heroku.

For instance:

# For a very simple example
curl 'http://demo.basiliq.io/public__peoples'

# For a more complex example
curl 'http://demo.basiliq.io/public__peoples?include=public__articles,public__comments&fields\[public__comments\]='

Running locally

One can install basiliq through docker. An example docker-compose script is available at the root of the repository. To start it:

# To start
docker-compose -f docker-compose.example.yml up -d

# At that point the database is empty,
# you can populate it with the following script
curl -L https://gitlab.com/basiliqio/basiliq_db_test_utils/-/jobs/artifacts/main/raw/basiliq_test.dump\?job\=pack_test_migrations | PGHOST=localhost PGUSER=postgres PGPASS=postgres psql

# Then you can restart the basiliq server so it rescans the database
docker-compose -f docker-compose.example.yml restart basiliq

# To stop
docker-compose -f docker-compose.example.yml down

Understanding the API

How to query

In the future, there should be a way to generate an OpenApi document to view exactly how the API is accessible.

The API response follows the JSON:API specifications.

By default, the endpoints are exposed in the format schema__table (i.e. for a table peoples in the public schema, the endpoint would be public__table).

By modifying the configuration one could change how those endpoints are exposed.

Example requests

Creating request

Notice the lack of id in the request.

Also, in the response, the fields that were not specified are set to their default

POST /public__peoples HTTP/1.1
Host: demo.basiliq.io
User-Agent: curl/7.76.1
Content-Type:application/vnd.api+json
Accept: application/json, */*
Content-Length: 174

{
    "data": {
        "type": "public__peoples",
        "attributes": {
            "first-name": "Somebody",
            "last-name": "Once_told_me_the_world",
            "gender": "F",
            "twitter": "@allstars"
        }
    }
}

HTTP/1.1 201 Created
Connection: keep-alive
Content-Type: application/vnd.api+json
Content-Length: 224
Date: Sun, 02 May 2021 20:20:52 GMT

{
    "jsonapi": {
        "version": "1.0"
    },
    "data": {
        "type": "public__peoples",
        "id": "d14e1928-9cae-441c-945d-144ebe6c94c8",
        "attributes": {
            "age": null,
            "first-name": "Somebody",
            "gender": "F",
            "last-name": "Once_told_me_the_world",
            "twitter": "@allstars"
        }
    }
}

Simple fetching request

GET /public__peoples HTTP/1.1
Host: demo.basiliq.io
User-Agent: curl/7.76.1
Accept: application/json, */*

HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/vnd.api+json
Content-Length: 598
Date: Sun, 02 May 2021 20:13:47 GMT

{
    "jsonapi": {
        "version": "1.0"
    },
    "data": [
        {
            "type": "public__peoples",
            "id": "1649b1e9-8a5f-4f52-b331-c07ce3bccc6f",
            "attributes": {
                "age": 22,
                "first-name": "Francis",
                "gender": "M",
                "last-name": "Le Roy",
                "twitter": null
            }
        },
        {
            "type": "public__peoples",
            "id": "777cc565-c66b-4942-ab44-8fc5f194b804",
            "attributes": {
                "age": 34,
                "first-name": "Somebody",
                "gender": "F",
                "last-name": "Wuhu",
                "twitter": "@randomhandle"
            }
        },
        {
            "type": "public__peoples",
            "id": "961e543a-4b22-4d48-a8e5-c1eafada950f",
            "attributes": {
                "age": null,
                "first-name": "AAAAAAAA",
                "gender": null,
                "last-name": "BBBBBBBBB",
                "twitter": null
            }
        }
    ]
}

Fetching requests including relationships and sparsing fields

You can find the attributes of the objects in the relationships key of each main resource in the included key below.

Notice that the comments object have only ids, because none of their fields was selected via the fields[public__comments]= query parameter.

GET /public__peoples?include=public__articles,public__comments&fields[public__comments]= HTTP/1.1
Host: demo.basiliq.io:80
User-Agent: curl/7.76.1
Accept: application/json, */*

HTTP/1.1 200 OK
content-type: application/vnd.api+json
content-length: 1879
date: Sun, 02 May 2021 20:08:12 GMT

{
    "jsonapi": {
        "version": "1.0"
    },
    "data": [
        {
            "type": "public__peoples",
            "id": "1649b1e9-8a5f-4f52-b331-c07ce3bccc6f",
            "attributes": {
                "age": 22,
                "first-name": "Francis",
                "gender": "M",
                "last-name": "Le Roy",
                "twitter": null
            },
            "relationships": {
                "public__articles": {
                    "data": [
                        {
                            "type": "public__articles",
                            "id": "fdf715dd-8772-498c-8196-6f4ccb64edef"
                        },
                        {
                            "type": "public__articles",
                            "id": "2dbf5d1a-b029-4456-af6b-339c75b1089c"
                        }
                    ]
                },
                "public__comments": {
                    "data": [
                        {
                            "type": "public__comments",
                            "id": "59f58abd-c9db-4074-9c34-ac33e9c838ce"
                        },
                        {
                            "type": "public__comments",
                            "id": "c2add83b-6f58-45a2-bf62-3ebc05c46192"
                        }
                    ]
                }
            }
        },
        {
            "type": "public__peoples",
            "id": "777cc565-c66b-4942-ab44-8fc5f194b804",
            "attributes": {
                "age": 34,
                "first-name": "Somebody",
                "gender": "F",
                "last-name": "Wuhu",
                "twitter": "@randomhandle"
            },
            "relationships": {
                "public__articles": {
                    "data": {
                        "type": "public__articles",
                        "id": "46c4fe50-8c56-4f26-935e-56ccfa496bb5"
                    }
                },
                "public__comments": {
                    "data": {
                        "type": "public__comments",
                        "id": "6ae9938f-d490-4707-b138-770c2a52465f"
                    }
                }
            }
        },
        {
            "type": "public__peoples",
            "id": "961e543a-4b22-4d48-a8e5-c1eafada950f",
            "attributes": {
                "age": null,
                "first-name": "AAAAAAAA",
                "gender": null,
                "last-name": "BBBBBBBBB",
                "twitter": null
            }
        }
    ],
    "included": [
        {
            "type": "public__articles",
            "id": "2dbf5d1a-b029-4456-af6b-339c75b1089c",
            "attributes": {
                "body": "Yeah I know ! Right ?!",
                "title": "Oh my g**"
            }
        },
        {
            "type": "public__articles",
            "id": "46c4fe50-8c56-4f26-935e-56ccfa496bb5",
            "attributes": {
                "body": "They feast on the blood of the departed draw their powers",
                "title": "Why devs require sacrifices"
            }
        },
        {
            "type": "public__articles",
            "id": "fdf715dd-8772-498c-8196-6f4ccb64edef",
            "attributes": {
                "body": "Yes",
                "title": "How to dead"
            }
        },
        {
            "type": "public__comments",
            "id": "59f58abd-c9db-4074-9c34-ac33e9c838ce"
        },
        {
            "type": "public__comments",
            "id": "6ae9938f-d490-4707-b138-770c2a52465f"
        },
        {
            "type": "public__comments",
            "id": "c2add83b-6f58-45a2-bf62-3ebc05c46192"
        }
    ]
}

Updating request

Notice that attributes that were not included in the PATCH request are not nulled.

PATCH /public__peoples/777cc565-c66b-4942-ab44-8fc5f194b804 HTTP/1.1
Host: demo.basiliq.io
User-Agent: curl/7.76.1
Content-Type:application/vnd.api+json
Accept: application/json, */*
Content-Length: 204

{
    "data": {
        "type": "public__peoples",
        "id": "777cc565-c66b-4942-ab44-8fc5f194b804",
        "attributes": {
            "first-name": "NotTheOriginalFirstName",
            "last-name": "NotTheOriginalLastName"
        }
    }
}

HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/vnd.api+json
Content-Length: 260
Date: Sun, 02 May 2021 20:24:50 GMT

{
    "jsonapi": {
        "version": "1.0"
    },
    "data": {
        "type": "public__peoples",
        "id": "777cc565-c66b-4942-ab44-8fc5f194b804",
        "attributes": {
            "age": 34,
            "first-name": "NotTheOriginalFirstName",
            "gender": "F",
            "last-name": "NotTheOriginalLastName",
            "twitter": "@randomhandle"
        }
    }
}

Deleting request

DELETE /public__peoples/777cc565-c66b-4942-ab44-8fc5f194b804 HTTP/1.1
Host: demo.basiliq.io
User-Agent: curl/7.76.1
Accept: application/json, */*

HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/vnd.api+json
Content-Length: 41
Date: Sun, 02 May 2021 20:25:54 GMT

{
    "jsonapi": {
        "version": "1.0"
    },
    "data": null
}

The configuration

Generating

Typically, one would first need to create a configuration, however this is not mandatory to run basiliq.

To create a configuration, one can use :

basiliq config generate

It would generate a file called basiliq_config.yaml in the current working directory.

What's in there

This file would look like the following :

---
resources:                 # The list of resources
  public__articles:        # The name of a resource. *It can be changed*
    target:                # The identifier object of the resource
      schema: public       # The schema this resource is bound to in the database
      table: articles      # The name of the table bound to this resource
    enabled: true          # `true` if this resource is enabled
    relationships:         # A list of relationships
      public__comments:    # Name of the relationship. *It can be changed*
        target:            # The identifier object of the resource
          schema: public   # The schema this relationship's resource is bound to in the database
          table: comments  # The name of the table bound to this relationship's resource
        enabled: true      # `true` if this relationship is enabled
        field: article     # The field to which this relationship is bound
      public__peoples:
        target:
          schema: public
          table: peoples
        through:           # Identifies the bucket table for Many-to-Many relationships 
          schema: public   # The schema of the bucket resource
          table: comments  # The table of the bucket resource
          field: author    # The field linking the relationship's resource and the bucket resource
        enabled: true
        field: id
[...]

Checking the configuration

After having generated the configuration, one might need to ensure it's correct after having modified it.

One could do that with the following command:

basiliq config check --input basiliq_config.yaml

Testing

To test this crate, you need to start a Postgres database and export the DATABASE_URL environment variable.

You can use the provided docker-compose plan

# To start the test database
docker-compose -f docker-compose.testing.yml up -d

# Don't forget to set the environment variable
export DATABASE_URL="postgres://postgres:postgres@localhost/postgres"

# Run the tests
cargo test

# To stop the test database
docker-compose -f docker-compose.testing.yml down -v

Plugin system
To allow for more modularity, a plugin system would be most welcome.

But multiple questions are to be asked before diving into the code:

[ ] How the plugins will interact the main process (Shared library, WASM module, sub-process, etc..)

[ ] How many plugin levels should there be (pre-requests, post-requests, pre-db-scanning, etc...)

[ ] How to ensure those plugins can be trusted (digital signature like PGP ?)

[ ] What are the performance problems induced by loading externals resources rather than developing those features in the main project.

enhancement help wanted question
opened by GrandChaman 0
Sign configuration
The configuration of Basiliq is very precious. If somebody were to tinker with it, it could have disastrous consequences for the service.

There should be an option to verify a signature along with the configuration to ensure that the configuration has only been tampered with by trusted members.

[ ] Verify detached PGP signatures

[ ] Import public keys from local storage (dangerous?)

[ ] Import public keys from the web (dangerous?)

enhancement help wanted
opened by GrandChaman 0
Configure which database schemas to scan
There should be an option to choose from which database schema is scanned.

[ ] Choose a single database schema to scan

[ ] Choose multiple database schemas to scan

enhancement good first issue
opened by GrandChaman 0
Use scapegoat tree

In many places of this project, there's heavy use of the BTreeMap from rust collection.

The scapegoat crate could help with performance by using stack allocated array for small maps.

The API of the crates looks very similar to BTreeMap's one, it shouldn't be too difficult to make the switch.
enhancement good first issue

opened by GrandChaman 0
Better support for Postgres errors
There should be a better support for Postgres errors. It should differentiate between multiple errors types.

[ ] Detect duplicate error

[ ] Detect CHECK constraint error

[ ] Detect uniqueness error

[ ] Detect missing field error

enhancement
opened by GrandChaman 0
Disable resources
There should be an option to disable a resource.

TODOs:

[ ] Option to prevent a resource from beeing accessed, as a relationship, or a as a main resource

[ ] Option to allow a disabled resource to be a "bucket" for "Many-to-Many" relationships, but not beeing accessible directly.

enhancement
opened by GrandChaman 0