Trace output during execution of a query that shows what's happening. Should make debugging a bit easier.

Notable features include.

• Showing all the applicable rules when querying a call. This shows which rules and the order they will be tried after filtering and sorting has been done.
• Showing Queries, Unifys, Matches, Comparisons and Math operations.
• ShowingFFIi lookups and the results that come back.
• Showing relevant bindings (bindings for variables used in the expressions being traced) alongside so it's easier to see what's happening.

Looks something like this.

query> f(1);
[trace]   QUERY: 'f(1)', BINDINGS: {}
[trace]     FILTERING RULES
[trace]       UNIFY: '1' = '_x_4', BINDINGS: {}
[trace]       UNIFY: '1' = '_x_6', BINDINGS: {}
[trace]       UNIFY: '1' = '_x_7', BINDINGS: {}
[trace]     SORTING RULES
[trace]     APPLICABLE_RULES: [
[trace]       f(x) if new (Foo{x: x}, _instance_2) and .(_instance_2, foo(), _value_1) and _value_1 = x;
[trace]       f(x) if x = 1;
[trace]       f(x) if x + 1 and _op_3 == 2;
[trace]     ]
[trace]     RULE:
[trace]     f(x) if
[trace]       new Foo{x: x}.foo() = x
[trace]     UNIFY: '1' = '_x_10', BINDINGS: {}
[trace]       QUERY: 'new (Foo{x: _x_10}, _instance_2_11) and .(_instance_2_11, foo(), _value_1_12) and _value_1_12 = _x_10', BINDINGS: {"_x_10": "1"}
[trace]         QUERY: 'new (Foo{x: _x_10}, _instance_2_11)', BINDINGS: {"_x_10": "1"}
[trace]           UNIFY: '_instance_2_11' = 'Foo{x: 1}', BINDINGS: {}
[trace]         QUERY: '.(_instance_2_11, foo(), _value_1_12) and _value_1_12 = _x_10', BINDINGS: {"_instance_2_11": "Foo{x: 1}", "_x_10": "1"}
[trace]           QUERY: '.(_instance_2_11, foo(), _value_1_12)', BINDINGS: {"_instance_2_11": "Foo{x: 1}"}
[trace]             LOOKUP: Foo{x: 1}.foo()
[trace]             => 2
[trace]           QUERY: '_value_1_12 = _x_10', BINDINGS: {"_x_10": "1", "_value_1_12": "2"}
[trace]             UNIFY: '_value_1_12' = '_x_10', BINDINGS: {"_value_1_12": "2", "_x_10": "1"}
[trace]             UNIFY: '2' = '1', BINDINGS: {}
[trace]             LOOKUP: Foo{x: 1}.foo()
[trace]             => No more results.
[trace]     RULE:
[trace]     f(x) if
[trace]       x = 1
[trace]     UNIFY: '1' = '_x_13', BINDINGS: {}
[trace]       QUERY: '_x_13 = 1', BINDINGS: {"_x_13": "1"}
[trace]         UNIFY: '_x_13' = '1', BINDINGS: {"_x_13": "1"}
[trace]         UNIFY: '1' = '1', BINDINGS: {}
True
[trace]     RULE:
[trace]     f(x) if
[trace]       x + 1 == 2
[trace]     UNIFY: '1' = '_x_14', BINDINGS: {}
[trace]       QUERY: '_x_14 + 1 and _op_3_15 == 2', BINDINGS: {"_x_14": "1"}
[trace]         QUERY: '_x_14 + 1', BINDINGS: {"_x_14": "1"}
[trace]           MATH: '1' + '1' = '_op_3_15', BINDINGS: {}
[trace]           UNIFY: '2' = '_op_3_15', BINDINGS: {}
[trace]         QUERY: '_op_3_15 == 2', BINDINGS: {"_op_3_15": "2"}
[trace]           CMP: '2' == '2', BINDINGS: {}
True
query>

Fixes #1552

This commit catches the assertion error when verifying the right_path does not exist when a path has been identified on the left side of the policy. This refactor was made to better clarify the cause of the error for developers who may not be familiar with the technical limitations of the current SQLAlchemy integration.

In my case, I had assumed that the use of foreign keys and paths on both sides would be more performant, and thus the best choice for a policy, when writing the authorization rules. The resulting error didn't include any context about what was wrong with the policy I had written. This commit aims to address this by providing a best effort guess to provide a starting point for developers to debug the policy issue themselves.

PR checklist:

• [ ] Added changelog entry.

ReferenceError: __dirname is not defined in ES module scope

Unfortunately, oso appears to have trouble running when I build it as part of an ESM project because it references __dirname in certain parts of the code below:

// [email protected]/node_modules/oso/dist/src/polar_wasm_api.js
var path2 = __require("path").join(__dirname, "polar_wasm_api_bg.wasm");

In my own code, where I load the .polar files, I work around this issue using the ESM recommended approach:

import path from 'path';
import { fileURLToPath } from 'url';

const dirname = path.dirname(fileURLToPath(import.meta.url));

await oso.loadFiles([path.resolve(dirname, 'main.polar')]);

• [ ] PolarClass may now be derived on Rust enums.

Hello :wave:

I'm attempting to add support for Rust enums on the #[derive(PolarClass)] proc macro. This involves having the procedural macro generate code to store each enum variant on the Class that is being constructed. A loop in register_class then registers each constant.

I sought to keep the API as simple as possible such that someone looking to add this functionality in their Rust project only needs to add #[derive(PolarClass)] onto their enums (thanks to @joshrotenberg for his input on this 🙂 ). However, this doesn't quite work; I'm hoping some additional eyes on it will help.

The core of the problem seems to be fact that the constants are initially stored on the class as trait objects. Those are then passed around a few times before they get handed to register_constant. The compiler complains about the size of the trait objects not being known. I believe there are ways of handling this in nightly Rust, but I'm not sure what the pattern for addressing this is on stable.

There's also the distinct possibility that I'm missing something simple 🙂

Hello again :slightly_smiling_face: I've just gone for the auto-formatting route this time, except for one change.

I've run isort & Black over all of the Python version of Oso (in that order,) removed the newlines under a few function docstrings that had them as per PEP 257. Black, as far as I know, doesn't have any guide on these, so it shouldn't interfere with anything.

Now, for the manual change - I've removed the newlines between :param:s in oso/oso.py, as they weren't anywhere else - for consistency's sake and all that. Also extra newlines at the end of 2 docstrings in that same file.

I do hope nothing breaks this time.

Here's a small fix to the Ruby gem for Oso that wasn't loading as expected out of the box. It fixes https://github.com/osohq/oso/issues/1089

I wasn't able to find an applicable changelog file to register this. Nor did I know if I was supposed to bump the version in the code. Please let me know if this solution works.

Since it's my first time contributing please give me instructions on accepting the contributors agreement.

Thank you for Oso and the help on Slack!

Hi OsoHQ,

Been enjoying the library, and I hope to use it for an authorization service, however I believe I found a memory leak in the Golang library.

Setup

I was able to reproduce using the oso-go-quickstart example. I forked that repo and the only changes I made were:

• upgraded to the latest oso v0.11.3 -> v0.12.3 (https://github.com/bhb603/oso-go-quickstart/blob/main/go.mod#L5)
• added a policy:

  allow(actor: String, "GET", expense: Expense) if  
          expense.SubmittedBy = actor;
  

• Dockerized the app and deployed it to a kubernetes cluster with this simple deployment (my ultimate use case would be to deploy in kubernetes, which is why I replicated that here)

Testing

I have some load-testing tooling which can generate requests against the service. Since this quickstart example only has one endpoint, it just requests:

curl -H "user: [email protected]" http://oso-go-quickstart.oso:5050/expenses/1

Which would check oso.IsAllowed(User("[email protected]"), "GET", expense) and return:

HTTP/1.1 200 OK
Content-Length: 40
Content-Type: text/plain; charset=utf-8

Expense{{500 coffee [email protected]}}

Results

Memory Usage increases linearly when the service is under load, until it reaches the limit I set for the container (256M) at which point Kubernetes kills the container (OOMKIlled) and it restarts, producing the saw tooth graph below.

The rate of memory increase is proportional to the load (reqs per second), i.e. it increases ~2.5x faster when I was hitting it at 25rps vs 10rps, and flattens out with 0rps.

Memory usage under load:

More Metrics:

Add a job that looks for changes in the docs made by a diff by building the docs against the branch and base branch.

If there are changes, will comment on the PR, as in #937 and require a comment to acknowledge the changes (you must manually re-run the job). The intent of this is to catch PRs that inadvertently break the docs, like changes to code included in code snippets.

If you make a PR changing the docs intentionally, just comment "Docs changes ok" after uploading, and the job will pass on the first run.

If you don't make any changes to the docs (#936), the job also passes, with no comment required.

PR checklist:

• [ ] Added changelog entry.

Hi!

Thanks for that awesome work on the Oso Framework! We are really enjoying working with it!

Currently, we are evaluating whether the sqlalchemy-oso library would fit our needs. Are there any plans regarding the new SQLAlchemy 2 update?

Trying to include it in our project leads to the following error message:

Because sqlalchemy-oso (0.6.0) depends on sqlalchemy (>=1.3.20,<1.4.0) and no versions of sqlalchemy-oso match >0.6.0,<0.7.0, sqlalchemy-oso (>=0.6.0,<0.7.0) requires sqlalchemy (>=1.3.20,<1.4.0).

Would love to hear from you!

• Bumps jnr-ffi to a newer version with Apple Silicon support.
• Updates release workflow to build a fat (x86_64 + aarch64) .dylib and to use that in the .jar build

Fixes the issue I mentioned in https://github.com/osohq/oso/issues/808#issuecomment-1023495977 (at least partially?).

~With the newer version I was able to build the .jar on a M1 machine and the final resulting .jar ran as expected with a aarch64 JRE. However the resulting .jar didn't run on the x86 JRE anymore (though I guess that is expected behavior).~

~Not sure how the .dylib for the release build is built (as the java Makefile references a debug build and specificlally makes a note that it shouldn't be used in release builds?), but I would assume that for properly building a fat .jar some additional step of cross-compiling and building a fat .dylib might be required.~

EDIT: updated PR to include a build of a fat .dylib

The documentation indicates that POLAR_LOG=1 enables tracing. Intuitively you would think that POLAR_LOG=0 disables tracing, particularly if you use the setting in a dotfile/docker-compose.yaml etc.

However, it seems that POLAR_LOG just needs to be set to anything to enable tracing, and to be unset to disable tracing. I've tested this with the go, node, python and rust libraries, and it's the same with all four:

Here's an example using the node library:

❯ which oso
/Users/alexhafner/.volta/bin/oso
❯ echo $POLAR_LOG
0
❯ oso
query> 1+2=3
[debug]   QUERY: 1 + 2 = _op_1 and _op_1 = 3, BINDINGS: {}
[debug]     QUERY: 1 + 2 = _op_1, BINDINGS: {}
[debug]     QUERY: _op_1 = 3, BINDINGS: {_op_1 = 3}
true
query> %
❯ unset POLAR_LOG
❯ oso
query> 1+2=3
true
query> %
❯ export POLAR_LOG=foo
❯ oso
query> 1+2=3
[debug]   QUERY: 1 + 2 = _op_1 and _op_1 = 3, BINDINGS: {}
[debug]     QUERY: 1 + 2 = _op_1, BINDINGS: {}
[debug]     QUERY: _op_1 = 3, BINDINGS: {_op_1 = 3}
true
query>

This was tested on main while reviewing the changes from https://github.com/osohq/oso/pull/1115

It would be helpful for developers if tracing could be disabled by setting POLAR_LOG=0.

This implements refactorings across multiple areas of the Python portions of the Oso framework / library.

1. Use contextlib.suppress rather than try / except in more locations
2. Use yield from instead of yield for a 15–20% performance boost
3. Use f-strings for string concatenation for improved performance and increased legibility
4. Use staticmethod decorators where appropriate
5. Update classmethods to have decorators where appropriate
6. Avoid shadowing built in variables or methods (id -> instance_id)
7. Use de morgans identities for conditionals
8. Use itertools.product for nested for-loop to improve performance of cartesian product generation
9. Do not use mutable default arguments (dictionary was being used, replace it with =None to avoid mutating it accidentally)
10. Use dict to generate dictionary rather than iteration
11. invert control flow to reduce conditional logic
12. raise else clauses that always execute to avoid unnecessary else clauses

PR checklist:

• [ ] Added changelog entry.

Given a table with an Python Enum requires different access pattterns depending on if we filter on the database site via SQL (with the SQLAlchemy integration) or on the Python side just with OSO.

import enum
from sqlalchemy import coveting

class AccessLevel(enum.Enum):  # <- Python Enum
    public = "private"
    private = "public"

class Post(Model):
    __tablename__ = "posts"
    id = Column(Integer, primary_key=True)
    content = Column(String, nullable=False)
    access_level = Column(Enum(AccessLevel), nullable=False)
    # ^- Enum DB column, but with a Python Enum as value (SQLAlchemy does the translation automatically).

For just checking permissions on Python (authorize(user_obj, "read", post_obj)), I have to write this rule with .value, as post.access_level is of type enum.Enum.

allow(_: User, "read", post: Post) if
  post.access_level.value = "public";

However when I try to filter on the database side, with an authorized_sessionmaker, the rule doesn't work any more. OSO parses the rule wrong, and believes that access_level is a relationship to another table with a column value. So you get stuck in some method converting the rule into an SQL query.

    s.query(Post).all()
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy/orm/query.py:2768: in all
    return self._iter().all()
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy/orm/query.py:2903: in _iter
    result = self.session.execute(
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy/orm/session.py:1693: in execute
    result = fn(orm_exec_state)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/session.py:311: in do_orm_execute
    filter = authorize_model(oso, user, action, session, entity)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/auth.py:111: in authorize_model
    filter = partial_to_filter(
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:124: in partial_to_filter
    return translate_expr(expression, session, model, get_model)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:139: in translate_expr
    return translate_and(expression, session, model, get_model)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:154: in translate_and
    translated = translate_expr(expression, session, model, get_model)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:137: in translate_expr
    return translate_in(expression, session, model, get_model)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:293: in translate_in
    return translate_dot(
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:338: in translate_dot
    return property.any(translate_dot(path[1:], session, model, func))
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:332: in translate_dot
    return func(session, model)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:363: in emit_subexpression
    return translate_expr(sub_expression, session, model, get_model)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:139: in translate_expr
    return translate_and(expression, session, model, get_model)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:154: in translate_and
    translated = translate_expr(expression, session, model, get_model)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:133: in translate_expr
    return translate_compare(expression, session, model, get_model)
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:236: in translate_compare
    return translate_compare(
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:228: in translate_compare
    return translate_dot(
../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:334: in translate_dot
    property, model, is_multi_valued = get_relationship(model, path[0])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

model = <class 'tables.Post'>, field_name = 'access_level'

    def get_relationship(model, field_name: str):
        """Get the property object for field on model. field must be a relationship field.

        :returns: (property, model, is_multi_valued)
        """
        property = getattr(model, field_name)
>       assert isinstance(property.property, RelationshipProperty)
E       AssertionError

../../../.local/share/virtualenvs/MoaO0l74/lib/python3.9/site-packages/sqlalchemy_oso/partial.py:347: AssertionError

It works with

allow(_: User, "read", post: Post) if
  post.access_level = "public";

But then of course it doesn't work on the Python side anymore.

Obviously it works if you define an Enum directly as in the https://docs.osohq.com/reference/frameworks/data_filtering/sqlalchemy.html tutorial, by using strings. With access_level = Column(Enum("public", "private"), nullable=False). However I think it's quite common to define the Enum separately on the Python side, and then reuse that enum in the ORM definition, and SQLAlchemy supports this since 1.1. It's just more safer, if you have Enums in the Python business logic, instead of having to deal with strings.

Scenario: I have a web application, whose operations I'd like to authorize. I'd like to be able to write

#[derive(Debug, Deserialize, PolarClass)]
struct BootstrapReq {}

#[post("bootstrap")]
async fn bootstrap(
    req: web::Json<BootstrapReq>,
    oso: web::Data<Oso>,
    identity: Identity,
) -> Result<impl Responder> {
    let req = req.into_inner();
    if oso
        .is_allowed(identity, "bootstrap", &req) // <-- borrow 'req'
        .map_err(oso_to_actix)?
    {
        Ok(format!("{:#?}", req)) // <-- do something interesting with 'req'
    } else {
        Err(error::ErrorForbidden("forbidden"))
    }
}

Unfortunately, the PolarClass derivation ends up providing an implementation of ToPolar for BootstrapReq (by value), but not for &BootstrapReq (by reference). This means that the authorization check actually consumes the value, making it unusable for the rest of the request body.

I can work around this for now with some defensive clone()s, but it's fairly unergonomic.

This looks straightforward, and I've implemented it locally in an extension trait:

use oso::{Oso, ToPolar};

/// Oso extension to add support for `allow_request` and request-based ACLs.
pub trait AllowRequest {
    /// Runs request-level authorization checks.
    fn is_request_allowed<Actor, Resource>(
        &self,
        actor: Actor,
        resource: Resource,
    ) -> oso::Result<bool>
    where
        Actor: ToPolar,
        Resource: ToPolar;
}

impl AllowRequest for Oso {
    fn is_request_allowed<Actor, Resource>(
        &self,
        actor: Actor,
        resource: Resource,
    ) -> oso::Result<bool>
    where
        Actor: ToPolar,
        Resource: ToPolar,
    {
        let mut query = self.query_rule("allow_request", (actor, resource))?;
        match query.next() {
            Some(Ok(_)) => Ok(true),
            Some(Err(e)) => Err(e),
            None => Ok(false),
        }
    }
}

Scenario: I have an identity class that looks like

/// An identity annotates a request and describes on whose behalf we are
/// processing it.
#[derive(Clone)]
pub enum Identity {
    /// The non-identity identity. Still exposes information like what host
    /// the request arrived from.
    Anonymous(SocketAddr),
    // future alternatives such as User(User, SocketAddr) and Service(Service, SocketAddr) here
}

This type can't be used with #[derive(PolarClass)], but that's okay, because the PolarClass trait is straightforward to implement manually. Thank you!

I'd like to be able to map this to Polar, so that I can use it in authorization checks as-is. However, the peer address is a mandatory part of the identity, and is used to say things like "this service is allowed to make this request only from this network range."

I can almost get there today, but ClassBuilder and Class hard-require that every polar class have a constructor, and that constructors be infallible. That's not possible in any reasonable way I can find for anything that has a SocketAddr argument: at some point in the process, I will have to parse a string, which is a fallible operation, and the only way Polar's constructor protocol can tolerate that is if the operation panics.

I'm working around this today with an Unknown variant, which is only used by Polar and is the result of constructing a new Identity in Polar. This isn't very satisfying, as it's dead code. I would love if I could write something like

        ClassBuilder::no_constructor()
            .name("Identity")
            // …other methods…
           .build()

in my PolarClass implementation. The closest I can get today is ClassBuilder::with_constructor(|| -> Self { panic!() }), which works, but makes my "never panic" aesthetic harder to achieve.