A fast, safe and easy to use reinforcement learning framework in Rust.


RSRL (api)

Crates.io Build Status Coverage Status

Reinforcement learning should be fast, safe and easy to use.


rsrl provides generic constructs for reinforcement learning (RL) experiments in an extensible framework with efficient implementations of existing methods for rapid prototyping.


rsrl = "0.8"

Note that rsrl enables the blas feature of its ndarray dependency, so if you're building a binary, you additionally need to specify a BLAS backend compatible with ndarray. For example, you can add these dependencies:

blas-src = { version = "0.2.0", default-features = false, features = ["openblas"] }
openblas-src = { version = "0.6.0", default-features = false, features = ["cblas", "system"] }

See ndarray's README for more information.


The code below shows how one could use rsrl to evaluate a QLearning agent using a linear function approximator with Fourier basis projection to solve the canonical mountain car problem.

See examples/ for more...

let env = MountainCar::default();
let n_actions = env.action_space().card().into();

let mut rng = StdRng::seed_from_u64(0);
let (mut ql, policy) = {
    let basis = Fourier::from_space(5, env.state_space()).with_bias();
    let q_func = make_shared(LFA::vector(basis, SGD(0.001), n_actions));
    let policy = Greedy::new(q_func.clone());

    (QLearning {
        gamma: 0.9,
    }, policy)

for e in 0..200 {
    // Episode loop:
    let mut j = 0;
    let mut env = MountainCar::default();
    let mut action = policy.sample(&mut rng, env.emit().state());

    for i in 0.. {
        // Trajectory loop:
        j = i;

        let t = env.transition(action);

        action = policy.sample(&mut rng, t.to.state());

        if t.terminated() {

    println!("Batch {}: {} steps...", e + 1, j + 1);

let traj = MountainCar::default().rollout(|s| policy.mode(s), Some(500));

println!("OOS: {} states...", traj.n_states());


Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate and adhere to the angularjs commit message conventions (see here).



  • Agents requiring mutability for evaluation (action selection)

    Agents requiring mutability for evaluation (action selection)

    Hi! First of all, big thanks for this crate.

    I had some questions regarding agents (or controllers) that require mutability (&mut self) for selecting an action that is currently not supported. An example class of agents are online solvers or planners in partially observable environments. These agents often learn an approximate policy, or some other heuristic, and use that heuristic to guide a policy search or other planning method relevant only to the current 'state'. However, this current state is usually some function of the entire history of actions and observations up until that point.

    Thus updating this current state requires mutating some field in the agent, and thus requires a &mut self. This also implies the agent needs some to be 'reset' after an episode, akin to the current handle_terminal of the OnlineLearner trait. Although a copy could also be sufficient to start from the blank 'initial state'.

    A concrete example of this is the infinite POMDP [1] (to which my research is related), but in fact it is relevant to any agent that incorporates data from the current episode to have an effect on planning.

    Now I was wondering:

    1. Would you be interested in having a compatible API for those agents in this crate?
    2. If so, how would you see an implementation be incorporated here? (I could make a PR)
    • Change the controller trait to take a &mut self and add a handle_terminal method. Conceptually this generalizes the Controller trait, as every struct implementing the current trait could trivially implement the generalization. But this does not seem to be required for the Deep RL agents this crate is focused on (and as such would dirty the interface).
    • Add an OnlineController trait with the proposed changes, and implement it for all the controllers (which should be trivial). Conceptually you have an implementation for OnlineController when you have one for Controller. It should be possible to express that using Rust's trait system (playground example).
    • Anything else you suggest.

    Any feedback would be appreciated. To me it seems useful to support this case (albeit in a type alone), as it would allow writing more agents to the type interface this crate provide. But I understand I might be biased.

    Thanks in advance.

    PS: I have also wondered if it would be useful to separate some types (mostly the learner, domain and controller ones) into a separate crate, as that would allow implementing an agent against these traits without pulling in all the dependencies for all other agents and domains. But I'll keep proposal that for a separate issue.

    [1] Doshi-velez, Finale. ‘The Infinite Partially Observable Markov Decision Process’. In Advances in Neural Information Processing Systems 22, edited by Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, and A. Culotta, 477–485. Curran Associates, Inc., 2009. http://papers.nips.cc/paper/3780-the-infinite-partially-observable-markov-decision-process.pdf.

    opened by wschella 8
  • `Shared<T>` can't be unwrapped or serialized

    `Shared` can't be unwrapped or serialized

    I'm interested in getting at the internals of a QLearning in order to serialize just the q_func member. It's the only part relevant to my application after training, and so I'd like to unwrap it from the shared pointer so I can serialize just the wrapped function approximation. However, because the Rc<RefCell<_>> contained within Shared is private, it seems to be impossible to unwrap the contained values in a safe manner---only borrowing is permitted. And, because Shared does not implement Serialize or Deserialize, I can't serialize it all as a unit.

    As an alternative, I would suggest making Shared a type alias:

    type Shared<T> = Rc<RefCell<T>>;

    That would make many useful methods available for working with these values, such as Rc::try_unwrap. After unwrapping a function approximator, I could then serialize it if it implements the proper traits, which many (or all?) in the lfa package do.

    Happy to send a pull request if you think this is a good direction to move in.

    opened by joshhansen 7
  • Add rendering, fix bugs in Acrobot

    Add rendering, fix bugs in Acrobot

    I have used this crate's Acrobot in my work lately. I noticed after a while that something was wrong: When I exported the states visited by my agent, it looked unnatural and maybe bugged. So I implemented rendering using ggez. Why ggez: because it is the only crate I found where you can create shapes as easily as in Python. Then I carefully made sure that everything in our Acrobot should be equivalent to what happens in the Python implementation (fixed a typo and reimplemented runge_kutta4). I found the result satisfactory - I tested it by playing with Acrobot myself using this code. Just cargo run and use left and right arrows. I think it is crucial to visualize or verify the dynamics of environments/domains we implement. What other domains have not been tested?

    • Fix a typo in Acrobot::grad
    • Add optional render() to Domain
    • Rewrite runge_kutta4 to be equivalent to the one used in the Python implementation of Acrobot.
    • Change name Acrobat -> Acrobot

    Important: I had to propagate the changes I did in runge_kutta4, to the domains hiv and cart_pole. I have not verified that the changes in these environments are correct yet, so this needs review.

    I would add the code that lets you play Acrobot yourself as an example or a bin maybe, but I can still not do any of cargo run --example .., cargo run --bin .. or cargo test because of #54.

    bug priority 
    opened by Ploppz 4
  • `DerefSlice` is difficult to implement

    `DerefSlice` is difficult to implement

    If I understand correctly, DerefSlice is how domain state is mapped to a feature vector for learning. I am finding that the &[f64] borrowed return type is difficult to work with. My game state is not intrinsically a Vec<f64> or similar, so I have to build the vector within deref_slice. Yet because it is a reference, it needs to point to a value that will live longer than the method call. I could put an owned Vec<f64> within the game state struct, but deref_slice takes an immutable &self so I could not update that vector during the deref_slice call before returning it. Which would mean that I have to modify the owned feature vector at some other time, such as by reacting to every mutation of the game state, which would be difficult to enforce.

    To work around this I have made my own DerefVec trait which returns a Vec<f64> rather than a slice.

    The alternative I see is making deref_slice take a mutable reference to self, but that cascades through the entire API, making every reference to a game state have to be mutable. Maybe that would be okay, though?

    The DerefVec approach is a simple change but I could send a pull request if you want. Not helpful performance-wise I'd imagine, but maybe the flexibility would be worth it? Or perhaps you could suggest a better approach than what I have been considering.

    opened by joshhansen 3
  • Mention need to select BLAS backend in README

    Mention need to select BLAS backend in README

    I hope the new text is self-explanatory. See also rust-ndarray/ndarray#757.

    Note that it may be useful to give the user the option whether or not to enable ndarray's blas feature (by adding a blas feature to rsrl that enables ndarray/blas), instead of always enabling it.

    cc @baloo

    opened by jturner314 3
  • Can't run examples - linker errors to blas

    Can't run examples - linker errors to blas

    cargo run --example a2c -> note: /usr/bin/ld: cannot find -lopenblas

    Then I installed openblas instead of blas (Arch Linux). Now it says:

      = note: /usr/bin/ld: /tmp/rsrl/target/debug/deps/liblfa-9d758421e5e2d7cc.rlib(lfa-9d758421e5e2d7cc.lfa.9q6n3def-cgu.1.rcgu.o): in function `ndarray::linalg::impl_linalg::<impl ndarray::ArrayBase<S, ndarray::dimension::dim::Dim<[usize; _]>>>::dot_impl':
              /home/ploppz/.cargo/registry/src/github.com-1ecc6299db9ec823/ndarray-0.12.1/src/linalg/impl_linalg.rs:115: undefined reference to `cblas_sdot'
              /usr/bin/ld: /home/ploppz/.cargo/registry/src/github.com-1ecc6299db9ec823/ndarray-0.12.1/src/linalg/impl_linalg.rs:115: undefined reference to `cblas_ddot'
              /usr/bin/ld: /tmp/rsrl/target/debug/deps/liblfa-9d758421e5e2d7cc.rlib(lfa-9d758421e5e2d7cc.lfa.9q6n3def-cgu.1.rcgu.o): in function `ndarray::linalg::impl_linalg::<impl ndarray::ArrayBase<S, ndarray::dimension::dim::Dim<[usize; _]>>>::dot_impl':
              /home/ploppz/.cargo/registry/src/github.com-1ecc6299db9ec823/ndarray-0.12.1/src/linalg/impl_linalg.rs:115: undefined reference to `cblas_sdot'
              /usr/bin/ld: /home/ploppz/.cargo/registry/src/github.com-1ecc6299db9ec823/ndarray-0.12.1/src/linalg/impl_linalg.rs:115: undefined reference to `cblas_ddot'
              /usr/bin/ld: /tmp/rsrl/target/debug/deps/liblfa-9d758421e5e2d7cc.rlib(lfa-9d758421e5e2d7cc.lfa.9q6n3def-cgu.12.rcgu.o): in function `ndarray::linalg::impl_linalg::mat_mul_impl':
              /home/ploppz/.cargo/registry/src/github.com-1ecc6299db9ec823/ndarray-0.12.1/src/linalg/impl_linalg.rs:422: undefined reference to `cblas_sgemm'
              /usr/bin/ld: /home/ploppz/.cargo/registry/src/github.com-1ecc6299db9ec823/ndarray-0.12.1/src/linalg/impl_linalg.rs:422: undefined reference to `cblas_dgemm'
              collect2: error: ld returned 1 exit status

    I tried to disable anything related to blas in Cargo.toml, changed lines look like:

    ndarray = { version = "0.12", features = ["serde-1"] }
    # ...
    blas-src = { version = "0.2", default-features = false, features = [] }
    openblas-src = { version = "0.6", default-features = false, features = ["system"] }

    But that did not help at all.

    framework priority 
    opened by Ploppz 3
  • Examples don't seem to build

    Examples don't seem to build

    I'm trying out the library for the first time and I'm having trouble getting it to build. rust 1.33: I pulled the code from here: https://github.com/tspooner/rsrl/blob/master/examples/greedy_gq.rs

    error[E0599]: no function or associated item named `scalar_output` found for type `lfa::lfa::LFA<_, _, _>` in the current scope
      --> src/main.rs:20:39
    20 |         let v_func = make_shared(LFA::scalar_output(bases.clone()));
       |                                  -----^^^^^^^^^^^^^
       |                                  |
       |                                  function or associated item not found in `lfa::lfa::LFA<_, _, _>`
    error[E0599]: no function or associated item named `vector_output` found for type `lfa::lfa::LFA<_, _, _>` in the current scope
      --> src/main.rs:21:39
    21 |         let q_func = make_shared(LFA::vector_output(bases, n_actions));
       |                                  -----^^^^^^^^^^^^^
       |                                  |
       |                                  function or associated item not found in `lfa::lfa::LFA<_, _, _>`
    error[E0061]: this function takes 2 parameters but 3 parameters were supplied
      --> src/main.rs:24:34
    24 |           let policy = make_shared(EpsilonGreedy::new(
       |  __________________________________^
    25 | |             Greedy::new(q_func.clone()),
    26 | |             Random::new(n_actions),
    27 | |             Parameter::exponential(0.3, 0.001, 0.99),
    28 | |         ));
       | |_________^ expected 2 parameters
    error: aborting due to 3 previous errors
    opened by cholcombe973 3
  • Introduce new prediction algorithms

    Introduce new prediction algorithms

    This PR adds a selection of new algorithms to the agents::prediction module. These include:

    1. TDC - corrected temporal difference learning
    2. GTD2 - gradient temporal difference learning (v2)
    3. EveryVisitMC

    These have been implemented based on the specification from the excellent book "Algorithms for Reinforcement Learning" by Csaba Szepesvári.

    As part of these changes, a new method equivalent was added to the Parameterised trait which compares the form of the function approximator (FA). Though very similar to the PartialEq and Eq built-in traits, this method does not compare weight values, just their representation. This is particularly relevant when using linear function approximation and require two FAs to have the same feature mapping.

    opened by tspooner 3
  • Messaging


    This PR includes a whole host of major changes to the framework. Included are changes to the sub-crate rsrl_domains, and a totally revised interface for algorithms and utilities. We now have traits like Function and Handler that abstract away a lot of the particulars of a given algorithm or methodology. This adds support and the possibility of integrating in a lot of nice features.

    opened by tspooner 2
  • Compatible-features inspired overhaul of FA

    Compatible-features inspired overhaul of FA

    This PR includes a ridiculous amount of changes and will take too long to explain in detail. It's taken a long time to get here but the framework is much more usable now. There are still a lot of things that need refinement (such as the gradients module in fa) but on the whole it's a major improvement.

    opened by tspooner 2
  • Native tile coding

    Native tile coding

    In this pull request we transition from using Sutton's original implementation of tile coding to our own, rust-native version. Based on Sutton's Tiles v3.0, we implement a standard fixed tile coding projector and a SparseLinear function approximator that makes use of the a prior known number of active features.

    We use the fxhash crate from Mozilla to confer the coordinates into a single fixed integer which is restricted into a finite range memory_size.

    opened by tspooner 2
  • Undefined symbols for architecture x86_64:

    Undefined symbols for architecture x86_64: "_cblas_sdot", referenced from

    I am getting the following error

    error: linking with `cc` failed: exit status: 1
      = note: "cc" "-m64" "-arch" "x86_64" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.100y407nzibn0v3o.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.12fpy1ksm5b5771v.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.14whfsbe2lcxoapk.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.192oyids7prk5eak.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1a8guv79wf0w2owl.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1fxtzy95z89s9yds.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1gbs1l25ecmtzmo3.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1ieojwfdnkvtrij2.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1iqk174wrz5bb6g7.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1ngo0ogi26nj7tfs.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1otzxuhrun3jzim7.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1qrk2qr4pbp511z9.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1y1yccjxg0tgog71.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1yzn9rhpag5bpwxn.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.20p8fx2bo2u1s4mj.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2956pe0rphowevhw.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2ak4z2efokpnj3td.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2bz8ycgnn6hhev0h.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2ckm2yzjpcl3jny6.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2fled2w4ooq90zuk.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2hwesdyv23sptms1.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2o8y11fk5tgujask.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2owsj8kl90dlaxxp.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2v2padmultfc015y.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.2wse9sp58p7tqbtr.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3dwvkphta04bi868.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3fb2rxk5jm893xyb.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3gd2labfrthxxd50.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3himc58hdgpw3unn.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3ii8jqj9gxl7f31m.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3j3wixlkhztc6z4t.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3kxtq6scafmnga6h.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3osa6k8a1pqx53v4.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3rf93atp0n8njfsv.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3rvs1gkzngol62tu.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3ttuv0ig93sp77rw.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3u5u630lg9kxy47f.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3uceqz4cjnmmfpij.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3y2cd2v75ov8r7ht.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.3z7a2debunokcznq.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4gqp4wxhenaovfkw.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4i7h9fqmrfxgd6ez.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4juan1oz6fo9855y.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4svygv1xlv7jqc3c.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4uee5ebjek885baa.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4uk04dte9ypkbwv0.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4uu1ew08p7ffmbq1.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4xr7585xl8e5i4wn.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4yxss856ryk9zxh2.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4z16n4prmh1dz2ej.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.4zbclpl7m4cb7k7p.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.55gegb2fvsj89bcp.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.5cy9y1snjlixn7t4.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.5da7irx8hhx62xvn.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.5dc08uor1t355s61.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.5govva21qv8w4ut8.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.5r1ln9fhqn2gpfq.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.6uj48lwsvm2hat5.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.ansa3210uweletc.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.c0ecfufw1vhhl68.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.gl13bbtwu50ag1q.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.ied7wclj6f5c0ww.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.k8sijrvuy9j0i7u.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.m1djheb0w9nvpvk.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.muggb6girkci1zj.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.tesie8admfr6rbq.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.txtvniqiyhn7lj3.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.udegtedpktr8urt.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.vwropns9g45fwkm.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.z9mh5j704mzse17.rcgu.o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e.1h90cwm9qwj8iurc.rcgu.o" "-L" "/Users/nbro/Desktop/rsrl/target/debug/deps" "-L" "/Users/nbro/Desktop/rsrl/target/debug/build/special-fun-5e8e40e4208f6444/out" "-L" "/Users/nbro/Desktop/rsrl/target/debug/build/special-fun-5e8e40e4208f6444/out" "-L" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librsrl-1b46111e84e0290f.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librstat-0b8ae75a6e0fde36.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libspecial_fun-35f87730bf5411c3.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libnum-5c5ed9a8701d379d.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libnum_rational-9049ae633a8bdb91.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libnum_iter-586682b9833ce2cd.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libnum_bigint-4b0e2b6a4cbe708c.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libfailure-1ceba649c9f6c838.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libbacktrace-29a4bced12e05068.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libobject-57fcdcd7d7e9e3cb.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libmemchr-7874de389e22e218.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libaddr2line-7d6c45590d806154.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libgimli-7e508834f36b14c6.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librustc_demangle-0f85c436a3cee7bb.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librsrl_domains-a475d05b4010bc16.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libndarray-3fbc10761f50bd8a.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libitertools-e4aea557565b531c.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libmatrixmultiply-ca40ec3728f4e9d5.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librawpointer-7dadbbfac92495af.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/liblfa-670d03b4b4804f77.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libspaces-320eca7f8e0cae32.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libitertools-b4b06e031faaf6a2.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libitertools-622a270bdb9ccd96.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libeither-5c484b219f9abba2.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librand_distr-c66d8a168421c7d6.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librand-23194a10b285f06d.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librand_chacha-ba74cf216790845e.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libppv_lite86-af0aa57e4bfe14bb.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librand_core-6d6a10f57adb54c6.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libgetrandom-73e7aa3655b4dfdc.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libcfg_if-7e74d31581507915.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libndarray_linalg-9257e937663b298f.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libcauchy-5ff5b9ffdeca56a3.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/liblapacke-2dc71c5e1ba4988a.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/liblapacke_sys-7584e61018d0e7f4.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/liblapack_src-b76f7bfea030cffc.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libblas_src-59200b4e01a5ed4b.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libndarray-95a88a976385f141.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libmatrixmultiply-f0d22bd9fca3a6da.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libnum_complex-fe0f325e0a48da84.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librand-2bb3c96a5044258b.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librand_core-67811649aed310c5.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librand_core-5fd8ef7f01bd81fb.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libserde-edbb9045479ea3ba.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libnum_integer-9d5ea53ce224789b.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/librawpointer-1e4f5908676fe080.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libapprox-8e2afd6254ec436b.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libnum_traits-8133e013eaf58d7a.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libcblas_sys-e4d2e585ba572f04.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/liblibc-d85ff30af4a50914.rlib" "/Users/nbro/Desktop/rsrl/target/debug/deps/libblas_src-459851aa1265fbbc.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-04b20da5d2b4e02d.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-00f5b50d82ace1e3.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libobject-45c041bae4e30a62.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libmemchr-1bc22f5f5be77a23.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libaddr2line-caa784d0cecbd501.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libgimli-0ad46247e89234f6.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-68dabd8f37218f7c.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd_detect-4c67204728564461.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-eca93a0d11ef9a39.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-6ddabe46ef182f8b.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libunwind-7dc1037a76f2c18c.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-100dc4191a6287d7.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liblibc-c596c47fc21af016.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc-c7163fef4a8cdd33.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-3463abc69f183e66.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcore-12a50039d8929e4e.rlib" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-a4134bbc9f4f0dab.rlib" "-framework" "Security" "-liconv" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/Users/nbro/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "-o" "/Users/nbro/Desktop/rsrl/target/debug/examples/q_learning-ffe1683a1ab5ed1e" "-Wl,-dead_strip" "-nodefaultlibs"
      = note: Undefined symbols for architecture x86_64:
                "_cblas_sdot", referenced from:
                    ndarray::linalg::impl_linalg::_$LT$impl$u20$ndarray..ArrayBase$LT$S$C$ndarray..dimension..dim..Dim$LT$$u5b$usize$u3b$$u20$1$u5d$$GT$$GT$$GT$::dot_impl::hdce8f1e718fcf2c1 in liblfa-670d03b4b4804f77.rlib(lfa-670d03b4b4804f77.lfa.1a38f248-cgu.7.rcgu.o)
                "_cblas_ddot", referenced from:
                    ndarray::linalg::impl_linalg::_$LT$impl$u20$ndarray..ArrayBase$LT$S$C$ndarray..dimension..dim..Dim$LT$$u5b$usize$u3b$$u20$1$u5d$$GT$$GT$$GT$::dot_impl::hdce8f1e718fcf2c1 in liblfa-670d03b4b4804f77.rlib(lfa-670d03b4b4804f77.lfa.1a38f248-cgu.7.rcgu.o)
              ld: symbol(s) not found for architecture x86_64
              clang: error: linker command failed with exit code 1 (use -v to see invocation)

    when I attempt to run the example q_learning.rs in rsrl/Cargo.toml with the command cargo run --example q_learning.


    • Mac OS version 12.3.1 (Monterey).
    • rustc 1.60.0
    • cargo 1.60.0

    It's not the first time that this error occurs. Here are other related issues

    • https://github.com/tspooner/rsrl/issues/77
    • https://github.com/tspooner/rsrl/issues/54

    Here you say that you used to use Mac too.

    In rsrl/Cargo.toml you only specify ndarray = "0.13" under [dependencies], and you specify the following

    blas-src = { version = "0.4", default-features = false, features = ["openblas"] }
    openblas-src = { version = "0.7", default-features = false, features = ["cblas", "system"] }

    under [dev-dependencies], so it seems that you wouldn't need to add the previous 2 lines under [dependencies] too. In fact, even if you don't have BLAS or openblas, ndarray should fallback to matrixmultiply (see this), so this error should not even occur. However, in the README, which was updated here, you write (or someone else wrote in a pull request)

    so if you're building a binary, you additionally need to specify a BLAS backend compatible with ndarray

    An example is indeed a binary, so I also tried to add the following lines

    blas-src = { version = "0.4", default-features = false, features = ["openblas"] }
    openblas-src = { version = "0.7", default-features = false, features = ["cblas", "system"] }

    under [dependencies], but I get the same linking error.

    Maybe this is obvious, but don't you need to install OpenBLAS to use it? So, I guess that, if I don't have it installed in my system, I can't use it. If you're using mac, how should you install it? This information probably should be in the README anyway. You could use brew (here), but is this the right thing to do here? I tried to install OpenBLAS via homebrew with brew install openblas, then

    1. export LDFLAGS="-L/usr/local/opt/openblas/lib"
    2. export CPPFLAGS="-I/usr/local/opt/openblas/include"
    3. export PKG_CONFIG_PATH="/usr/local/opt/openblas/lib/pkgconfig"

    As suggested at the end of the brew installation, then cargo clean and cargo run --example q_learning, but I still get the same error.

    Can't we simply use rsrl without OpenBLAS? I just wanted to try the q-learning example and I don't care about performance now (by the way, why do you use openblas? is it because of performance?).

    opened by nbro 1
  • Roadmap needed

    Roadmap needed

    This project seems to be a good starting point for a good RL library.

    However, it doesn't seem to have a roadmap and it wouldn't be easy for other people to understand what features should or not be implemented, and why.

    If I wanted to contribute to this project, I wouldn't know how to start. Yes, sure, I can look at the code and maybe try to implement some feature, but who knows if you would accept this pull request. It would quite be a waste of time. This project seems to have started as a personal project, but it could become a good library for RL and compete with other RL libraries (like stable-baselines).

    Here are some of the questions that the ROADMAP could/should answer.

    • What is the ultimate goal of this library?
    • What features do we really want to support?
    • How do we decide whether a feature is worth implementing or not? Should we use a voting mechanism?
    • What features have already been implemented?
    • What are the short and long-term goals?
    • How do we organize the efforts?
    • How about the DevOps part?
      • Should all features be tested before being released?
      • How about the versions of the crate? How do you handle them? What about stability?
      • Should we use branches when introducing a new feature?
    opened by nbro 0
  • Extensibility to multi-agent settings

    Extensibility to multi-agent settings

    Hello, I'm very new to this repo. Is this easily extensible to multi-agent settings? A bit about the problem I want to codify:

    • N agents uniformly distributed on the real line, with N odd.
    • Reward for agent $i$ at timestep $t$ is the magnitude of the intervals that are closest to $i$ and not $j$ forall $j\neq i$.
    • If two agents share the same interval, they each either half the reward or I'll flip a coin with probability 1/2 (or 1/k for k sharing) to select the agent to reward.

    If you're familiar, it's a hotelling model with no Nash equilibrium. Gonna dive into the repo but maybe you can point me to a good place to start! Thanks :)

    opened by drewstone 2
  • Not able to run the openai code

    Not able to run the openai code

    When trying to run a code with the openai platform getting the below error

       Compiling rsrl v0.6.0 (/home/nineleaps/thinking/programming-languages/rust-lang/rsrl)
    error[E0432]: unresolved imports `crate::geometry::dimensions`, `crate::geometry::RegularSpace`
     --> src/domains/openai/mod.rs:4:5
    4 |     dimensions::{Continuous, Discrete},
      |     ^^^^^^^^^^ could not find `dimensions` in `geometry`
    5 |     RegularSpace,
      |     ^^^^^^^^^^^^ no `RegularSpace` in the root
    error: aborting due to previous error
    For more information about this error, try `rustc --explain E0432`.
    error: Could not compile `rsrl`.
    To learn more, run the command again with --verbose.

    Seems like there has been a lot of code changes since the openai features had been integrated.

    bug framework priority 
    opened by infinite-Joy 1
  • 0.3.0(Feb 12, 2018)

    This includes major changes to the agent interface. The ControlAgent and PredictionAgent traits have been unified into a single trait called Agent with an associated type which distinguishes between control and predictions tasks. The methods associated with choosing actions etc are then defined in Controller for example.

    This makes it much easier to build a library of agents that have some, but not all, guarantees about behaviour. This will also make it easier to move into continuous actions space support and better combinability of the various agents (say, when using a predictor inside an actor critic method).

    In addition, this push comes with some new agent implementations based on "true online" gradient methods.

    Source code(tar.gz)
    Source code(zip)
  • 0.2.3(Feb 6, 2018)

    In this release we include some internal naming changes, additional features for the geometry module (in particular geometry::spaces) and the addition of a new domain that interacts with the OpenAI gym via a python interpreter instance. At the moment there are some unresolved changes needed to better handle infinite dimensions (which are common in the OpenAI gym - see #24), but one can run experiments and upload results to the OpenAI leaderboard.

    Source code(tar.gz)
    Source code(zip)
  • 0.2.2(Jan 18, 2018)

    This version includes an implementation of the Polynomial basis for linear function approximation, an associated usage example and some refactors to the codebase, including:

    • Dedicated type aliases for Array1 and Array2: Vector and Matrix, respectively; these have a default type argument of f64.
    • Cleanup of the cartesian_product utility function.
    Source code(tar.gz)
    Source code(zip)
  • 0.2.1(Jan 15, 2018)

    Along with some minor additions and changes, this PR includes a fix for the eligibility trace algorithms QLambda and SARSALambda which were observed to diverge during testing. As explained in 5919e802f9b193446d4fcb48ca269dde646fdae9, inconsistencies between the application of normalisation in the methods for updating/evaluating the Linear function approximator lead to instability.

    These have been corrected and an example was added to show convergence of the SARSALambda algorithm on the MountainCar domain.

    Source code(tar.gz)
    Source code(zip)
  • 0.2.0(Jan 8, 2018)

    This breaking update changes the method used to optimise for dense/sparse basis projectors. Specifically, we replace SparseLinear and DenseLinear with a single unified struct Linear which handles both dense and sparse feature vectors.

    The new Projection enum is returned from a Projector (originally called Projection itself) which requires explicitly match statements to handle either a dense or sparse vector. A Projector must now also reveal an expand_projection method which converts a Projection into a dense feature vector (of type Array1<f64>).

    All code has been refactored and optimised with respect to these changes and further tests added.

    Source code(tar.gz)
    Source code(zip)
  • 0.1.1(Jan 1, 2018)

    This patch update brings improved normalisation methods and corrections for the existing projection classes, leading to improved stability of function approximation.

    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Dec 24, 2017)

Thomas Spooner
Research Scientist - reinforcement learning, optimisation and explainability. The best software is open source. /// Rustacean
Thomas Spooner
Reinforcement learning library written in Rust

REnforce Reinforcement library written in Rust This library is still in early stages, and the API has not yet been finalized. The documentation can be

Niven Achenjang 20 Jun 14, 2022
Reinforcement learning with Rust

ReLearn: A Reinforcement Learning Library A reinforcement learning library and experiment runner. Uses pytorch as the neural network backend via the t

Eric Langlois 10 Jun 14, 2022
Border is a reinforcement learning library in Rust

Border Border is a reinforcement learning library in Rust. For reusability of both RL environments and agents, this library provides a reference imple

Taku Yoshioka 1 Dec 15, 2022
NEATeRS is a library for training a genetic neural net through reinforcement learning.

NEATeRS NEATeRS is a library for training a genetic neural net through reinforcement learning. It uses the NEAT algorithm developed by Ken Stanley whi

TecTrixer 3 Nov 28, 2022
Tangram - makes it easy for programmers to train, deploy, and monitor machine learning models.

Tangram is the all-in-one machine learning toolkit for programmers. Train a model from a CSV file on the command line. Make predictions from Elixir, G

Tangram 1.3k May 3, 2022
Machine learning framework for building object trackers and similarity search engines

Similari Similari is a framework that helps build sophisticated tracking systems. The most frequently met operations that can be efficiently implement

In-Sight 71 Dec 28, 2022
A Rust library for interacting with OpenAI's ChatGPT API, providing an easy-to-use interface and strongly typed structures.

ChatGPT Rust Library A Rust library for interacting with OpenAI's ChatGPT API. This library simplifies the process of making requests to the ChatGPT A

Arend-Jan 6 Mar 23, 2023
A Rust machine learning framework.

Linfa linfa (Italian) / sap (English): The vital circulating fluid of a plant. linfa aims to provide a comprehensive toolkit to build Machine Learning

Rust-ML 2.2k Jan 2, 2023
A Machine Learning Framework for High Performance written in Rust

polarlight polarlight is a machine learning framework for high performance written in Rust. Key Features TBA Quick Start TBA How To Contribute Contrib

Chris Ohk 25 Aug 23, 2022
Xaynet represents an agnostic Federated Machine Learning framework to build privacy-preserving AI applications.

xaynet Xaynet: Train on the Edge with Federated Learning Want a framework that supports federated learning on the edge, in desktop browsers, integrate

XayNet 196 Dec 22, 2022
Tangram is an automated machine learning framework designed for programmers.

Tangram Tangram is an automated machine learning framework designed for programmers. Run tangram train to train a model from a CSV file on the command

Tangram 1.4k Dec 30, 2022
High performance distributed framework for training deep learning recommendation models based on PyTorch.

PERSIA (Parallel rEcommendation tRaining System with hybrId Acceleration) is developed by AI platform@Kuaishou Technology, collaborating with ETH. It

null 340 Dec 30, 2022
A Framework for Production-Ready Continuous Machine Learning

CML "Domain generalization is dead, Continuous Machine Learning lives forever." —— an iKun CML is a framework for production-ready continuous machine

Yu Sun 3 Aug 1, 2023
A high level, easy to use gpgpu crate based on wgpu

A high level, easy to use gpgpu crate based on wgpu. It is made for very large computations on powerful gpus

null 18 Nov 26, 2022
Fwumious Wabbit, fast on-line machine learning toolkit written in Rust

Fwumious Wabbit is a very fast machine learning tool built with Rust inspired by and partially compatible with Vowpal Wabbit (much love! read more abo

Outbrain 115 Dec 9, 2022
convolutions-rs is a crate that provides a fast, well-tested convolutions library for machine learning

convolutions-rs convolutions-rs is a crate that provides a fast, well-tested convolutions library for machine learning written entirely in Rust with m

null 10 Jun 28, 2022
[WIP] An experimental Java-like language and it's virtual machine, for learning Java and JVM.

Sky VM An experimental Java-like language and it's virtual machine, for learning Java and JVM. Dependencies Rust (rust-lang/rust) 2021 Edition, dual-l

Kk Shinkai 2 Jan 3, 2022
Label Propagation Algorithm by Rust. Label propagation (LP) is graph-based semi-supervised learning (SSL). LGC and CAMLP have been implemented.

label-propagation-rs Label Propagation Algorithm by Rust. Label propagation (LP) is graph-based semi-supervised learning (SSL). A simple LGC and a mor

vaaaaanquish 4 Sep 15, 2021
A safe Rust wrapper around a subset of cuFFT.

cufft_rust This is a safe Rust wrapper around CUDA FFT (cuFFT). It only supports a subset of the API which I need for private projects. For now this o

Pim van den Berg 1 Jan 11, 2022