Framework for large distributed pipelines

Related tags

Command-line rust distributed-computing pipelines python-api workflows
Overview

Rain

PyPI Crates.io Build Status Gitter

Rain is an open-source distributed computational framework for processing of large-scale task-based pipelines.

Rain aims to lower the entry barrier to the world of distributed computing. Our intention is to provide a light yet robust distributed framework that features an intuitive Python API, straightforward installation and deployment with insightful monitoring on top.

Despite that this is an early release of Rain, it is a fully functional project that can be used out-of-the box. Being aware that there is still a lot that can be improved and added, we are looking for external users and collaborators to help to move this work forward. Talk to us online at Gitter or via email and let us know what your projects and use-cases need, submit bugs or feature requests at GitHub or even contribute with pull requests.

Features

  • Dataflow programming. Computation in Rain is defined as a flow graph of tasks. Tasks may be built-in functions, Python/C++/Rust code, or an external applications, short and light or long-running and heavy. The system is designed to integrate any code into a pipeline, respecting its resource requirements, and to handle very large task graphs (hundreds thousands tasks).

  • Easy to use. Rain was designed to be easy to deployed anywhere, ranging from a single node deployments to large-scale distributed systems and clouds ranging thousands of cores.

  • Rust core, Python/C++/Rust API. Rain is written in Rust for safety and efficiency and has a high-level Python API to Rain core infrastructure, and even supports Python tasks out-of-the-box. Rain also provides libraries for writing own tasks in C++ and Rust.

  • Monitoring. Rain is designed to support both online and postmortem monitoring.

    Dashboard screencast

Documentation

Overview • Quickstart • User guide • Python API • Examples

Quick start

  • Download binary
$ wget https://github.com/substantic/rain/releases/download/v0.4.0/rain-v0.4.0-linux-x64.tar.xz
$ tar xvf rain-v0.4.0-linux-x64.tar.xz
  • Install Python API
$ pip3 install rain-python
  • Start server & a single local governor
$ ./rain-v0.4.0-linux-x64/rain start --simple
  • Rain "Hello world" in Python
from rain.client import Client, tasks, blob

client = Client("localhost", 7210)

with client.new_session() as session:
    task = tasks.Concat((blob("Hello "), blob("world!")))
    task.output.keep()
    session.submit()
    result = task.output.fetch().get_bytes()
    print(result)

Installation via cargo

If you have installed Rust, you can install and start Rain as follows:

$ cargo install rain_server

$ pip3 install rain-python

$ rain start --simple

Read the docs for more examples.

Comments
  • Roadmap after v0.2

    Roadmap after v0.2

    We have received many feedbacks from our reddit post (https://www.reddit.com/r/rust/comments/89yppv/rain_rust_based_computational_framework/). I think that now is time to recap our plans and maybe reconsider priorities to reflect real needs of users. This is meant as a kind of brainstorming white board; each individal task should have own issue at the end. Also, I would like to focus on a relatively short term road plan (let us say things that could be finished within 3 months), maybe we can create a similar post for our long term plans.

    EDIT (gavento): Moved @spirali's TODO to a post below.

    enhancement meta 
    opened by spirali 9
  • Cannot create log file error

    Cannot create log file error 

    $ ./rain start --worker-host-file=hosts.txt
🌧  INFO 2018-04-11T08:15:45Z Starting Rain 0.2.0
🌧  INFO 2018-04-11T08:15:45Z Log directory: /tmp/rain-logs/worker-p1-16085
🌧  INFO 2018-04-11T08:15:45Z Starting local server (0.0.0.0:7210)
🌧  INFO 2018-04-11T08:15:45Z Dashboard: http://p1:8080/
🌧  INFO 2018-04-11T08:15:45Z Server pid = 16086
🌧  INFO 2018-04-11T08:15:45Z Process 'server' is ready
🌧  INFO 2018-04-11T08:15:45Z Starting 1 remote worker(s)
🌧  INFO 2018-04-11T08:15:45Z Connecting to jupiter (remote log dir: "/tmp/rain-logs/worker-p1-16085")
🌧 ERROR 2018-04-11T08:15:45Z Remote process at jupiter, the following error occurs: Cannot create log file

🌧  INFO 2018-04-11T08:15:45Z Error occurs; clean up started processes ...

    I believe the log directory should be created first by calling mkdir -p {log_dir} before trying to create the log file at https://github.com/substantic/rain/blob/f3ec16c96409e1e6ffc9fa0d4688fb110c9d9970/src/start/ssh.rs#L98

    touch does not create parent directories on my Debian server.

    bug 
    opened by rekka 6
  • Python client segfaults with server running under WSL

    Python client segfaults with server running under WSL

    I tried executing the example from the README using Windows Subsystem for Linux. The server starts up with no apparent errors but executing the Python client script results in the client crashing with a SIGSEGV address boundary error. I tried running it with strace but there nothing obvious in the output. I can include the strace output if you want, but I'm not sure what other information I can gather to help diagnose.

    I'm using Windows 10 Enterprise 10.0.16299 with Ubuntu 16.04 in WSL. Something to note is that I had to install the cython system package before running pip3 install rain-python. I used virtualenv to create a virtualized Python install into which I installed rain-python.

    Testing the same, but without the virtualized Python, in a Ubuntu 16.04 Docker container works so it's something to do with WSL.

    Is it possible to get the server working on Windows directly? I tried compiling it but it makes use of crates that provide Unix-only functions like gethostname. Is there no cross-platform equivalent in Rust std or another crate?

    bug 
    opened by doxxx 5
  • Rust subworker

    Rust subworker

    Locally tested rust subworker with many tests, some finished APIs, docs.

    Must be tested with: cargo test -- --test-threads=1 (parallel tests clash their working dir settings)

    Missing/WIP pieces include:

    • [x] update on mutability/shared access to outputs and context from the task function
    • [x] macros to add task functions with positional attributes
    • attributes and content types with file/directory checking
    • [x] (mostyl) more helper methods to Context and possibly DataInstance/Output.
    • [x] build+testing in CI
    enhancement 
    opened by gavento 5
  • Session events

    Session events

    This PR adds session close event, enables querying for multiple event types and adds event tests.

    It takes a while before the events are written to the DB, so the test sleeps for 1 sec. But since testing this interface is non-critical, I think that it can be marked as a slow test and it's not a problem that it is a bit flaky.

    opened by Kobzol 5
  • Change task types to class hierarchy

    Change task types to class hierarchy

    In the Python API, all the tasks are currently an instance of the Task class with different types indicated by task_type. It would be more pythonic to implement them as a hierarchy of classes, one for each task_type (or even one for every Program or python Remote func. - TBD). This would have the advantages to allow using isinstance and better introspection (__repr__), access to parameters specific to the task type and allow subclassing.

    Independently of this change, we can keep the task-creation operations as functions (and lower-cased), e.g. keep tasks.concat in additon to having tasks.ConcatTask.

    This should be mostly non-disruptive change and needs a more detailed spec of Task interface.

    enhancement python API 
    opened by gavento 4
  • Add Arrow serialization content_type

    Add Arrow serialization content_type

    This commits add support for the Arrow serialization format.

    The added test is very bare-bones, I didn't find any content_type specific tests, so I just slapped it onto an existing end-to-end serialization test.

    The pyarrow package should probably be optional, but it's kind of impractical to specify optional dependencies for pip (AFAIK).

    opened by Kobzol 3
  • Add stop command to server

    Add stop command to server

    The start command conveniently launches a cluster (server + workers), but it makes it a bit cumbersome to stop the cluster.

    For a short term, hacky solution, a simple killall on Linux could solve it, or the start command could run the server in-process, so that it would block while the server lives. But in general I think that the server should have a more general and graceful option to stop itself, defined in its API.

    Proposal: The stop event would stop all the connected workers and then shutdown the server. The command could look like this: rain stop server:1234 It would probably make sense to stop individual workers too, possibly with the same command and a --worker flag.

    The ability for clients to shutdown the server is a non-issue from a security standpoint, since there is already no client authentication and the cluster supposes that it's run in a trusted environment.

    There could also be an option for a "soft stop", that would cause the server to stop scheduling tasks, but it would first wait for the current in-progress tasks to finish (and maybe checkpoint them to disk) before stopping the cluster.

    enhancement 
    opened by Kobzol 3
  • Attributes refactor and specification

    Attributes refactor and specification

    The attributes are currently not well-specified and there seem to be ways to implement them better before we release the (much more stable) v0.3.

    Goals:

    • Rust structures (with serde) with typed values for known attributes (and a map for user/unknown)
    • Split Task and DataObject attributes (required by the above, will allow checking in Python and other clients)
    • Specify a fixed encoding for rich and user-specified values (current proposal: JSON with base64 binary data) (note that non-trivial binary data in attributes are generally discouraged)
    • Create a rich Attributes class in Python (underway in #60)
    • Fix and document the new structure semantics
    enhancement python API server governor client 
    opened by gavento 2
  • Create a C++ subworker library

    Create a C++ subworker library

    Implement a library to make C++ subworker implementation easier. In spirit, this should consist of I/O and Context types, way to declare and register task functions and a simple mainloop function, similarly to #34. Would you sketch the API, @spirali?

    enhancement governor 
    opened by gavento 2
  • dependency problem of crate log 0.3.x

    dependency problem of crate log 0.3.x

    When rain is built or installed with log 0.3.x, it fails.

    error[E0432]: unresolved import `log::Level`
   --> src/bin.rs:349:21
    |
349 |                 use log::Level;
    |                     ^^^^^^^^^^ no `Level` in the root

error: aborting due to previous error

    It seems to depend on a change rust-lang-nursery/log#162. So If we use log 0.4.x, build or install success.

    So, we should change "*" in Cargo.toml to >=0.4. I'll make a PR.

    opened by aimof 2
  • Bump qs and express in /dashboard

    Bump qs and express in /dashboard

    Bumps qs, qs and express. These dependencies needed to be updated together. Updates qs from 6.4.0 to 6.5.3

    Changelog

    Sourced from qs's changelog.

    6.5.3

    • [Fix] parse: ignore __proto__ keys (#428)
    • [Fix] utils.merge: avoid a crash with a null target and a truthy non-array source
    • [Fix] correctly parse nested arrays
    • [Fix] stringify: fix a crash with strictNullHandling and a custom filter/serializeDate (#279)
    • [Fix] utils: merge: fix crash when source is a truthy primitive & no options are provided
    • [Fix] when parseArrays is false, properly handle keys ending in []
    • [Fix] fix for an impossible situation: when the formatter is called with a non-string value
    • [Fix] utils.merge: avoid a crash with a null target and an array source
    • [Refactor] utils: reduce observable [[Get]]s
    • [Refactor] use cached Array.isArray
    • [Refactor] stringify: Avoid arr = arr.concat(...), push to the existing instance (#269)
    • [Refactor] parse: only need to reassign the var once
    • [Robustness] stringify: avoid relying on a global undefined (#427)
    • [readme] remove travis badge; add github actions/codecov badges; update URLs
    • [Docs] Clean up license text so it’s properly detected as BSD-3-Clause
    • [Docs] Clarify the need for "arrayLimit" option
    • [meta] fix README.md (#399)
    • [meta] add FUNDING.yml
    • [actions] backport actions from main
    • [Tests] always use String(x) over x.toString()
    • [Tests] remove nonexistent tape option
    • [Dev Deps] backport from main

    6.5.2

    • [Fix] use safer-buffer instead of Buffer constructor
    • [Refactor] utils: module.exports one thing, instead of mutating exports (#230)
    • [Dev Deps] update browserify, eslint, iconv-lite, safer-buffer, tape, browserify

    6.5.1

    • [Fix] Fix parsing & compacting very deep objects (#224)
    • [Refactor] name utils functions
    • [Dev Deps] update eslint, @ljharb/eslint-config, tape
    • [Tests] up to node v8.4; use nvm install-latest-npm so newer npm doesn’t break older node
    • [Tests] Use precise dist for Node.js 0.6 runtime (#225)
    • [Tests] make 0.6 required, now that it’s passing
    • [Tests] on node v8.2; fix npm on node 0.6

    6.5.0

    • [New] add utils.assign
    • [New] pass default encoder/decoder to custom encoder/decoder functions (#206)
    • [New] parse/stringify: add ignoreQueryPrefix/addQueryPrefix options, respectively (#213)
    • [Fix] Handle stringifying empty objects with addQueryPrefix (#217)
    • [Fix] do not mutate options argument (#207)
    • [Refactor] parse: cache index to reuse in else statement (#182)
    • [Docs] add various badges to readme (#208)
    • [Dev Deps] update eslint, browserify, iconv-lite, tape
    • [Tests] up to node v8.1, v7.10, v6.11; npm v4.6 breaks on node < v1; npm v5+ breaks on node < v4
    • [Tests] add editorconfig-tools

    ... (truncated)

    Commits
    • 298bfa5 v6.5.3
    • ed0f5dc [Fix] parse: ignore __proto__ keys (#428)
    • 691e739 [Robustness] stringify: avoid relying on a global undefined (#427)
    • 1072d57 [readme] remove travis badge; add github actions/codecov badges; update URLs
    • 12ac1c4 [meta] fix README.md (#399)
    • 0338716 [actions] backport actions from main
    • 5639c20 Clean up license text so it’s properly detected as BSD-3-Clause
    • 51b8a0b add FUNDING.yml
    • 45f6759 [Fix] fix for an impossible situation: when the formatter is called with a no...
    • f814a7f [Dev Deps] backport from main
    • Additional commits viewable in compare view

    Updates qs from 6.5.2 to 6.5.3

    Changelog

    Sourced from qs's changelog.

    6.5.3

    • [Fix] parse: ignore __proto__ keys (#428)
    • [Fix] utils.merge: avoid a crash with a null target and a truthy non-array source
    • [Fix] correctly parse nested arrays
    • [Fix] stringify: fix a crash with strictNullHandling and a custom filter/serializeDate (#279)
    • [Fix] utils: merge: fix crash when source is a truthy primitive & no options are provided
    • [Fix] when parseArrays is false, properly handle keys ending in []
    • [Fix] fix for an impossible situation: when the formatter is called with a non-string value
    • [Fix] utils.merge: avoid a crash with a null target and an array source
    • [Refactor] utils: reduce observable [[Get]]s
    • [Refactor] use cached Array.isArray
    • [Refactor] stringify: Avoid arr = arr.concat(...), push to the existing instance (#269)
    • [Refactor] parse: only need to reassign the var once
    • [Robustness] stringify: avoid relying on a global undefined (#427)
    • [readme] remove travis badge; add github actions/codecov badges; update URLs
    • [Docs] Clean up license text so it’s properly detected as BSD-3-Clause
    • [Docs] Clarify the need for "arrayLimit" option
    • [meta] fix README.md (#399)
    • [meta] add FUNDING.yml
    • [actions] backport actions from main
    • [Tests] always use String(x) over x.toString()
    • [Tests] remove nonexistent tape option
    • [Dev Deps] backport from main

    6.5.2

    • [Fix] use safer-buffer instead of Buffer constructor
    • [Refactor] utils: module.exports one thing, instead of mutating exports (#230)
    • [Dev Deps] update browserify, eslint, iconv-lite, safer-buffer, tape, browserify

    6.5.1

    • [Fix] Fix parsing & compacting very deep objects (#224)
    • [Refactor] name utils functions
    • [Dev Deps] update eslint, @ljharb/eslint-config, tape
    • [Tests] up to node v8.4; use nvm install-latest-npm so newer npm doesn’t break older node
    • [Tests] Use precise dist for Node.js 0.6 runtime (#225)
    • [Tests] make 0.6 required, now that it’s passing
    • [Tests] on node v8.2; fix npm on node 0.6

    6.5.0

    • [New] add utils.assign
    • [New] pass default encoder/decoder to custom encoder/decoder functions (#206)
    • [New] parse/stringify: add ignoreQueryPrefix/addQueryPrefix options, respectively (#213)
    • [Fix] Handle stringifying empty objects with addQueryPrefix (#217)
    • [Fix] do not mutate options argument (#207)
    • [Refactor] parse: cache index to reuse in else statement (#182)
    • [Docs] add various badges to readme (#208)
    • [Dev Deps] update eslint, browserify, iconv-lite, tape
    • [Tests] up to node v8.1, v7.10, v6.11; npm v4.6 breaks on node < v1; npm v5+ breaks on node < v4
    • [Tests] add editorconfig-tools

    ... (truncated)

    Commits
    • 298bfa5 v6.5.3
    • ed0f5dc [Fix] parse: ignore __proto__ keys (#428)
    • 691e739 [Robustness] stringify: avoid relying on a global undefined (#427)
    • 1072d57 [readme] remove travis badge; add github actions/codecov badges; update URLs
    • 12ac1c4 [meta] fix README.md (#399)
    • 0338716 [actions] backport actions from main
    • 5639c20 Clean up license text so it’s properly detected as BSD-3-Clause
    • 51b8a0b add FUNDING.yml
    • 45f6759 [Fix] fix for an impossible situation: when the formatter is called with a no...
    • f814a7f [Dev Deps] backport from main
    • Additional commits viewable in compare view

    Updates express from 4.16.3 to 4.18.2

    Release notes

    Sourced from express's releases.

    4.18.2

    4.18.1

    • Fix hanging on large stack of sync routes

    4.18.0

    ... (truncated)

    Changelog

    Sourced from express's changelog.

    4.18.2 / 2022-10-08

    4.18.1 / 2022-04-29

    • Fix hanging on large stack of sync routes

    4.18.0 / 2022-04-25

    ... (truncated)

    Commits

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 0
  • Bump express from 4.16.3 to 4.18.2 in /dashboard

    Bump express from 4.16.3 to 4.18.2 in /dashboard

    Bumps express from 4.16.3 to 4.18.2.

    Release notes

    Sourced from express's releases.

    4.18.2

    4.18.1

    • Fix hanging on large stack of sync routes

    4.18.0

    ... (truncated)

    Changelog

    Sourced from express's changelog.

    4.18.2 / 2022-10-08

    4.18.1 / 2022-04-29

    • Fix hanging on large stack of sync routes

    4.18.0 / 2022-04-25

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 0
  • Bump decode-uri-component from 0.2.0 to 0.2.2 in /dashboard

    Bump decode-uri-component from 0.2.0 to 0.2.2 in /dashboard

    Bumps decode-uri-component from 0.2.0 to 0.2.2.

    Release notes

    Sourced from decode-uri-component's releases.

    v0.2.2

    • Prevent overwriting previously decoded tokens 980e0bf

    https://github.com/SamVerschueren/decode-uri-component/compare/v0.2.1...v0.2.2

    v0.2.1

    • Switch to GitHub workflows 76abc93
    • Fix issue where decode throws - fixes #6 746ca5d
    • Update license (#1) 486d7e2
    • Tidelift tasks a650457
    • Meta tweaks 66e1c28

    https://github.com/SamVerschueren/decode-uri-component/compare/v0.2.0...v0.2.1

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 0
  • Bump capnp from 0.8.17 to 0.13.7

    Bump capnp from 0.8.17 to 0.13.7

    Bumps capnp from 0.8.17 to 0.13.7.

    Commits
    • 79a761d prepare for capnp-v0.13.7 release
    • feb7c98 remove list pointer munging and add tests
    • b736f2b prepare for 0.13.6 release
    • 8257850 Add blanket impl Allocator for &mut A where A: Allocator
    • fb230d4 update pubsub example to use tokio 0.3.0
    • a0bf5bb update calculator rpc example to use tokio 0.3.0
    • 1d76ce1 update hello-world rpc example to use tokio 0.3.0
    • 9c2a11b add comment for get_connection_state
    • 7fb33a3 minor simplification in twoparty::VatNetwork::connect()
    • d7b1791 prepare for capnp-v0.13.5 release
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies rust 
    opened by dependabot[bot] 0
  • Bump loader-utils and react-scripts-ts in /dashboard

    Bump loader-utils and react-scripts-ts in /dashboard

    Bumps loader-utils to 1.4.2 and updates ancestor dependency react-scripts-ts. These dependencies need to be updated together.

    Updates loader-utils from 0.2.17 to 1.4.2

    Release notes

    Sourced from loader-utils's releases.

    v1.4.2

    1.4.2 (2022-11-11)

    Bug Fixes

    v1.4.1

    1.4.1 (2022-11-07)

    Bug Fixes

    v1.4.0

    1.4.0 (2020-02-19)

    Features

    • the resourceQuery is passed to the interpolateName method (#163) (cd0e428)

    v1.3.0

    1.3.0 (2020-02-19)

    Features

    • support the [query] template for the interpolatedName method (#162) (469eeba)

    v1.2.3

    1.2.3 (2018-12-27)

    Bug Fixes

    • interpolateName: don't interpolated hashType without hash or contenthash (#140) (3528fd9)

    v1.2.2

    1.2.2 (2018-12-27)

    Bug Fixes

    ... (truncated)

    Changelog

    Sourced from loader-utils's changelog.

    1.4.2 (2022-11-11)

    Bug Fixes

    1.4.1 (2022-11-07)

    Bug Fixes

    1.4.0 (2020-02-19)

    Features

    • the resourceQuery is passed to the interpolateName method (#163) (cd0e428)

    1.3.0 (2020-02-19)

    Features

    • support the [query] template for the interpolatedName method (#162) (469eeba)

    1.2.3 (2018-12-27)

    Bug Fixes

    • interpolateName: don't interpolated hashType without hash or contenthash (#140) (3528fd9)

    1.2.2 (2018-12-27)

    Bug Fixes

    ... (truncated)

    Commits
    Maintainer changes

    This version was pushed to npm by evilebottnawi, a new releaser for loader-utils since your current version.


    Updates react-scripts-ts from 2.16.0 to 4.0.8

    Changelog

    Sourced from react-scripts-ts's changelog.

    1.1.5 (August 24, 2018)

    • react-scripts

      • Update the webpack-dev-server dependency

    • react-dev-utils

      • #4866 Fix a Windows-only vulnerability (CVE-2018-6342) in the development server (@​acdlite)
      • Update the sockjs-client dependency

    Committers: 1

    Migrating from 1.1.4 to 1.1.5

    Inside any created project that has not been ejected, run:

    npm install --save --save-exact [email protected]

    or

    yarn add --exact [email protected]

    1.1.4 (April 3, 2018)

    :bug: Bug Fix

    Committers: 1

    Migrating from 1.1.3 to 1.1.4

    Inside any created project that has not been ejected, run:

    npm install --save --save-exact [email protected]

    or

    </tr></table>

    ... (truncated)

    Commits

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 0
  • Bump css-what from 2.1.0 to 2.1.3 in /dashboard

    Bump css-what from 2.1.0 to 2.1.3 in /dashboard

    Bumps css-what from 2.1.0 to 2.1.3.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 0
Releases(v0.4.0)
Owner
Substantic
Substantic
GitHub https://substantic.github.io/rain/docs/
A fast and robust MLOps tool for managing data and pipelines

xvc A Fast and Robust MLOps Swiss-Army Knife in Rust ⌛ When to use xvc? Machine Learning Engineers: When you manage large quantities of unstructured d

Emre Sahin 6 Dec 15, 2022
Write CI/CD pipelines using TypeScript

Katoa Katoa is a community fork of Cidada, a tool created by Fig which was sunset in late 2023 following acquisition by AWS. This fork and the underly

Katoa 47 Oct 6, 2023
EmbedAnything is a powerful python library designed to streamline the creation and management of embedding pipelines

EmbedAnything is a powerful python library designed to streamline the creation and management of embedding pipelines. Built in Rust with no heavy dependencies.

Starlight 39 May 7, 2024
An ultra-fast CLI app that fixes JSON files in large codebase or folders

minosse An ultra fast CLI app that fixes json files in large codebase or folders USAGE: minosse [OPTIONS] <input-dir> FLAGS: -h, --help Prints

Antonino Bertulla 5 Oct 17, 2022
Codemod - Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention

Codemod - Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention. Codemod was developed at Facebook and released as open source.

Meta Archive 4k Dec 29, 2022
auto-rust is an experimental project that aims to automatically generate Rust code with LLM (Large Language Models) during compilation, utilizing procedural macros.

Auto Rust auto-rust is an experimental project that aims to automatically generate Rust code with LLM (Large Language Models) during compilation, util

Minsky 6 May 14, 2023
🦀Rust + Large Language Models - Make AI Services Freely and Easily. Inspired by LangChain

llmchain: Modern Data Transformations with LLM ?? + Large Language Models, inspired by LangChain. Features Models: LLMs & Chat Models & Embedding Mode

Shafish Labs 63 Jun 22, 2023
Tool to allow parsing large JSON files without laoding into memory

Tool to allow parsing large JSON files without laoding into memory. Developed in Rust with adapters in other programming langauges for easy adoption

Salaah Amin 7 Jul 11, 2023
Attempt to summarize text from `stdin`, using a large language model (locally and offline), to `stdout`

summarize-cli Attempt to summarize text from stdin, using a large language model (locally and offline), to stdout. cargo build --release target/releas

null 4 Aug 23, 2023
A rust crate for rendering large text to the terminal using font8x8 and ratatui.

tui-big-text tui-big-text is a rust crate that renders large pixel text as a ratatui widget using the glyphs from the font8x8 crate. Installation carg

Josh McKinney 7 Sep 7, 2023
Slack chat bot written in Rust that allows the user to interact with a large language model.

A Slack chat bot written in Rust that allows the user to interact with a large language model. Creating an App on Slack, first steps Go to https://api

Marco Inacio 13 Nov 2, 2023
Terminal UI to chat with large language models (LLM) using different model backends, and integrations with your favourite editors!

Oatmeal Terminal UI to chat with large language models (LLM) using different model backends, and integrations with your favourite editors! Overview In

Dustin Blackman 88 Dec 4, 2023
Cloud Native Buildpack that builds an OCI image with Ollama and a large language model.

Ollama Cloud Native Buildpack This buildpack builds an OCI image with Ollama and a large language model. Configure your model by an Ollama Modelfile o

Manuel Fuchs 3 Mar 19, 2024
NodeCraft - Crafting seamless node operations for distributed systems

NodeCraft Crafting seamless node operations for distributed systems, which provides foundational traits for node identification and address resolution

Al Liu 3 Oct 9, 2023
interactcli-rs is a command-line program framework used to solve the problem of the integration of command-line and interactive modes, including functions such as unification of command-line interactive modes and sub-command prompts. The framework integrates clap and shellwords.

interactcli-rs 简体中文 interactcli-rs is a command-line program framework used to solve the problem of the integration of command-line and interactive mo

JiaShiwen 12 Nov 5, 2022
A minimal CLI framework written in Rust

seahorse A minimal CLI framework written in Rust Features Easy to use No dependencies Typed flags(Bool, String, Int, Float) Documentation Here Usage T

Keisuke Toyota 223 Dec 30, 2022
Bashly - Bash CLI Framework and Generator

Bashly - Bash CLI Framework and Generator Create feature-rich bash scripts using simple YAML configuration

Danny Ben Shitrit 1.4k Jan 4, 2023
CLI toolkit for GTD framework.

GTDF_Crabby CLI toolkit for GTD framework. How to use crabby 0. Parameters Crabby is a CLI toolkit and gets parameters as input. All the main options

akrck02 2 Feb 13, 2022
Black-box integration tests for your REST API using the Rust and its test framework

restest Black-box integration test for REST APIs in Rust. This crate provides the [assert_api] macro that allows to declaratively test, given a certai

IOmentum 10 Nov 23, 2022