Ergo is a low-code IFTTT/Zapier style application, built with Rust and Svelte

Related tags

Task scheduling ergo
Overview

Ergo is a low-code IFTTT/Zapier style application, built with Rust and Svelte. Tasks are customizable with Javascript and can contain state machines for more advanced task handling.

This project is in very early stages right now with no current plans for proper publishing, but that will probably come some day.

Considered use cases

  • Scrape web sources and send emails based on some content in them
  • Receive a URL, run youtube-dl and save the video locally
  • Periodically download Roam database
  • Some sort of Twitter list archiving?
  • Scrape websites and populate a local database
  • Fetch filled brokerage orders

Anticipated Features

  • Inputs
    • from POST to an endpoint
    • some events are sent based on some periodic check that triggers when it sees a condition
    • trigger events unconditionally on a schedule
  • Actions
    • Spawn docker containers (and/or Nomad jobs?)
    • Query HTTP endpoints
    • Run a script
    • Run some other command
    • Link actions to accounts when required
    • Embedded scripting for actions
    • Return data for use by the next task in the chain
  • Data Schemas
    • Each action can specify the types of data that it accepts
    • Duck typing for events
  • State machines will take data from an event, modify it somehow, and pass it on
    • Embed JavaScript to write state machine logic
    • Persistent context for state machines
    • Optional schema input
    • Optional type checking on state machine return value
  • Tasks
    • Trigger a task based on events
    • Run actions
    • Run state machines in response to events
    • Sequences - Tasks can be chained together and optionally pass information between them (file locations, etc.)
    • Tasks can clone themselves, and further inputs for that clone are routed to them.
  • Templates for events, actions, and transformers
  • Extensive logging of events, actions, etc.

Roadmap

0.1

  • Events can be triggered by REST endpoint
  • Tasks consists of one event that triggers one or more actions
  • Actions can run local scripts
  • Log everything that happens
  • Events go into a queue for processing
  • Actions are triggered from a queue

0.2

  • Simple Web UI
  • Ability to run Nomad jobs as actions
Comments
  • Simple script editor

    Simple script editor

    Script editor useful for actions, tasks, etc.

    • [X] Syntax highlighting, other standard editor features
    • [X] Autovalidate actions/inputs/payload/schemas where applicable
    • [ ] Simulate scripts client-side
    opened by dimfeld 2
  • Comprehensive Tests

    Comprehensive Tests

    In the experimentation phase of the project I forewent writing extensive tests. Now that the core design has crystalized, it's time to fix that.

    • [X] Write test stubs
    • [X] Tests can create and populate a new PostgreSQL database for each test
    • [X] Tests can create Redis queues with custom (random-ish) names
    • [X] Tests can run their own instances of the server
    • [X] Write the tests!
    opened by dimfeld 1
  • Add ability for task triggers to apply immediately and return result

    Add ability for task triggers to apply immediately and return result

    Currently task triggers go into a queue. For POST endpoint-triggered inputs, we should be able to run the trigger server in a mode where it bypasses the queue and executes it immediately. This allows the task to generate a payload that can be sent directly in the HTTP response, which may be useful for things like Slack/Discord outgoing webhook implementations, among others.

    opened by dimfeld 1
  • Clean up ergonomics of getting a connection from the pool

    Clean up ergonomics of getting a connection from the pool

    Right now you have to do &mut state.pg.acquire().await.unwrap().conn. By implementing the Executor trait from sqlx this can just be &state.pg.

    opened by dimfeld 1
  • Move back to mainline sqlx

    Move back to mainline sqlx

    Currently using a fork to resolve conflicting indexmap version requirements. We can move back to the mainline once https://github.com/denoland/deno/blob/004d07dccd69c9fae8370665632d90401f770938/core/Cargo.toml#L18 is changed.

    opened by dimfeld 0
  • Periodic Task Triggers

    Periodic Task Triggers

    • [X] Add information about pending scheduled tasks to the input logs
    • [X] Cron format
    • [X] API support for adding/updating periodic task triggers
    • [X] Enqueue periodic triggers when they are added or updated.
    • [X] Reschedule a periodic trigger when it is edited.
    • [X] Re-enqueue periodic triggers after processing one.
    • [X] Make sure periodic triggers get rescheduled when errors occur and the maximum number of retries is hit.
    • [X] UI support for adding schedules to triggers
    opened by dimfeld 0
  • Simple user management

    Simple user management

    First version won't have a UI but will work for basic authentication.

    • User table
    • Roles and organizations tables, for good measure. These won't do much yet.
    • Data structure made with OAuth in mind
    • Users can have API keys
    • Event trigger endpoint checks API key
    opened by dimfeld 0
  • Better SIGINT handler

    Better SIGINT handler

    Currently the long-running tasks for renewing Vault auth use their own sigint listeners which are only active during the wait. Instead, there should be a single SIGINT listener that closes a broadcast channel, and all other things currently using sigint listeners should listen to that instead.

    opened by dimfeld 0
  • State Machine Dashboard

    State Machine Dashboard

    Feature Ideas

    • Show all the machines in a grid with some visualization
    • State machines can be configured with buttons that send them events when clicked.
    opened by dimfeld 0
  • Figure out how to get results back from external actions

    Figure out how to get results back from external actions

    Some Options:

    • Provide an HTTP endpoint to hit with the output payload.
      • This has the downside that you have to provide a URL that is actually resolvable by the task.
      • Providing the base hostname or IP in the Ergo config would be a good way to do this.
    • Provide a file path to write with the output payload. For docker containers this would be a volume mounted into the container.
      • For task orchestrators, can we do something similar?
    opened by dimfeld 0
Owner
Daniel Imfeld
CTO at Carevoyance, where I develop new ways to analyze and visualize relationships in medical data.
Daniel Imfeld
delicate A lightweight and distributed task scheduling platform written in rust

A lightweight and distributed task scheduling platform written in rust.

BinCheng 529 Jan 9, 2023
Time-manager of delayed tasks. Like crontab, but synchronous asynchronous tasks are possible, and dynamic add/cancel/remove is supported.

delay-timer Time-manager of delayed tasks. Like crontab, but synchronous asynchronous tasks are possible, and dynamic add/cancel/remove is supported.

BinCheng 255 Dec 30, 2022
Fang - Background job processing library for Rust.

Fang Background job processing library for Rust. Currently, it uses Postgres to store state. But in the future, more backends will be supported.

Ayrat Badykov 421 Dec 28, 2022
Rust single-process scheduling. Ported from schedule for Python

Rust single-process scheduling. Ported from schedule for Python, in turn inspired by clockwork (Ruby), and "Rethinking Cron" by Adam Wiggins.

Ben Lovy 13 May 30, 2022
Rust library to ease the task of creating daemons

Rust library to ease the task of creating daemons

Matheus Xavier 38 Nov 25, 2022
Simple async library for triggering IFTTT events using webhooks.

IFTTT Webhook A simple Rust async library for triggering IFTTT events using webhooks. Installation Installation can be performed using cargo add: carg

Leo Dutra 1 Mar 11, 2021
🏪 Modern emoji picker popup for desktop, based on Emoji Mart, built with Tauri and Svelte

Emoji Mart desktop popup Modern emoji picker popup app for desktop, based on the amazing Emoji Mart web component. ?? Built as a popup: quick invocati

Vincent Emonet 10 Jul 3, 2023
⚙️ Workshop Publishing Utility for Garry's Mod, written in Rust & Svelte and powered by Tauri

⚙️ gmpublisher Currently in Beta development. A powerful and feature-packed Workshop publisher for Garry's Mod is finally here! Click for downloads Ar

William 484 Jan 7, 2023
Log your spending in seconds with short text snippets. Powered by Rust, Cloudflare Workers and Svelte.

FastSpend Log your daily spending lightning fast with short text snippets! FastSpend is a tool to log your spending in seconds, powered by a lightning

Phoomparin Mano 24 Sep 13, 2022
A WIP svelte parser written in rust. Designed with error recovery and reporting in mind

Svelte(rs) A WIP parser for svelte files that is designed with error recovery and reporting in mind. This is mostly a toy project for now, with some v

James Birtles 3 Apr 19, 2023
Vue, React, Solid, Angular, Svelte, and Liquid From JS Objects.

Vue, React, Solid, Angular, Svelte, and Liquid From JS Objects.

JSX One 11 Sep 21, 2022
Vue, React, Solid, Angular, Svelte, and Liquid From JS Objects.

Vue, React, Solid, Angular, Svelte, and Liquid From JS Objects.

JSX One 18 Jul 17, 2021
UpVent Website (Powered by Rust + Svelte & Fernet Branca).

UpVent Website Source Build Status Description This is the source code for the UpVent's website under upvent.codes. Development occurs on the master b

UpVent 3 Dec 2, 2022
Configurable HTML/Vue/Svelte/Jinja/Twig formatter, with dprint integration.

markup_fmt markup_fmt is a configurable HTML/Vue/Svelte/Jinja/Twig formatter. Notes for Vue and Svelte Users This formatter provides some options such

Pig Fang 17 Nov 15, 2023
Sleek is a CLI tool for formatting SQL. It helps you maintain a consistent style across your SQL code, enhancing readability and productivity.

Sleek: SQL Formatter ✨ Sleek is a CLI tool for formatting SQL. It helps you maintain a consistent style across your SQL code, enhancing readability an

Nick Rempel 40 Apr 20, 2023
A low-level assembly language for the Ethereum Virtual Machine built in blazing-fast pure rust.

huff-rs • huff-rs is a Huff compiler built in rust. What is a Huff? Huff is a low-level programming language designed for developing highly optimized

Huff 276 Dec 31, 2022
Generates Solidity code to verify EIP-712 style signatures

eip712 Generates Solidity code to verify EIP-712 style signatures. Usage First, create an abstract contract implementing the functionality you want: /

Sam Wilson 11 Dec 22, 2022
An implementation of Code Generation and Factoring for Fast Evaluation of Low-order Spherical Harmonic Products and Squares

sh_product An implementation of Code Generation and Factoring for Fast Evaluation of Low-order Spherical Harmonic Products and Squares (paper by John

Simon Brown 7 Dec 2, 2022
ChatGPT-Code-Review is a Rust application that uses the OpenAI GPT-3.5 language model to review code

ChatGPT-Code-Review is a Rust application that uses the OpenAI GPT-3.5 language model to review code. It accepts a local path to a folder containing code, and generates a review for each file in the folder and its subdirectories.

Greg P. 15 Apr 22, 2023
Verified Rust for low-level systems code

See Goals for a brief description of the project's goals. Building the project The main project source is in source. tools contains scripts for settin

Secure Foundations Lab 95 Dec 27, 2022