Efficient ML solutions for long-tailed demands.

Overview

MegFlow

GitHub license ubuntu macos

MegFlow 是一个面向视觉应用的流式计算框架, 目标是简单、高性能、帮助机器学习应用快速落地。

Features

  • 基于 async-std[features=tokio1] 的高效异步运行时调度器
  • 简洁的基于 toml 的建图描述格式
  • 支持静态、动态、共享子图
  • 支持 Rust/Python 多语言共存(会 Python 即可)
  • 支持资源管理(多层级跨任务共享)
  • 支持异常处理(异常任务会终止所在静态图)
  • 支持 demux/reorder/transform 等通用函数式组件
  • Python 插件内置有栈协程,不依赖 asyncio
  • 基础测试工具,支持插件沙盒,用于单测插件

HowTo

Current Support List

系统环境 支持情况
Windows 10 WSL ubuntu18.04 ✔️
x86 Ubuntu16.04 有 GPU ✔️
x86 Ubuntu18.04 无 GPU ✔️
x86 macos ✔️
ARM /
Python 版本 支持情况
3.6 ✔️
3.7 ✔️
3.8 ✔️
3.9 /

Built-in Applications

  • 猫猫围栏
  • 电梯电动车报警

Coming Soon

  • 进程级别的节点、子图支持
  • 插件自动化测试部署
  • 调试工具,建图实时预览 profile 工具
  • 性能监控,inspect 工具
  • 更多内置应用和组件

Contact Us

  • Issue: github.com/MegEngine/MegFlow/issues
  • Email: [email protected]
  • Forum: discuss.megengine.org.cn
  • QQ Group: 1029741705
  • OPENI: openi.org.cn/MegEngine

License

Acknowledgement

MegFlow 模型和推理学习了这些项目:

MegFlow Python 使用了下列项目:

MegFlow Rust 使用了以下项目:

Issues
Releases(v0.1.0-beta0)
  • v0.1.0-beta0(Sep 7, 2021)

    release beta

    | 系统 | 支持情况 | | - | - | | Windows 10 WSL ubuntu18.04 | ✔️ | | x86 Ubuntu16.04 有 GPU | ✔️ | | x86 Ubuntu18.04 无 GPU | ✔️ | | x86 macos | ✔️ |

    | Python 版本 | 支持情况 | | - | - | | 3.6 | ✔️ | | 3.7 | ✔️ | | 3.8 | ✔️ |

    初始化项目

    Rust 新增

    • toml 建图
    • 进程内子图共享、异常处理
    • 计算图性能统计接口
    • Image/Video Server
    • Python 共存

    Python 新增

    • 3 个 Examples
      • logical_test
      • cat_finder
      • MEMD
    • 1 个 github action、ubuntu 环境 Dockerfile
    • py3.6/3.7/3.8 版本 prebuilt 脚本
    Source code(tar.gz)
    Source code(zip)
Owner
旷视天元 MegEngine
旷视天元 MegEngine
Execute genetic algorithm (GA) simulations in a customizable and extensible way.

genevo genevo provides building blocks to run simulations of optimization and search problems using genetic algorithms (GA). The vision for genevo is

Innoave 64 Jul 21, 2021
Distributed compute platform implemented in Rust, and powered by Apache Arrow.

Ballista: Distributed Compute Platform Overview Ballista is a distributed compute platform primarily implemented in Rust, powered by Apache Arrow. It

Ballista 2.3k Sep 16, 2021
A Rust🦀 implementation of CRAFTML, an Efficient Clustering-based Random Forest for Extreme Multi-label Learning

craftml-rs A Rust implementation of CRAFTML, an Efficient Clustering-based Random Forest for Extreme Multi-label Learning (Siblini et al., 2018). Perf

Tom Dong 12 Apr 21, 2021
Qdrant - vector similarity search engine with extended filtering support

Vector Similarity Search Engine with extended filtering support Qdrant (read: quadrant ) is a vector similarity search engine. It provides a productio

qdrant 560 Sep 12, 2021
Narwhal and Tusk A DAG-based Mempool and Efficient BFT Consensus.

This repo contains a prototype of Narwhal and Tusk. It supplements the paper Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus.

Facebook Research 20 Aug 25, 2021
HNSW ANN from the paper "Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs"

hnsw Hierarchical Navigable Small World Graph for fast ANN search Enable the serde feature to serialize and deserialize HNSW. Tips A good default for

Rust Computer Vision 69 Sep 11, 2021
C library for finding nearest (most similar) element in a set

VP-tree nearest neighbor search A relatively simple and readable Rust implementation of Vantage Point tree search algorithm. The VP tree algorithm doe

Kornel 27 Aug 5, 2021
Simple neural network library for classification written in Rust.

Cogent A note I continue working on GPU stuff, I've made some interesting things there, but ultimately it made me realise this is far too monumental a

Jonathan Woollett-Light 28 Jul 15, 2021
A deep learning library for rust

Alumina An experimental deep learning library written in pure rust. Breakage expected on each release in the short term. See mnist.rs in examples or R

zza 86 Aug 25, 2021
A simple neural net implementation.

PROPHET - Neural Network Library Linux Windows Codecov Coveralls Docs Crates.io A simple neural net implementation written in Rust with a focus on cac

Robin Freyler 40 Aug 23, 2021
An 8080 Space Invaders emulator in Rust

Space Invade.rs An 8080 Space Invaders emulator written in Rust This is an 8080 emulator running the 1978 Space Invaders game by Taito, written in Rus

Cedric Beust 14 Sep 13, 2021
Open Machine Intelligence Framework for Hackers. (GPU/CPU)

Leaf • Introduction Leaf is a open Machine Learning Framework for hackers to build classical, deep or hybrid machine learning applications. It was ins

Autumn 5.5k Sep 10, 2021
Tensors and differentiable operations (like TensorFlow) in Rust

autograd Differentiable operations and tensors backed by ndarray. Motivation Machine learning is one of the field where Rust lagging behind other lang

Ryo ASAKURA 326 Sep 15, 2021
High-performance automatic differentiation of LLVM.

The Enzyme High-Performance Automatic Differentiator of LLVM Enzyme is a plugin that performs automatic differentiation (AD) of statically analyzable

William Moses 527 Sep 15, 2021
A fast, safe and easy to use reinforcement learning framework in Rust.

RSRL (api) Reinforcement learning should be fast, safe and easy to use. Overview rsrl provides generic constructs for reinforcement learning (RL) expe

Thomas Spooner 109 Sep 5, 2021
l2 is a fast, Pytorch-style Tensor+Autograd library written in Rust

l2 • ?? A Pytorch-style Tensor+Autograd library written in Rust Installation • Contributing • Authors • License • Acknowledgements Made by Bilal Khan

Bilal Khan 136 Aug 17, 2021
Deep recommender systems for Rust

sbr An implementation of sequence recommenders based on the wyrm autdifferentiaton library. sbr-rs sbr implements efficient recommender algorithms whi

Maciej Kula 98 Aug 27, 2021
🚀 efficient approximate nearest neighbor search algorithm collections library written in Rust 🦀 .

?? efficient approximate nearest neighbor search algorithm collections library written in Rust ?? .

Hora-Search 2.1k Sep 12, 2021
Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference

Sonos' Neural Network inference engine. This project used to be called tfdeploy, or Tensorflow-deploy-rust. What ? tract is a Neural Network inference

Sonos, Inc. 1.1k Sep 17, 2021