Elemental System Designs is an open source project to document system architecture design of popular apps and open source projects that we want to study

Overview

Elemental System Designs

All Contributors

What is this?

Elemental System Designs is an open source project to document system architecture design of popular apps and open source projects that we want to study. The project focuses on the high level abstraction architecture based on use cases of an app that we can observe while using them in real life.

We will focus on features with high-impact of the selected apps, while proposing high scalable and resilient architecture that could scale infinitely. We welcome anyone who is actually in the same field as the app of proposed architecture to give us ideas around industry specific bottlenecks and your solution to the problem.

What is System Design?

System Architecture Design, often called System Design is a conceptual representation of the components and subcomponents that reflects the behaviour of the system as a whole.

System Architects are who make decisions on components to use based on the use cases and requirements given by the business, and consider trade-offs and bottleneck while designing a system.

What I cannot create, I do not understand. — Richard Feynman

I am writing this project for me to better understand system architecture of various popular apps that I use daily. While the system design proposals are hypothetical, we bring industry best practices in the proposals, and have them available for anyone who is interested in pursuing the field of system architecture designs as references.

If you spend enough time on planning, coding is easy.

I believe a good architecture is the key to delivering a successful project. A good architecture is required even if you are building a side project or a system for a company while being paid. Once a good architecture is available in front of you well documented, you are more confident and faster as a programmer to hit the deadline of delivering an application (or even a feature within a large ecosystem).

System Designs

  1. tiktok
  2. [WIP] Dropbox

Structure of system designs

1. Introduction to the system

This section briefly explains the app. Any useful text from the app's official webiste or Wikipedia would fit for this

2. Use cases and Requirements

This section details use cases of the app that we are studying. And followed by requirements for this system design.

When picking requirements, we try to pick the most high-impact features of the app, for example in Tiktok example, we've picked viewing videos, uploading videos, and receiving personalised feeds.

3. Constraints

  • Identify traffic and data handling at scale
  • Scale of the system such as requests per second, request types, data written per second, data read per second
  • Special system requirements such as multi-threading

4. Drawing Application Architecture Diagram

What is Application Architecture Diagram?

An application architecture diagram comprises a high-level overview of the components and fundamental interactions within the system, e.g. microservices, databases, etc. The application architecture diagram primarily addresses the “What” in relation to the system.

We want to keep our system designs to show an abstraction of important components. The diagram should contain following features

  • Be simple
  • All important components of the system to facilitate the requirements
  • System boundaries
    • In an architecture diagram, it is a concept line that divides the system you want to study from 'everything else
    • It is to include elements that you are interested / specified in the use cases / requirements
  • Show system interactions
    • Use simple shapes and lines to indicate process flows and the way the different elements communicate to each others
  • Include useful annotations
    • Add helpful explanation to critical pieces of your diagram

In the diagram, we want to include following components

  • Application service layers
  • Data storage layers
  • Usually a scalable system includes load balancer, event stream, queue, databases (master/slave, replication or sharding model) and in-memory key-value stores such as Redis

For the icons used in the diagram, please check this link

5. Component Design

  • Component + specific APIs required for each of them.

    • The project focuses on the high level design, we should use component design section to enhance the detail that could be too vague in the application architecture diagram. This is the section to write anything important per component level
  • Database schema design

    • A simplified database schema design that focused on only important ones in relation to the requirements and use cases

6. Understanding Bottlenecks

  • Describe any industry specific bottleneck (e.g. security concerns in financial applications)
  • Maybe the application needs a CDN? Load balancer? Data is so huge so database should be sharded?
  • Maybe the application needs complex query to fetch data, caching the result somewhere such as in-memory cache store would be beneficial?

7. Scaling

Vertical Scaling

Provide suggestions on adding more power to each component where it's necessary and important.

Horizontal Scaling

Wherever possible, service components should be horizontally scalable.

Caching suggestions

Can the design take advantage of any following caching strategies?

  • Can any component take an advantage of application layer caching LRU / LFU?
  • Database layer caching?
  • In-memory caches such as Redis or Memcache?
  • CDN to cache static assets?

References

Contributing

Please check CONTRIBUTING.md

Contributors

Thanks goes to these wonderful people (emoji key):


Jason Shin

🚇 🤔

Tyronne Jeong

🤔

GummyBearr

🤔

Dawoud T

🤔

This project follows the all-contributors specification. Contributions of any kind welcome!

You might also like...
Featured Dioxus projects on how to build clean user interfaces in Rust
Featured Dioxus projects on how to build clean user interfaces in Rust

Example projects with Dioxus This repository holds the code for a variety of example projects built with Dioxus. Each project has information on how t

Super-simple, fully Rust powered
Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc.

memex Super simple "memory" for LLM projects, semantic search, etc. Running the service Note that if you're running on Apple silicon (M1/M2/etc.), it'

Open-source Autonomy Software in Rust-lang with gRPC for the Roomba series robot vacuum cleaners
Open-source Autonomy Software in Rust-lang with gRPC for the Roomba series robot vacuum cleaners

CleanIt Open-source Autonomy Software in Rust-lang with gRPC for the Roomba series robot vacuum cleaners Motivation Motivation is to build a complete

Maai_core is an open source AI that is written in rust

maai_core is an open source AI that is written in rust. It is more focused on being lightweight and speedy.

An open source WCH-Link library/command line tool written in Rust.

wlink - WCH-Link command line tool NOTE: This tool is still in development and not ready for production use. Known Issue: Only support binary firmware

An open-source UCIe implementation developed at UC Berkeley.

UCIe An open-source UCIe implementation developed at UC Berkeley. This project is very early-stage. If you are interested in contributing, see below.

This project contains small exercises to get you used to reading and writing Rust code
This project contains small exercises to get you used to reading and writing Rust code

rustlings 🦀 ❤️ Greetings and welcome to rustlings. This project contains small exercises to get you used to reading and writing Rust code. This inclu

Manas project aims to create a modular framework and ecosystem to create robust storage servers adhering to Solid protocol in rust.

मनस् | Manas Solid is a web native protocol to enable interoperable, read-write, collaborative, and decentralized web, truer to web's original vision.

A learning project/fun experiment in internet protocol

Piper a learning project/fun experiment in internet protocol Version 0.4.0 (SEMVER) Goals Piper is Simple. A page is a page. There are no secondary re

Comments
  • Welcome to Elemental System Designs

    Welcome to Elemental System Designs

    This is a project born out of interest and for learning! All contributions are welcome, even if it's just raising an issue to describe a bug fix, a request to design a system that you are interested in, and improvements to the existing designs.

    We can use this issue to add new contributors using all-contributors bot.

    opened by JasonShin 12
Owner
Jason Shin
Node, Python, Rust, Machine Learning, Functional Programming, React, Vue, Kubernetes and Scala.
Jason Shin
Reef: A zkSNARK system for proving that a committed document matches a regex

Reef This is an implementation of Reef, a system for generating zero-knowledge proofs that a committed document matches or does not match a regular ex

University of Pennsylvania | Distributed Systems Lab 3 Dec 15, 2023
Game development practices with Rust programming language. I want to use different crates for this.

Hazır Oyun Motorlarını Kullanarak Rust Dili Yardımıyla Oyunlar Geliştirmek Rust programlama dilinde oyun geliştirmek için popüler birkaç hazır çatıyı

Burak Selim Senyurt 16 Dec 27, 2022
A catalogue of Rust design patterns, anti-patterns and idioms

Rust Design Patterns An open source book about design patterns and idioms in the Rust programming language that you can read here. Contributing You ar

null 6.5k Jan 9, 2023
A Domain Driven Design example application in Rust.

Rust Domain Driven Design Example rust-ddd Rust Domain-Driven-Design (DDD) Summery This repository is used to present how I find implementing DDD in R

Behrouz R.Farsi 6 Nov 15, 2022
Rust, cargo and QEMU setup for multi-architecture OS development.

rust-osdev-jumpstart Rust, cargo and QEMU setup for multi-architecture OS development. Goal This repo should give you a boost in starting a bare-metal

Alister Lee 27 Nov 20, 2022
A new blockchain architecture under active development, with a strong focus on scalability, privacy and safety

Project Slingshot Accelerating trajectory into interstellar space. Slingshot is a new blockchain architecture under active development, with a strong

Stellar 378 Dec 18, 2022
Simple Event-Driven Microservice Architecture in Rust

Simple Event-Driven Microservice Architecture in Rust is an open-source project showcasing the simplicity and efficiency of building microservices in Rust. This minimalistic project demonstrates an e-commerce backend system, featuring just two microservices: Catalog and Order.

James Mallon 4 Dec 19, 2023
Repo for apps for the Pocket RISC-V core for Analogue Pocket/OpenFPGA. Multiple branches.

This is a repo meant to host Rust programs for agg23's Pocket RISC-V platform. While Rust can be built out of the openfpga-litex repo directly, this r

null 3 Dec 12, 2023
Diosic is an open source web-based music collection server and streamer

Diosic is an open source web-based music collection server and streamer. Mainly suitable for users who need to deploy on servers with low hardware specifications.

Jinker 45 Jan 28, 2023
An AI-native lightweight, reliable, and high performance open-source vector database.

What is OasysDB? OasysDB is a vector database that can be used to store and query high-dimensional vectors. Our goal is to make OasysDB fast and easy

Oasys 3 Dec 25, 2023