Elemental System Designs is an open source project to document system architecture design of popular apps and open source projects that we want to study

Overview

Elemental System Designs

All Contributors

What is this?

Elemental System Designs is an open source project to document system architecture design of popular apps and open source projects that we want to study. The project focuses on the high level abstraction architecture based on use cases of an app that we can observe while using them in real life.

We will focus on features with high-impact of the selected apps, while proposing high scalable and resilient architecture that could scale infinitely. We welcome anyone who is actually in the same field as the app of proposed architecture to give us ideas around industry specific bottlenecks and your solution to the problem.

What is System Design?

System Architecture Design, often called System Design is a conceptual representation of the components and subcomponents that reflects the behaviour of the system as a whole.

System Architects are who make decisions on components to use based on the use cases and requirements given by the business, and consider trade-offs and bottleneck while designing a system.

What I cannot create, I do not understand. β€” Richard Feynman

I am writing this project for me to better understand system architecture of various popular apps that I use daily. While the system design proposals are hypothetical, we bring industry best practices in the proposals, and have them available for anyone who is interested in pursuing the field of system architecture designs as references.

If you spend enough time on planning, coding is easy.

I believe a good architecture is the key to delivering a successful project. A good architecture is required even if you are building a side project or a system for a company while being paid. Once a good architecture is available in front of you well documented, you are more confident and faster as a programmer to hit the deadline of delivering an application (or even a feature within a large ecosystem).

System Designs

  1. tiktok
  2. [WIP] Dropbox

Structure of system designs

1. Introduction to the system

This section briefly explains the app. Any useful text from the app's official webiste or Wikipedia would fit for this

2. Use cases and Requirements

This section details use cases of the app that we are studying. And followed by requirements for this system design.

When picking requirements, we try to pick the most high-impact features of the app, for example in Tiktok example, we've picked viewing videos, uploading videos, and receiving personalised feeds.

3. Constraints

  • Identify traffic and data handling at scale
  • Scale of the system such as requests per second, request types, data written per second, data read per second
  • Special system requirements such as multi-threading

4. Drawing Application Architecture Diagram

What is Application Architecture Diagram?

An application architecture diagram comprises a high-level overview of the components and fundamental interactions within the system, e.g. microservices, databases, etc. The application architecture diagram primarily addresses the β€œWhat” in relation to the system.

We want to keep our system designs to show an abstraction of important components. The diagram should contain following features

  • Be simple
  • All important components of the system to facilitate the requirements
  • System boundaries
    • In an architecture diagram, it is a concept line that divides the system you want to study from 'everything else
    • It is to include elements that you are interested / specified in the use cases / requirements
  • Show system interactions
    • Use simple shapes and lines to indicate process flows and the way the different elements communicate to each others
  • Include useful annotations
    • Add helpful explanation to critical pieces of your diagram

In the diagram, we want to include following components

  • Application service layers
  • Data storage layers
  • Usually a scalable system includes load balancer, event stream, queue, databases (master/slave, replication or sharding model) and in-memory key-value stores such as Redis

For the icons used in the diagram, please check this link

5. Component Design

  • Component + specific APIs required for each of them.

    • The project focuses on the high level design, we should use component design section to enhance the detail that could be too vague in the application architecture diagram. This is the section to write anything important per component level
  • Database schema design

    • A simplified database schema design that focused on only important ones in relation to the requirements and use cases

6. Understanding Bottlenecks

  • Describe any industry specific bottleneck (e.g. security concerns in financial applications)
  • Maybe the application needs a CDN? Load balancer? Data is so huge so database should be sharded?
  • Maybe the application needs complex query to fetch data, caching the result somewhere such as in-memory cache store would be beneficial?

7. Scaling

Vertical Scaling

Provide suggestions on adding more power to each component where it's necessary and important.

Horizontal Scaling

Wherever possible, service components should be horizontally scalable.

Caching suggestions

Can the design take advantage of any following caching strategies?

  • Can any component take an advantage of application layer caching LRU / LFU?
  • Database layer caching?
  • In-memory caches such as Redis or Memcache?
  • CDN to cache static assets?

References

Contributing

Please check CONTRIBUTING.md

Contributors ✨

Thanks goes to these wonderful people (emoji key):


Jason Shin

πŸš‡ πŸ€”

Tyronne Jeong

πŸ€”

GummyBearr

πŸ€”

Dawoud T

πŸ€”

This project follows the all-contributors specification. Contributions of any kind welcome!

You might also like...
A learning project/fun experiment in internet protocol

Piper a learning project/fun experiment in internet protocol Version 0.4.0 (SEMVER) Goals Piper is Simple. A page is a page. There are no secondary re

 Gecko is a high-level, general-purpose programming language built on top of the LLVM project.
Gecko is a high-level, general-purpose programming language built on top of the LLVM project.

Gecko is a high-level, general-purpose programming language built on top of the LLVM project. Gecko Technology & principles Gecko is a general-purpose

Quickly set up a `probe-run` + `defmt` + `flip-link` embedded project

app-template Quickly set up a probe-run + defmt + flip-link embedded project Dependencies 1. flip-link: $ cargo install flip-link 2. probe-run: $ # ma

This repository serves as the backend for the Torrust Index project.
This repository serves as the backend for the Torrust Index project.

Torrust Index Backend πŸ“’ Important Updates πŸ“’ None at the moment ACCESS ALL UPDATES Index PROJECT DESCRIPTION PROJECT ROADMAP DOCUMENTATION INSTALLATI

A project for developing Rust applications

rust_dev This is a project for developing Rust applications. The goal of this project is to provide a solid foundation for building robust and scalabl

The source code that accompanies Hands-on Rust: Effective Learning through 2D Game Development and Play by Herbert Wolverson
The source code that accompanies Hands-on Rust: Effective Learning through 2D Game Development and Play by Herbert Wolverson

Hands-on Rust Source Code This repository contains the source code for the examples found in Hands-on Rust. These are also available from my publisher

Tool to convert variable and function names in C/C++ source code to snake_case

FixNameCase Tool to convert variable and function names in C/C++ source code to snake_case. Hidden files and files listed in .gitignore are untouched.

Source code from Atlas, our 64k demo presented at Revision 2019 with Macau Exports

Atlas source code dump This is a dump of the source code for the engine, graphics tool and player for Atlas, our 64k demo released with Macau Exports

Comments
  • Welcome to Elemental System Designs

    Welcome to Elemental System Designs

    This is a project born out of interest and for learning! All contributions are welcome, even if it's just raising an issue to describe a bug fix, a request to design a system that you are interested in, and improvements to the existing designs.

    We can use this issue to add new contributors using all-contributors bot.

    opened by JasonShin 12
Owner
Jason Shin
Node, Python, Rust, Machine Learning, Functional Programming, React, Vue, Kubernetes and Scala.
Jason Shin
A catalogue of Rust design patterns, anti-patterns and idioms

Rust Design Patterns An open source book about design patterns and idioms in the Rust programming language that you can read here. Contributing You ar

null 6.5k Jan 9, 2023
A Domain Driven Design example application in Rust.

Rust Domain Driven Design Example rust-ddd Rust Domain-Driven-Design (DDD) Summery This repository is used to present how I find implementing DDD in R

Behrouz R.Farsi 6 Nov 15, 2022
Rust, cargo and QEMU setup for multi-architecture OS development.

rust-osdev-jumpstart Rust, cargo and QEMU setup for multi-architecture OS development. Goal This repo should give you a boost in starting a bare-metal

Alister Lee 27 Nov 20, 2022
A new blockchain architecture under active development, with a strong focus on scalability, privacy and safety

Project Slingshot Accelerating trajectory into interstellar space. Slingshot is a new blockchain architecture under active development, with a strong

Stellar 378 Dec 18, 2022
Diosic is an open source web-based music collection server and streamer

Diosic is an open source web-based music collection server and streamer. Mainly suitable for users who need to deploy on servers with low hardware specifications.

Jinker 45 Jan 28, 2023
Featured Dioxus projects on how to build clean user interfaces in Rust

Example projects with Dioxus This repository holds the code for a variety of example projects built with Dioxus. Each project has information on how t

Dioxus 189 Jan 8, 2023
Open-source Autonomy Software in Rust-lang with gRPC for the Roomba series robot vacuum cleaners

CleanIt Open-source Autonomy Software in Rust-lang with gRPC for the Roomba series robot vacuum cleaners Motivation Motivation is to build a complete

Kristoffer Rakstad Solberg 216 Dec 13, 2022
Maai_core is an open source AI that is written in rust

maai_core is an open source AI that is written in rust. It is more focused on being lightweight and speedy.

Max Beier 2 Aug 1, 2022
An open source WCH-Link library/command line tool written in Rust.

wlink - WCH-Link command line tool NOTE: This tool is still in development and not ready for production use. Known Issue: Only support binary firmware

WCH MCU for Rust 22 Mar 7, 2023
This project contains small exercises to get you used to reading and writing Rust code

rustlings ?? ❀️ Greetings and welcome to rustlings. This project contains small exercises to get you used to reading and writing Rust code. This inclu

Cynthia Tran 1 May 24, 2022