Add your suggestion as a comment!

Background

Our Substrate code base is starting to get more complicated, and it would be a benefit to harmonise the set of major conventions we follow, so as to follow good best practices, and make reviews more efficient. The goal of this issue is to accumulate suggestions over time, as replies, which we can turn into an eventual convention document. This document can further be turned into rules for our CI linter.

Major questions that

• How to decide what is its own module, vs. combining with existing module?
• Modules deserve their own repo? every module?

Initial suggestions

• All maps must map to Option to avoid default construction behaviour of StorageMap from allowing us to be lazy about checking ::exists on the same map before lookup.

• Always try to make a module easily reusable for another runtime, by:

  • If possible, always provide your own traits for your expectations on other modules, rather than relying on public traits, or traits in the module you are expecting to use.
  • Don't define and implement traits for your own module, see point above.
  • If possible, strive to implement the business logic of your module separately from the runtime module itself, in a substrate agnostic way.

• Assert as many invariants as possible!

Background

So far, we have not pinned down the token supply, and actually operated with a denomination where $JOY was the base currency unit (i.e. Satoshis of Joystream).

Proposal

• Token Asset Name: $JOY
• Base Unit Name: BDH (joking :rofl: ... seriously we need a name here...! ask Atlas God-King kdembler or ask community?)
• $JOY Token Denomination: 10,000,000,000 (ten Billion) BDH
• Initial $JOY supply: 1,000,000,000 (1 Billion) $JOY
• Initial BDH supply: Initial $JOY supply x Token Denomination = 1e^19 BDH

┆Issue is synchronized with this Asana task by Unito

Notes

• $CRT = generic creator token
• creator = anyone that is the owner of a channel, has the right to issue a own $CRT

Pallet components

Supporting library

(token core implementation): I am in favor of using assets: Dependencies:

codec = { package = "parity-scale-codec", version = "3.0.0", default-features = false }
scale-info = { version = "2.0.1", default-features = false, features = ["derive"] }
sp-std = { version = "4.0.0", default-features = false, path = "../../primitives/std" }
# Needed for various traits. In our case, `OnFinalize`.
sp-runtime = { version = "6.0.0", default-features = false, path = "../../primitives/runtime" }
# Needed for type-safe access to storage DB.
frame-support = { version = "4.0.0-dev", default-features = false, path = "../support" }
# `system` module provides us with all sorts of useful stuff and macros depend on it being around.
frame-system = { version = "4.0.0-dev", default-features = false, path = "../system" }
frame-benchmarking = { version = "4.0.0-dev", default-features = false, path = "../benchmarking", optional = true }

This will provide us with a bunch of primitives such as :

• burn(token_id, account, amount) here
• mint(token_id, account, amount) here
• transfer(token_id, from, to, amount) here

CRT token core

• initial creator allocation, whose implementation details needs to be developed
• vesting schedule, same
• existential amount: smallest amount that an account can hold under which the account is "destroyed" and the balance is burned. Also we want add a custom crt_token::burn function that ensure this constraint is respected whenever balance of freshly burned accounts drop below the existential_amount threshold.
• crt_token::{stake, unstake} functions, that I suppose can be build on top of freeze, thaw [freeze][(https://docs.rs/pallet-assets/latest/src/pallet_assets/lib.rs.html#190](https://docs.rs/pallet-assets/latest/src/pallet_assets/lib.rs.html#626)), thaw
• Liqudity bootstrapping state: Idle, Sell, AMM. Allowed transitions: Idle -> AMM, Idle -> Sale, Sale -> AMM

Transfer

this encapsules how transferring CRT works, What I am thinking is to build on top of transfer primitive in order to change behavior according to the selected Policy (Permissioned, Permissionless). The main goal of this section is to have a crt_token::transfer(token_id, from, to, amount) that:

1. Ensure outstanding policy constraint are respected
2. amount of token_id is transferred from from to to.
3. if the balance in from < existential_deposit then balance(from) is burned and from account is removed. There should be also support for multi output payments, by allowing transfer (or a variant of) to accept a list [(to, amount)]

Minting & Patronage

What I am thinking here is to build on top of mint primitive in order to:

1. mint selected amount into specified account with mint(token_id, amount, account)
2. credit extra inflationary_amount = amount * PATRONAGE_RATE/100 to the creator account It is creator's responsibility to cash out such amount, by calling appropriate extrinsic Once completed we should have the crt_token::mint(token_id,account, amount), whose purpose is twofold: mint amount + credit creator with patronage. There should be an extrinsic allowing the creator to reduce the patronage rate

Sale

Requires: crt_token::{mint, burn} First method in order to bootstrap liquidity for $CRT. A sale period defined by (start, duration) is issued which can be updated any time prior. Optional whitelist containing member with respective cap amounts that they can receive from the sale. This member whitelist can be used as starting whitelist if the initial transfer policy for $CRT is Permissioned. creator $CRT/$JOY for the sale. Proceeds (liquidity) go into creator JOY account, while tokens "sold" are crt_token::minted into the buyer $CRT account. A percentage of JOY is burned on top of each sale. Council control this percentage parameter

Revenue split

Requires: crt_token::{stake, unstake} The creator, holds Y $JOY in its account by issuing a Sale announces that he intends to distribute percentage S of amount X (in JOY). any token holder interested must crt_token::stake V/Q * 100% (percentage of total $CRT issuance) in order to claim S/100 * X * V/Q $JOY at the end of the split period. After the payout has been received in the specified JOY account the $CRT amount can be crt_token::unstaked.

AMM

Terminal state for $CRT and alternative to a Sale for bootstrapping liquidty. Requires: crt_token::{mint, burn}

• Inner workings are still to be fully decided and inherently complex
• AMM buys $JOY from buyer and in turns it crt_token::mints $CRT into the buyer's account.
• AMM sell $JOY to seller and in turns it crt_token::burns $CRT from the seller's account

Timeline

Stage 0:

• Research on supporting library: establish feasibility of using the substrate asset pallet

Stage 1:

Each of these components can be implemented concurrently and independently (more or less)

• CRT core: add extra features
• Transfer
• Patronage

Stage 2:

This will build on Stage1 being reviewed and tested. The following components can be build concurrently & independently

• Revenue Split
• Sale
• AMM (I am doing this)

Questions

1. What do we do when a $CRT holder gets removed from whitelist in a Permissioned Transfer policy? Can he still transact with whitelisted members?

Open problems / Comments

• establish feasibility of using the substrate asset pallet
• can we eventually add vesting feature to the substrate assets pallet?

Background

Using tokens as a way to finance creative projects and also reward early evangelists and community members has been a long standing idea in the crypto space, with attempts such as Smart Media Tokens and TatianaCoin. The idea is to turn a creative project into something where a community of token holds can vote on key governance decisions about how to manage the project, and possibly also receive a share of any value captured by the project.

Goal

Introduce DAOs with their own governance token, in the form of a new runtime module daos, which can act in the Joystream chain, primarily in the content directory & storage system at this point, but in the future they may act in any part of the system where normal memberships can, so stand for council, make proposals and so on.

Requirements

• Must be implemented as a native runtime module.
• There is no supported suitable asset protocol to use which supports minting new tokens over time so we must build it in.
• Has to be purely event based, no timers, no iteration, because we want an unconstrained number of DAOs, each having an unconstrained number of stakeholders and proposals.

Governance Token

Each DAO has an associated governance token, which is a fungible asset controlled by normal substrate accounts, and with normal currency semantics. It has an issuance, and an issuance policy, set when the token is created, which either freezes the issuance upper bound, or which allows new tokens to be minted by the DAO. In either case the issuance does decline when tokens are burned. Tokens can also be locked for the purpose of participating in the governance process of the DAO.

In the v2 of these DAOs we will introduce the capability for DAOs to buy back and burn the governance token on the open market in exchange for JOY tokens in the treasury, which itself is described in the next section.

Metadata

All DAOs have the following metadata which aid

• An immutable handle.
• A mutable title.
• A avatar image.
• A description of the purpose of the DAO.

Treasury

Each DAO also controls a pool of native JOY tokens, called a treasury, and can spend from this pool to any Substrate account. In the future this pool will be credited for with revenue shares from content owned by channels owned by the DAO. The treasury can also be credited by any normal

Operator

Each DAO has up to one explicit member set as the operator of the DAO. This operator is responsible for conducting various ongoing tasks in the DAO, such as possibly managing any channels under the control of the DAO.

The operator receives rewards in the governance token over time into a reward account. Importantly, the rewards are not automatically paid out, as this would not scale as the number of DAOs increases. Hence a pull based method is required, e.g. similar to this:

https://github.com/Joystream/joystream/issues/1305

but be aware that this issue is about a more complex situation, the key point is just that its pull based periodic payments, so use it as inspiration.

Actions

All DAOs have the same set of things they can do in the Joystream system, called actions. It is a static list, and it deals both with the controls of the DAO itself, but also interacting with the rest of the runtime and system. Here are some example actions

• Change the operator.
• Update reward terms of the operator.
• Update the DAO metadata.
• Spend from the treasury to some account.
• Mint new tokens to some account.
• Upload something to the storage system.
• Delete something owned the storage system.
• Creating a channel in the content system.
• All actions an owner of a channel can do in the content system.
• Update action settings.

The proposal system, described later, allows the governance process of the DAO to trigger any such action. Some subset of actions may also allow the operator to trigger an action, and there are so called action settings in each DAO which describe this.

Creation

DAOs are created by any member, and creating a DAO reserves you the right to set the initial operator. The token is created at the same time as the DAO, and this requires both the issuance and and the issuance policy. All initial tokens are set to be owned by the operator. Lastly, the initial action settings are also set during creation.

Proposal System

The proposal system in the DAO is the means by which the token holders exercise governance over the operation of the DAO. The system process proposals, which are propositions about executing specific actions in the DAO, so there is one proposal type for each action available. Each type as its own values for the required quorum and threshold for passing.

Proposals

Each proposal is made up of the following set of properties when first created

• Type: What type of action is being triggered.
• Parameter Values: Values for the parameters of the given action.
• Proposer: The member proposing the proposal.
• Stake: Some quantity of the governance token is locked up for the duration of the proposal life-cycle. The stake does not count towards the outcome, and must be an exact amount which is identical across all proposals. The only purpose of this stake is to induce proposals to clean up their proposal from the blockchain state when it is over.
• Metadata: Raw metadata field which can provide rich human friendly explainer payload, with such things as titles, descriptions, etc.

Stages

A proposal has the following stages

• Announced: This is the initial stage of a proposal when created. During this stage the only action which can occur is that the proposer may withdraw the proposal, removing it from the system, this is not possible after this stage.
• Voting: This is the next stage, during which any token holder may vote on the proposal. The impact of a vote is weighted by how many tokens are locked, and the quorum and threshold semantics are the same as in the normal proposal system.
• Gracing: This is a quiescent stage where nothing can happen, and the final tally will be publicly available, and therefore the outcome.
• Ready: At any time, anyone can tell the proposal to execute - if it passed, if it did not pass, then nothing is executed. In either case, the initial stake is unlocked. The proposal may succeed or fail, depending on whether business logic involved in executing the action is compatible with the state of the system at the time of execution.

The length of each stage is hard coded, but possibly different for different types.

Voting

When voting, you vote as a member, this means the account holding the tokens must be bound to the membership up front. A single account can vote at most one time on a given proposal, however, it can simultaneously be used to vote on an unlimited number of proposals. Voting entails locking up some quantity of funds until the Ready period begins, at which point a second transaction is needed to remove a given lock. This means you may need to send multiple such transactions over time, for the same account, if you have used it for multiple proposal votes. Funds locked for staking for proposals can also be reused for voting. Lastly, a vote is either for or against the proposal passing.

Fundraising & Buy-back Burns [RnD Missing]

Two kinds of actions are needed, but no design has been conceived.

• Fundraising: This is the act of minting new tokens and auctioning them off for JOY, which ends up in the treasury. This should only be possible a single time in the very beginning of the life time of a new DAO with a fixed issuance policy.
• Buy-back & Burn: This is the act of buying back governance tokens from the open market in exchange for JOY in the treasury, and burning the obtained tokens.

There are different auction types worth looking into, multi unit auction, dutch auction, reverse dutch auction, etc. Doing these within our computational constraints, and also without being badly vulnerable to attacks or fraud, will require some care. The primary concern here is a small faction of stakeholders, or even the operator, setting up one of these quickly and with very bad terms, and thereby exploiting the rest. Long announcement and grace periods can help ameliorate this, and ultimately intervention from working group if there is sufficient time to block.

DAO Working Group

There will be a DAO working group which can intervene in the operation of any DAO by specifically blocking a proposal at any stage of its life-cycle prior to being executed.

┆Issue is synchronized with this Asana task by Unito

Text was too long for Github issue, made Gist: https://gist.github.com/bedeho/1b231111596e25b215bc66f0bd0e7ccc

This proposal badly needs review for outright mistakes and omissions, keep discussion here, not in Gist.

Purpose

The purpose of this issue to track overall progress on the storage system in the Giza release. This work is complex, in particular in the in that it introduce two new nodes, coordinated by a new large runtime module, and which have to interact in large numbers with each other. It will be critical to have the lead developers on this system to stay tightly synchronized, and keep other people depending on their work (e.g. the release manager, devops, etc.) updated on their progress.

Problem

It was found during the audit that any forum user can use the forum's extrinsic create_thread to fill up the blockchain storage and exhaust the number of threads of a given category for a small fee since this requires no deposit.

This problem exists in two forms across the runtime: Either we allow with no deposit filling up the storage of a given module with no bound or with bounds.

In the bounded case, this would allow a denial of service attack where other users won't be able to use the storage.

In the unbounded case, this'd allow to grow the storage maliciously.

There are different permission levels that'd allow a user to abuse these extrinsics, either a lead or a worker/member.

When a lead can fill up a bounded storage is not really a problem because there is no other user that it can be denied of service but I leave those cases in here just to keep them in mind.

Pallets

• forum

  • create_thread: Any forum user can create a thread, the thread storage for a given category is bounded.
  • add_post: Any forum user can create a post, the post storage for a given thread is bounded.
  • create_category: The lead can create categories with no cost, the category storage is bounded. (Same goes for sub-categories)

• working_group

  • add_opening: A lead can use this extrinsic to create openings unbounded.
  • apply_on_opening: A working group member can apply multiple times, filling blockchain storage(when no stakes) unbounded.
  • add_opening + apply_on_opening + fill_opening: A lead can add an opening with no stakes, apply on it multiple times and fill it, and occupy completely the worker storage.

• membership

  • add_staking_account_candidate: A user could use multiple accounts to fill up the blockchain storage, unbounded.

• proposals_discussion

  • add_post: A member can add posts to any thread, unbounded.

• storage/data_directory

  • add_content: A member can use it to fill up the storage unbounded.

• content_directory

  • add_curator_group: A lead can add groups to fill up the storage unbounded.
  • add_curator_to_group: A lead can add curators to a group to fill it up bounded.

• service_discovery:

  • set_ipns_id: Any worker can use it to fill up storage unbounded.

• data_object_type_registry

  • register_data_object_type: A lead can use it to fill up storage unbounded.

• Drop data object id association in video: maybe_data_objects_id_set
• Drop incrementing ChannelRecord::num_videos in update_video.
• introduce atomic data object deletion in: for all of these, user provides objects: BTreeSet<DataObjectId>, for all each, they will check that deletion is feasible (by checking bag membership of all objects), and then this is done as first side-effect after mutation barrier.
  • update_video
  • update_channel
  • delete_video
  • delete_channel

Proposal

T.O.C.

1. Goal
2. Post structure
   1. Post
   2. Replies / Comments
3. Storage Elements
   1. Posts
   2. Replies
4. Extrinsics 1. Create Post 2. Create Reply 3. Edit Post 4. Edit Reply 5. Delete Post 6. Delete Reply 7. React to Post
5. Questions & precisations

Target Branch: new_channel_features

Module: content

Goal

Enrich each video with features for express a rating and comments, something similar to a Youtube/Vimeo page. Basic idea A video should start a thread post (since in a thread all the posted elements tends to be threated as simple posts/replies). Instead what we want is a post that starts with a video and thereafter there are only replies (to the post or to other replies). Effectively it should be something along the lines of:

Post structure

Basic idea: A video that lives in a channel should start a post. Once the post is set up only text replies are allowed by post owner and other users We also want the video to be in the channel owned by the post owner

Post

We want the video to be in the channel owner by the post owner

  pub struct Post<ChannelOwner,
		  Hash,
		  Balance,
		  BlockNumber,
		  ReplyId, 
		  VideoCategoryId
		  > {
      /// Id of thread to which this post corresponds.
n
      /// Hash of current text
      pub title_hash: Hash,

      /// Author of post.
      pub author: ChannelOwner,

      /// Cleanup pay off
      pub cleanup_pay_off: Balance,

      /// When it was created or last edited
      pub last_edited: BlockNumber,

      /// Overall replies counter
      pub replies_count: ReplyId,

      /// video associated to the post (instead of the body hash as in the blog module)
      pub video: VideoId,

      /// category for this video
      pub category: CategoryId
  }

The category field allows to use the same structure for moderation (Moderators, Lead …) as in the forum module.

Replies / Comments

Reply is analogous to a Youtube comment, so every post (video) will have zero or more replies, and also a reply can have zero or more reply to it. It makes sense therefore to identify as a Parent a Post | Reply that can have zero or more replies to it.

pub enum ParentId<ReplyId, PostId: Default> {
    Reply(ReplyId),
    Post(PostId),
}

And a reply struct will be something like

pub struct Reply<BlockNumber,
		 ReplyId,
		 PostId,
		 UserId,
		 Balance,
		 Hash> {
    /// Reply text hash
    text_hash: Hash,
    /// Associated with reply owner
    owner: UserId, 
    /// Reply`s parent id
    parent_id: ParentId<T::ReplyId, PostId>,
    /// Pay off by deleting post
    cleanup_pay_off: Balance,
    /// Last time reply was edited
    last_edited: BlockNumber,
}

Storage Elements

Posts

Stored by means of a single map

pub PostById get(fn post_by_id) config() : map
    hasher(blake2_128_concat) PostId => Post;

Replies

Stored by means of a double map (PostId,

ReplyById get (fn reply_by_id): double_map hasher(blake2_128_concat) PostId, hasher(blake2_128_concat) ReplyId => Reply>;

Extrinsics

Create Post

We have to make sure that the post owner is the owner of the channel that the video belongs to.

  pub fn create_post(origin,
		     title: Hash,
		     video: VideoId,
		     category: CategoryId
  ) -> DispatchResult 
{
    ...
	let post_owner = <ChannelById>::get(
	    <VideoById>::get(video)
		.in_channel)
	.channel_owner;
    ...
}

Create Reply

pub fn create_reply(
    origin,
    owner: UserId,
    post_id: PostId,
    reply_id: Option<ReplyId>,
    text: Hash,
) -> DispatchResult {

reply_id will be different from None in case of nested replies.

Edit Post

pub fn edit_post(
    origin,
    post_id: PostId,
    new_title: Option<Hash>
    new_video: Option<VideoId>
) -> DispatchResult {

We must make sure that origin is signed by the post owner. If new_title (resp. new_video) are none title (video) stays the same.

Edit Reply

pub fn edit_reply(
    origin,
    owner: UserId,
    post_id: PostId,
    reply_id: T::ReplyId,
    new_text: Hash
) -> DispatchResult

Replace the reply [ReplyId,PostId] (if it exists) in the ReplyById map using the new parameters, after ensuring that origin is signed by the owner of the reply in question.

Delete Post

pub fn delete_post(
    origin,
    owner: ChannelOwner,
    post: PostId,
) -> DispatchResult

After making sure that origin is signed by the post owner, delete the post and possibly all the replies to it.

Delete Reply

pub fn delete_replies(
    origin,
    owner: UserId,
    reply: ReplyId,
) -> DispatchResult

After ensuring that origin is signed by the owner of reply, delete the reply in question and possibly all the replies to it.

React to Post

fn react_post(origin,
	      post: PostId,
	      react: PostReactionId
) -> DispatchResult

Add a reaction to the post

Questions & precisations

1. Should I write the code in a new file and in a new pallet
2. When I delete a reply should also delete all the replies to it? Same thing when deleting a post?
3. I used UserId to represent a reply owner different from ChannelOwner on purpouse: you can reply even if you don't own a channel. Effectively UserId will be common::MemberId (that is the member of the system).
4. Should I add a reaction_score field to the Post data structure in order to keep track of the reaction? It appears that in the forum module a reaction causes only an Event deposit

Take a benchmarked runtime and any other required information as given,

1. For a specific proposed weight price w (s/JOY) and byte price f (bytes/JOY), by what methodology to classify this combination as acceptable or not acceptable.
2. For two such combinations(w_1,f_1) and (w_2,f_2), by what methodology can we order them to say (w_1,f_1) >=(w_2,f_2) in terms of desirability.

Without this it seems hard to come up with suitable tools of analysis for determining such prices.

Various fixes primarily to deploy playground, and fix multi-arch docker build.

• Update cargo package librocksdb-sys which was not building correctly for armv7. Since olympia is going to be a new chain it is safe to use newer version. Docker builds now works in the build-node-docker action so caching images is back.
• Fix in Deploy Playground to async wait for build to complete when a new joystream-node image is being built (when there is no cached image)
• Apply same idea from https://github.com/Joystream/joystream/pull/3158
• Install newer docker-compose in setup.sh

The ability to get a client side computation of the circulating supply schedule would greatly help with the credibility of the supply claims of the system, something which is particularly important since Joystream is a Substrate L1, for which there is limited tooling and infra for third parties to inspect this without help. Basic information needed for such a command is to be able to ask how many tokens will both be locked and unlocked at

1. the current block
2. any block in the future
3. all blocks for a given future interval, with controllable block resolution (daily, weekly, monthly...)

Importantly, locking here only refers to vesting locks, other locks, like staking, nomination etc. are not relevant for this credibility use case, as these locking dynamics are volatile and not predictable.

Seems like it should be part of CLI, as it seems rough to add a totally separate tool for this.

Looking at Pioneer codebase for reading locks can be a good guide if one is unfamiliar with this topic.

I just learned the minimum salary per block can be less than what the CLI allows. That limit no longer makes sense.

In pioneer the only condition is

rewardPerBlock: BNSchema.test(moreThanMixed(0, 'Amount must be greater than zero')) https://github.com/Joystream/pioneer/blob/dev/packages/ui/src/proposals/modals/AddNewProposal/helpers.ts#L268

This PR addresses:

• #4128

~~AFAICS from the code this problem should have only affected proposals.~~

In short the problem was that createType('Bytes', string).toHuman() returns the original string when string is pure ASCII but weirdly it returns the hex of the string when it contains non ASCII chars.

bytesToString encodes the bytes into a UTF8 string and escapes it.

Rationale

The current implementation of unstaking ("cooldown") period is not very useful because there usually is nothing to cool and the filled WG lead position in passing is blocking the council from hiring a new lead, impacting DAO productivity.

Furthermore the unstaking length in blocks has to be set before hiring the lead when creating the opening proposal, usually months before lead behavior can be assessed. This leaves the council with potentially harmful leads that can't be removed or replaced. Hence this function carries high risk and is unlikely to be used in the current form.

The only option i see this may be of use for is that the council is not satisfied with lead performance but can't afford the position to be empty while assessing new candidates. In this case the "cooldown" would give the council more time to make a better choice while the WG stays functional.

It can however be useful for the council having the option to keep stake of passing leads locked and slashable.

Scope

Not mutually exclusive options

1. Allow two or more lead positions at the same time to ensure smooth onboarding of the new lead while the terminated one can still be harmful to the system.
2. A fired lead is no use, there's no need for an unstaking period to block new leads. The stake can stay locked while allowing new hires.
3. Allow to set a new lead via proposal if the former lead was terminated. Replace the lead position on proposal execution.

Option 3 seems favorable to me. This was discussed in #2358, the suggested solution was #2368 (apparently closed without implementation).