Regarding atrium-api/src/com/atproto/sync/subscribe_repos.rs

It would be great to see some sort of example on how to use this. From brief reading of the code, it seems to not be implemented yet, as I cannot find anything regarding reading streams, events or anything like that. Is that correct?

Hey, you folks could use a set of crates to assemble the generated code in a more structured manner:

• Use https://crates.io/crates/quote to write the individual pieces of code
• Use https://crates.io/crates/syn and https://crates.io/crates/proc-macro2 to have structs for pieces of Idents for the struct names etc. and TokenStream
• Use https://crates.io/crates/prettyplease to pretty print the whole shebang

The basic idea:

1. Assemble lots of small TokenStreams to a large one
2. Convert TokenStream to &str using https://docs.rs/proc-macro2/1.0.56/proc_macro2/struct.TokenStream.html#method.to_string
3. Pipe the generated string through https://docs.rs/syn/2.0.15/syn/fn.parse_file.html to generate a https://docs.rs/syn/2.0.15/syn/struct.File.html
4. Pipe https://docs.rs/syn/2.0.15/syn/struct.File.html through https://docs.rs/prettyplease/0.2.4/prettyplease/fn.unparse.html to have a pretty printed String

Given this approach of having a TokenStream you get proper error messages, when something is not a legit AST token and you could add your own pretty printer, ...

If the idea is not clear enough, please ask :)

resolves #11

• Add atrium-xrpc-server member which handles message frames from subscription streams
• Add examples/firehose, it subscribes and prints all texts of posts
  • ref: https://github.com/bluesky-social/feed-generator/blob/main/src/util/subscription.ts
• Update codegen and APIs for these subscription messages

https://github.com/sugyan/atrium/assets/80381/061fb8c4-4cca-4b90-9865-b642dbcd6c45

Switch from serde_urlencoded to a library named serde_qs. serde_urlencoded does not support arrays and has no plan to, so this switches to a fork called serde_qs, which does support common but non-standard query parameters in URLs.

This was breaking get_posts (and probably others). I tested after and get_posts works now. I initially made the changes to codegen and regenerated atrium-api and tested it after that - but it also picked up some other unrelated changes due to changes to the lexicons, so I revered that and made only the relevant changes manually with sed.

I was unable to use refresh_session() and, looking around, I noticed the atrium-xrpc crate's implementation of XrpcClient has an is_refresh parameter to the auth method and a corresponding way to detect refreshes and set that parameter, enabling refreshes. I basically just copied it into the atrium-api version of the trait, which lets me successfully use refresh_session(). I don't think there's a way for refresh_session() to work without this.

atrium_xrpc_server::stream::frames::Frame::try_from returns Result<Self, Box<dyn std::error::Error>>. The problem is that dyn std::error::Error is not Send, which means that if the result of Frame::try_from is potentially used across an await, the entire future is not Send.

To reproduce this, modify the firehose example to spawn the firehose running method instead of running it directly with #[tokio::main]. I tested this at commit 7d99ccd36c8bb562276732d1b9bb888f42530151.

It would be helpful if the error type was Send + Sync + 'static, to also make it compatible with the anyhow crate. But that can be handled by rewriting the error as a string, whereas the future not being Send is more problematic for usage.

resolves #8

Modified to generate code from proc_macro2::TokenStream using quote and syn. I tried to use prettyplease code formatting, but the indentation was a bit weird, so I decided not to use it. In this project, only I (or a specific user with privileges) execute the lexgen command, so I didn't need to consider the environment without rustfmt or the performance of the execution.

🤖 New release

• atrium-api: 0.3.1 -> 0.3.2 (✓ API compatible changes)
• atrium-cli: 0.1.0
• atrium-codegen: 0.1.0
• atrium-lex: 0.1.0
• atrium-xrpc-server: 0.1.0
• firehose: 0.1.0
• atrium: 0.1.0

This PR was generated with release-plz.

🤖 New release

• atrium-api: 0.3.0 -> 0.3.1 (✓ API compatible changes)
• atrium-cli: 0.1.0
• atrium-xrpc: 0.1.0 -> 0.2.0 (⚠️ API breaking changes)
• atrium-codegen: 0.1.0
• atrium-lex: 0.1.0
• atrium-xrpc-server: 0.1.0
• firehose: 0.1.0
• atrium: 0.1.0

⚠️ atrium-xrpc breaking changes

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-check/tree/v0.21.0/src/lints/enum_variant_added.ron

Failed in:
  variant Error:SerdeQs in /tmp/.tmp43KpB4/atrium/atrium-xrpc/src/error.rs:77

--- failure enum_variant_missing: pub enum variant removed or renamed ---

Description:
A publicly-visible enum has at least one variant that is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-check/tree/v0.21.0/src/lints/enum_variant_missing.ron

Failed in:
  variant Error::SerdeUrlencoded, previously in file /tmp/.tmpvWq5Ll/atrium-xrpc/src/error.rs:77

This PR was generated with release-plz.

🤖 New release

• atrium-cli: 0.1.0
• atrium-xrpc: 0.1.0
• atrium-codegen: 0.1.0
• atrium-lex: 0.1.0
• atrium-xrpc-server: 0.1.0
• firehose: 0.1.0
• atrium: 0.1.0

This PR was generated with release-plz.

🤖 New release

• atrium-api: 0.2.0 -> 0.3.0 (⚠️ API breaking changes)
• atrium-cli: 0.1.0
• atrium-xrpc: 0.1.0
• atrium-codegen: 0.1.0
• atrium-lex: 0.1.0
• atrium-xrpc-server: 0.1.0
• firehose: 0.1.0
• atrium: 0.1.0

⚠️ atrium-api breaking changes

--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-check/tree/v0.20.1/src/lints/constructible_struct_adds_field.ron

Failed in:
  field RepoOp.cid in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/sync/subscribe_repos.rs:57
  field Commit.blobs in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/sync/subscribe_repos.rs:10
  field Commit.blocks in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/sync/subscribe_repos.rs:13
  field Commit.commit in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/sync/subscribe_repos.rs:14
  field Commit.prev in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/sync/subscribe_repos.rs:17

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-check/tree/v0.20.1/src/lints/enum_variant_added.ron

Failed in:
  variant ActionViewDetailSubjectEnum:RepoViewNotFound in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/admin/defs.rs:226
  variant ActionViewDetailSubjectEnum:RecordViewNotFound in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/admin/defs.rs:230
  variant ViewRecordEnum:AppBskyFeedDefsGeneratorView in /tmp/.tmpLCrufW/atrium/atrium-api/src/app/bsky/embed/record.rs:64
  variant Record:AppBskyFeedGenerator in /tmp/.tmpLCrufW/atrium/atrium-api/src/records.rs:9
  variant Error:RepoNotFound in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/admin/get_repo.rs:29
  variant ReportViewDetailSubjectEnum:RepoViewNotFound in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/admin/defs.rs:254
  variant ReportViewDetailSubjectEnum:RecordViewNotFound in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/admin/defs.rs:258
  variant Error:RecordNotFound in /tmp/.tmpLCrufW/atrium/atrium-api/src/com/atproto/admin/get_record.rs:31
  variant PreferencesItem:SavedFeedsPref in /tmp/.tmpLCrufW/atrium/atrium-api/src/app/bsky/actor/defs.rs:112

This PR was generated with release-plz.

I've written a quick example client to learn about the API. One feature I wanted for my client was to use interior mutability so that the client could refresh its tokens transparently to the consumer, even behind an immutable reference. As you can see, I used a mutex for this - but the fact that auth returns Option<&str> made that pretty difficult. It would also be difficult if you wanted to, for instance, retrieve a token from the environment dynamically in the auth method. My app runs against a branch in my fork that makes the simple change of returning Option<String> instead. You mentioned you were in the middle of refactoring this trait so I thought I'd make this suggestion now.

Thanks for all the work on this! It's been fun to use.

• Update API (https://github.com/bluesky-social/atproto/commit/4a6c9764e3bc1b9d1a3de9babebde4592ac79dbe)
• Update atrium_api::xrpc interface, errors
• Add atrium_api::agent
  • Implement all XRPC traits for this agent
• Update codegen for these changes

Hey, love the work you did so far for atrium. Do you plan on adding axum support for the xrpc-server or what are your plans with this crate?

Best regards icepuma