in-place allocation-reusing queues for Rust

Overview

FAQs

  • Q: Why did you make this?

    A: For tracing, I wanted to be able to send formatted log lines to a dedicated worker thread that writes them to a file. Right now, we do this using crossbeam-channel. However, this has the sad disadvantage that we have to allocate Strings, send them through the channel to the writer, and immediately drop them. It would be nice to do this while reusing those allocations. Thus...StringBuf.

  • Q: Is it lock-free?

    A: Extremely.

  • Q: Why is there only a bounded variant?

    A: Because unbounded queues are of the Devil.

  • Q: Isn't this just a giant memory leak?

    A: If you use it wrong, yes.

  • Q: Why is it called that?

    A: Originally, I imagined it as a kind of ring buffer, so (as a pun on "ringbuf"), I called it "stringbuf". Then, I realized you could do this with more than just strings. In fact, it can be generalized to arbitrary...things. So, "thingbuf".

  • Q: Why don't the Ref types implement Deref and DerefMut?

    A: Blame loom for this.

Comments
  • crate does not compile as no_std

    crate does not compile as no_std

    cargo check --no-default-features --features alloc
    

    Results in:

        Checking thingbuf v0.1.1 (/Users/chris/repos/github.com/hawkw/thingbuf)
    error[E0433]: failed to resolve: could not find `unreachable_unchecked` in the crate root
       --> src/macros.rs:100:16
        |
    100 |         crate::unreachable_unchecked!(@inner , format_args!(": {}", format_args!($($arg)*)))
        |                ^^^^^^^^^^^^^^^^^^^^^ could not find `unreachable_unchecked` in the crate root
        |
       ::: src/util/panic.rs:25:9
        |
    25  |         unreachable_unchecked!("code compiled with no_std cannot unwind!")
        |         ------------------------------------------------------------------ in this macro invocation
        |
        = note: this error originates in the macro `unreachable_unchecked` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:169:9
        |
    169 |         test_println!("Core::close");
        |         ---------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:185:9
        |
    185 |         test_println!("push_ref");
        |         ------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:224:25
        |
    224 |                         test_println!("claimed tail slot [{}]", idx);
        |                         -------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:235:33
        |
    235 | ...                   test_println!("-> initialized");
        |                       ------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:240:33
        |
    240 | ...                   test_println!("-> recycled");
        |                       ---------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:270:21
        |
    270 |                     test_println!("channel full");
        |                     ----------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:285:9
        |
    285 |         test_println!("pop_ref");
        |         ------------------------ in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:322:25
        |
    322 |                         test_println!("claimed head slot [{}]", idx);
        |                         -------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:352:25
        |
    352 |                         test_println!("--> channel full!");
        |                         ---------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/lib.rs:478:9
        |
    478 |         test_println!("drop Ref<{}>", core::any::type_name::<T>());
        |         ---------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
      --> src/macros.rs:7:17
       |
    7  |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
       |                 ^^^^^^^
       |
      ::: src/wait.rs:79:9
       |
    79 |         test_println!("WAKING TASK {:?} (from {:?})", self, thread::current());
       |         ---------------------------------------------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:602:9
        |
    602 |         test_println!("List::remove({:p})", node);
        |         ----------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:578:9
        |
    578 |         test_println!("List::dequeue({:?}) -> {:p}", new_state, last);
        |         ------------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:549:9
        |
    549 |         test_println!("List::enqueue({:p})", waiter);
        |         -------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:473:9
        |
    473 |         test_println!("Waiter::remove({:p})", self);
        |         ------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:444:9
        |
    444 |         test_println!("WaitQueue::close()");
        |         ----------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:399:9
        |
    399 |         test_println!("WaitQueue::notify_slow(state: {})", state);
        |         --------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:371:9
        |
    371 |         test_println!("WaitQueue::notify()");
        |         ------------------------------------ in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:327:9
        |
    327 |         test_println!("WaitQueue::continue_wait_slow({:p})", node);
        |         ---------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:302:9
        |
    302 |         test_println!("WaitQueue::continue_wait({:p})", node);
        |         ----------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:209:9
        |
    209 |         test_println!("WaitQueue::start_wait_slow({:p})", node);
        |         ------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/queue.rs:189:9
        |
    189 |         test_println!("WaitQueue::start_wait({:p})", node);
        |         -------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/cell.rs:161:9
        |
    161 |         test_println!("notifying; close={:?};", close);
        |         ---------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/wait/cell.rs:112:17
        |
    112 |                 test_println!("-> was notified; state={:?}", actual);
        |                 ---------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
      --> src/macros.rs:7:17
       |
    7  |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
       |                 ^^^^^^^
       |
      ::: src/wait/cell.rs:89:9
       |
    89 |         test_println!("-> locked!");
       |         --------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
      --> src/macros.rs:7:17
       |
    7  |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
       |                 ^^^^^^^
       |
      ::: src/wait/cell.rs:66:9
       |
    66 |         test_println!("registering waiter");
       |         ----------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
      --> src/macros.rs:7:17
       |
    7  |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
       |                 ^^^^^^^
       |
      ::: src/util.rs:60:9
       |
    60 |         test_println!("thread::yield_now()");
       |         ------------------------------------ in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
      --> src/macros.rs:7:17
       |
    7  |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
       |                 ^^^^^^^
       |
      ::: src/util.rs:57:13
       |
    57 |             test_println!("hint::spin_loop() (x{})", 1 << self.0);
       |             ----------------------------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
      --> src/macros.rs:7:17
       |
    7  |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
       |                 ^^^^^^^
       |
      ::: src/util/mutex/spin_impl.rs:93:9
       |
    93 |         test_println!("unlocked!");
       |         -------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
      --> src/macros.rs:7:17
       |
    7  |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
       |                 ^^^^^^^
       |
      ::: src/util/mutex/spin_impl.rs:48:9
       |
    48 |         test_println!("-> locked {}!", core::any::type_name::<T>());
       |         ----------------------------------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
      --> src/macros.rs:7:17
       |
    7  |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
       |                 ^^^^^^^
       |
      ::: src/util/mutex/spin_impl.rs:40:9
       |
    40 |         test_println!("locking {}...", core::any::type_name::<T>());
       |         ----------------------------------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
        --> src/macros.rs:7:17
         |
    7    |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
         |                 ^^^^^^^
         |
        ::: src/mpsc/async_impl.rs:1358:9
         |
    1358 |         test_println!("SendRefFuture::drop({:p})", self);
         |         ------------------------------------------------ in this macro invocation
         |
         = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
        --> src/macros.rs:7:17
         |
    7    |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
         |                 ^^^^^^^
         |
        ::: src/mpsc/async_impl.rs:1299:9
         |
    1299 |         test_println!("SendRefFuture::poll({:p})", self);
         |         ------------------------------------------------ in this macro invocation
         |
         = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/mpsc.rs:484:9
        |
    484 |         test_println!("notifying tx ({})", core::any::type_name::<N>());
        |         --------------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/mpsc.rs:476:9
        |
    476 |         test_println!("notifying rx ({})", core::any::type_name::<N>());
        |         --------------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/mpsc.rs:395:21
        |
    395 |                     test_println!("-> yield");
        |                     ------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/mpsc.rs:382:13
        |
    382 |             test_println!("poll_recv_ref => loop");
        |             -------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/mpsc.rs:380:9
        |
    380 |         test_println!("poll_recv_ref");
        |         ------------------------------ in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error: cannot find macro `println` in this scope
       --> src/macros.rs:7:17
        |
    7   |                 println!("[PANIC {:>30}:{:<3}] {}", file!(), line!(), format_args!($($arg)*))
        |                 ^^^^^^^
        |
       ::: src/mpsc.rs:322:13
        |
    322 |             test_println!("draining_queue");
        |             ------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/mpsc.rs:322:13
        |
    322 |             test_println!("draining_queue");
        |             ------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/mpsc.rs:380:9
        |
    380 |         test_println!("poll_recv_ref");
        |         ------------------------------ in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/mpsc.rs:382:13
        |
    382 |             test_println!("poll_recv_ref => loop");
        |             -------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/mpsc.rs:395:21
        |
    395 |                     test_println!("-> yield");
        |                     ------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/mpsc.rs:476:9
        |
    476 |         test_println!("notifying rx ({})", core::any::type_name::<N>());
        |         --------------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/mpsc.rs:484:9
        |
    484 |         test_println!("notifying tx ({})", core::any::type_name::<N>());
        |         --------------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
        --> src/macros.rs:11:34
         |
    11   |                     crate::loom::thread::current().id(),
         |                                  ^^^^^^ could not find `thread` in `loom`
         |
        ::: src/mpsc/async_impl.rs:1299:9
         |
    1299 |         test_println!("SendRefFuture::poll({:p})", self);
         |         ------------------------------------------------ in this macro invocation
         |
         = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
        --> src/macros.rs:11:34
         |
    11   |                     crate::loom::thread::current().id(),
         |                                  ^^^^^^ could not find `thread` in `loom`
         |
        ::: src/mpsc/async_impl.rs:1358:9
         |
    1358 |         test_println!("SendRefFuture::drop({:p})", self);
         |         ------------------------------------------------ in this macro invocation
         |
         = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
      --> src/macros.rs:11:34
       |
    11 |                     crate::loom::thread::current().id(),
       |                                  ^^^^^^ could not find `thread` in `loom`
       |
      ::: src/util/mutex/spin_impl.rs:40:9
       |
    40 |         test_println!("locking {}...", core::any::type_name::<T>());
       |         ----------------------------------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
      --> src/macros.rs:11:34
       |
    11 |                     crate::loom::thread::current().id(),
       |                                  ^^^^^^ could not find `thread` in `loom`
       |
      ::: src/util/mutex/spin_impl.rs:48:9
       |
    48 |         test_println!("-> locked {}!", core::any::type_name::<T>());
       |         ----------------------------------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
      --> src/macros.rs:11:34
       |
    11 |                     crate::loom::thread::current().id(),
       |                                  ^^^^^^ could not find `thread` in `loom`
       |
      ::: src/util/mutex/spin_impl.rs:93:9
       |
    93 |         test_println!("unlocked!");
       |         -------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
      --> src/macros.rs:11:34
       |
    11 |                     crate::loom::thread::current().id(),
       |                                  ^^^^^^ could not find `thread` in `loom`
       |
      ::: src/util.rs:57:13
       |
    57 |             test_println!("hint::spin_loop() (x{})", 1 << self.0);
       |             ----------------------------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
      --> src/macros.rs:11:34
       |
    11 |                     crate::loom::thread::current().id(),
       |                                  ^^^^^^ could not find `thread` in `loom`
       |
      ::: src/util.rs:60:9
       |
    60 |         test_println!("thread::yield_now()");
       |         ------------------------------------ in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
      --> src/macros.rs:11:34
       |
    11 |                     crate::loom::thread::current().id(),
       |                                  ^^^^^^ could not find `thread` in `loom`
       |
      ::: src/wait/cell.rs:66:9
       |
    66 |         test_println!("registering waiter");
       |         ----------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
      --> src/macros.rs:11:34
       |
    11 |                     crate::loom::thread::current().id(),
       |                                  ^^^^^^ could not find `thread` in `loom`
       |
      ::: src/wait/cell.rs:89:9
       |
    89 |         test_println!("-> locked!");
       |         --------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/cell.rs:112:17
        |
    112 |                 test_println!("-> was notified; state={:?}", actual);
        |                 ---------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/cell.rs:161:9
        |
    161 |         test_println!("notifying; close={:?};", close);
        |         ---------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:189:9
        |
    189 |         test_println!("WaitQueue::start_wait({:p})", node);
        |         -------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:209:9
        |
    209 |         test_println!("WaitQueue::start_wait_slow({:p})", node);
        |         ------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:302:9
        |
    302 |         test_println!("WaitQueue::continue_wait({:p})", node);
        |         ----------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:327:9
        |
    327 |         test_println!("WaitQueue::continue_wait_slow({:p})", node);
        |         ---------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:371:9
        |
    371 |         test_println!("WaitQueue::notify()");
        |         ------------------------------------ in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:399:9
        |
    399 |         test_println!("WaitQueue::notify_slow(state: {})", state);
        |         --------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:444:9
        |
    444 |         test_println!("WaitQueue::close()");
        |         ----------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:473:9
        |
    473 |         test_println!("Waiter::remove({:p})", self);
        |         ------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:549:9
        |
    549 |         test_println!("List::enqueue({:p})", waiter);
        |         -------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:578:9
        |
    578 |         test_println!("List::dequeue({:?}) -> {:p}", new_state, last);
        |         ------------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/wait/queue.rs:602:9
        |
    602 |         test_println!("List::remove({:p})", node);
        |         ----------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
      --> src/macros.rs:11:34
       |
    11 |                     crate::loom::thread::current().id(),
       |                                  ^^^^^^ could not find `thread` in `loom`
       |
      ::: src/wait.rs:79:9
       |
    79 |         test_println!("WAKING TASK {:?} (from {:?})", self, thread::current());
       |         ---------------------------------------------------------------------- in this macro invocation
       |
       = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: use of undeclared crate or module `thread`
      --> src/wait.rs:79:61
       |
    79 |         test_println!("WAKING TASK {:?} (from {:?})", self, thread::current());
       |                                                             ^^^^^^ use of undeclared crate or module `thread`
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:169:9
        |
    169 |         test_println!("Core::close");
        |         ---------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: use of undeclared crate or module `std`
       --> src/lib.rs:170:12
        |
    170 |         if std::thread::panicking() {
        |            ^^^ use of undeclared crate or module `std`
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:185:9
        |
    185 |         test_println!("push_ref");
        |         ------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:224:25
        |
    224 |                         test_println!("claimed tail slot [{}]", idx);
        |                         -------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:235:33
        |
    235 | ...                   test_println!("-> initialized");
        |                       ------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:240:33
        |
    240 | ...                   test_println!("-> recycled");
        |                       ---------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:270:21
        |
    270 |                     test_println!("channel full");
        |                     ----------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:285:9
        |
    285 |         test_println!("pop_ref");
        |         ------------------------ in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:322:25
        |
    322 |                         test_println!("claimed head slot [{}]", idx);
        |                         -------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:352:25
        |
    352 |                         test_println!("--> channel full!");
        |                         ---------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0433]: failed to resolve: could not find `thread` in `loom`
       --> src/macros.rs:11:34
        |
    11  |                     crate::loom::thread::current().id(),
        |                                  ^^^^^^ could not find `thread` in `loom`
        |
       ::: src/lib.rs:478:9
        |
    478 |         test_println!("drop Ref<{}>", core::any::type_name::<T>());
        |         ---------------------------------------------------------- in this macro invocation
        |
        = note: this error originates in the macro `test_println` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    error[E0412]: cannot find type `Box` in this scope
      --> src/mpsc/async_impl.rs:72:16
       |
    72 |         slots: Box<[Slot<T>]>,
       |                ^^^ not found in this scope
       |
    help: consider importing this struct
       |
    1  | use alloc::boxed::Box;
       |
    
    error[E0412]: cannot find type `Box` in this scope
       --> src/lib.rs:519:56
        |
    519 |     pub(crate) fn make_boxed_array(capacity: usize) -> Box<[Self]> {
        |                                                        ^^^ not found in this scope
        |
    help: consider importing this struct
        |
    5   | use alloc::boxed::Box;
        |
    
    error[E0392]: parameter `T` is never used
      --> src/mpsc/async_impl.rs:57:25
       |
    57 |     pub struct Receiver<T, R = recycling::DefaultRecycle> {
       |                         ^ unused parameter
       |
       = help: consider removing `T`, referring to it in a field, or using a marker such as `PhantomData`
       = help: if you intended `T` to be a const parameter, use `const T: usize` instead
    
    error[E0392]: parameter `T` is never used
      --> src/mpsc/async_impl.rs:66:23
       |
    66 |     pub struct Sender<T, R = recycling::DefaultRecycle> {
       |                       ^ unused parameter
       |
       = help: consider removing `T`, referring to it in a field, or using a marker such as `PhantomData`
       = help: if you intended `T` to be a const parameter, use `const T: usize` instead
    
    Some errors have detailed explanations: E0392, E0412, E0433.
    For more information about an error, try `rustc --explain E0392`.
    error: could not compile `thingbuf` due to 85 previous errors
    

    At a glance:

    • There are printlns gated behind if cfg!(test) || cfg!(all(thingbuf_trace, feature = "std")) instead of #[cfg(any(test, all(thingbuf_trace, feature = "std")))].
    • Core::close references std::thread::panicking()
    • Box is used without being imported from alloc::boxed::Box.
    opened by ccbrown 2
  • wait queue is quite bad

    wait queue is quite bad

    the current wait queue implementation isn't very good --- it's essentially a spinlock around a Vec. this is similar to what crossbeam uses, but worse :)

    the downside of this approach is that when there's a large number of senders, there's severe tail latency because the vec has to be reallocated. also, memory usage doesn't decrease when the queue drains.

    note this violin plot: when there are a lot of waiters, the tail latency is quite bad: image

    opened by hawkw 2
  • Confusing documentation around single/multi consumer

    Confusing documentation around single/multi consumer

    • ThingBuf documentation - "A fixed-size, lock-free, multi-producer multi-consumer (MPMC) queue."
    • The original algorithm is MPMC
    • Readme description: "I’m at the buffer pool. I’m at the MPSC channel. I’m at the combination MPSC channel and buffer pool."
    • Loom models are called "mpsc_xxx"
    opened by kvark 1
  • chore: add test for mpsc::try_recv_ref, enable some loom tests

    chore: add test for mpsc::try_recv_ref, enable some loom tests

    • add a simple test for try_recv_ref
    • changes mpsc_sync in loom.yml to mpsc_blocking, make CI works
    • also enable some tests blocked by tokio-rs/loom#246
    opened by name1e5s 1
  • `thingbuf` can't compile on macos

    `thingbuf` can't compile on macos

    I think this is definitely a problem with the std implementation, so i asked in the zulip lib channel, but there is probably a way around this with AssertUnwindSafe:

    2022-03-16 8:29:56 I > cargo +1.56.0 build
       Compiling testing v0.1.0 (/Users/gus/repos/testing)
    error[E0277]: the type `UnsafeCell<libc::unix::bsd::apple::pthread_cond_t>` may contain interior mutability and a reference may not be safely transferrable across a catch_unwind boundary
     --> src/main.rs:6:5
      |
    6 |     safe(std::thread::current());
      |     ^^^^ `UnsafeCell<libc::unix::bsd::apple::pthread_cond_t>` may contain interior mutability and a reference may not be safely transferrable across a catch_unwind boundary
      |
      = help: within `std::thread::Inner`, the trait `RefUnwindSafe` is not implemented for `UnsafeCell<libc::unix::bsd::apple::pthread_cond_t>`
      = note: required because it appears within the type `std::sys::unix::condvar::Condvar`
      = note: required because it appears within the type `*const std::sys::unix::condvar::Condvar`
      = note: required because it appears within the type `Unique<std::sys::unix::condvar::Condvar>`
      = note: required because it appears within the type `Box<std::sys::unix::condvar::Condvar>`
      = note: required because it appears within the type `std::sys_common::condvar::Condvar`
      = note: required because it appears within the type `Condvar`
      = note: required because it appears within the type `std::sys_common::thread_parker::generic::Parker`
      = note: required because it appears within the type `std::thread::Inner`
      = note: required because of the requirements on the impl of `UnwindSafe` for `Arc<std::thread::Inner>`
      = note: required because it appears within the type `Thread`
    note: required by a bound in `safe`
     --> src/main.rs:1:12
      |
    1 | fn safe<U: std::panic::UnwindSafe>(u: U) {}
      |            ^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `safe`
    
    For more information about this error, try `rustc --explain E0277`.
    error: could not compile `testing` due to previous error
    
    opened by guswynn 1
  • feat(mpsc): add support for statically-allocated MPSC channels

    feat(mpsc): add support for statically-allocated MPSC channels

    This branch adds new StaticChannel types that can be used to construct statically allocated MPSC channel variants. The static channels can be used without requiring any heap allocations. This is intended primarily for embedded systems and bare metal programming where allocators may not be available or heap memory is constrained.

    Closes #17

    opened by hawkw 1
  • perf(mspc): replace bad VecDeque wait queue with intrusive list

    perf(mspc): replace bad VecDeque wait queue with intrusive list

    The new wait queue implementation doesn't allocate, and doesn't require resizing a VecDeque inside a lock. This should improve performance and (hopefully) make it possible to use the MPSC queue without any allocations on no-std (once I figure out the static stuff).

    Performance for blocking/yielding send is now very competitive after switching to the new wait queue, compare the new violin plot of the integer MPSC benchmark with violin plot with the old wait queue that i posted in #14.

    Old'n'busted: image

    New hotness: image

    Big comparison benchmark: image

    opened by hawkw 1
  • chore: prepare to release v0.1.3

    chore: prepare to release v0.1.3

    v0.1.3 (2022-05-13)

    Features

    • mpsc: add Receiver::try_recv method (#60) (e04661fc)

    Bug Fixes

    • mpsc: try_recv_ref should return RecvRef (#61) (47f16f59)
    opened by hawkw 0
  • fix: compilation errors with `--no-default-features`

    fix: compilation errors with `--no-default-features`

    This fixes compilation for no-std targets. My bad --- I was checking this manually and never added a CI job for it.

    This branch adds a CI check that the crate compiles with --no-default-features, and fixes a number of issues that broke the no-std build:

    • f8d9527 fix(no_std): fix warning in resume_unwind
    • cbf943a fix(no_std): fix compilation error in unreachable_unchecked
    • 8ae178a fix(no_std): remove use of thread::panicking
    • dab0d49 fix(alloc): fix missing Box imports
    • 208defc fix(no_std): wrong feature gate for println macros

    Fixes #58

    opened by hawkw 0
  • Using in tracing?

    Using in tracing?

    Hello, potentially a newbie question. You mentioned in the FAQ that this was intended for tracing Subscribers: how would you resolve the case where the buffer is full? Would this involve blocking until it can be sent? AFAIK the Subscriber API is entirely synchronous, so I assume you can't async send. I'm curious because tracing-forest currently uses a Tokio unbounded queue, and I'm curious about alternatives.

    opened by QnnOkabayashi 0
  • chore(ci): remove wrong test job that never ran

    chore(ci): remove wrong test job that never ran

    Due to a typo, we were accidentally trying to run tests on the operating system "m". Since GitHub Actions doesn't have runners running the "m" operating system (because, as far as I know, it doesn't actually 'exist'), this test job would never actually run.

    I don't know how this ended up getting merged to main, I really thought I fixed it on the branch? But, whatever...

    Signed-off-by: Eliza Weisman [email protected]

    opened by hawkw 0
  • Adding `remaining()` and `capacity()` methods to `Sender` and `Receiver`

    Adding `remaining()` and `capacity()` methods to `Sender` and `Receiver`

    Hello! This crate is very neat. It would be very helpful for metrics if mpsc::Sender and mpsc::Receiver channel handles exposed a way to determine the capacity and remaining slots. The ThingBuf queue has a remaining() and a capacity() method, so it seems odd that Sender or Receiver don't. I'm open to writing up a PR to add this, but I noticed that the implementation of Sender and Receiver was a bit different than ThingBuf -- is there any reason these weren't exposed? From a quick glance it seems like they would be easy to add to Sender and Receiver by just using the len() and capacity() methods from self.inner.core.core

    enhancement good first issue 
    opened by jcrevier 1
  • Cancelling existing `SendRef`

    Cancelling existing `SendRef`

    Wonder if allowing to cancel existing SendRef would be possible? E.g. something along the lines of

    impl<'a, ...> SendRef<'a, ...> {
        /// Drops this send ref *without* sending the data. The slot is returned to the pool.
        pub fn cancel(self) {
            ...
        }
    }
    

    Here's a particular sample use case:

    • We're reading network messages from multiple threads into a single thingbuf, trying to avoid all allocations and copying if possible (and recycling really helps with that).
    • Reading those messages is a complicated process - they don't arrive as &[u8] but have to be combined and processed in a streaming fashion, so each message is a BufRead reader.
    • In order to avoid copying, we reserve a SendRef slot and .read_to_end() directly into it.
    • However, sometimes it turns out that those messages have to be filtered out and should never reach the thingbuf receiver...
    • ... but alas, it's too late. As soon as SendRef is dropped, it will be sent away.
    • We can either use some hacky convention like sending empty messages, or some enums, or maybe do reading into a separate buffer first - all of these will work as workarounds, but the cleanest way would be to able to send_ref.cancel() which would cancel the reservation and return the slot to the pool - hence this question.

    Thanks!

    enhancement 
    opened by aldanor 2
  • static mpsc should be able to hand out senders after splitting

    static mpsc should be able to hand out senders after splitting

    it would be nice to have a

    pub fn sender(&'static self) -> Option<StaticSender<T, R>>
    

    on the StaticChannel types. The split method can only be called once because it creates the receiver, and this is a single-receiver channel, but the channel type should be able to create multiple senders, rather than just returning a StaticSender that can be Cloned.

    the method would need to return an Option (or Result?) though, because it shouldn't be able to make a new sender once the channel has closed (if either the rx was dropped or all the txs have been dropped).

    enhancement 
    opened by hawkw 0
  • mpsc: add owned refs

    mpsc: add owned refs

    Currently, the SendRef and RecvRef types returned by thingbuf::mpsc's senders and receivers contain a borrowed reference to the channel's queue. This means that a Ref from a channel cannot live for the 'static lifetime. However, because the channel's queue is either internally reference counted (in the case of the dynamically allocated MPSCs) or lives in a static variable (in the case of the static MPSCs), it should also be possible to create a reference to a queue slot that is 'static.

    We could add an API like tokio::sync::mpsc's Sender::reserve_owned, which clones the internal Arc around the channel's shared data and returns a permit whose lifetime is not bound to a borrow of the Sender handle that produced it.

    This could potentially enable some neat stuff, like using a thingbuf mpsc::Receiver of Vec<u8>s as a http_body::Body without having to copy data out of the thingbuf --- in this case, an OwnedRecvRef type could be passed directly into hyper or another HTTP library, and when the library finishes sending the body chunk, that ref is dropped, returning the buffer to the pool.

    enhancement 
    opened by hawkw 1
  • add async <--> blocking

    add async <--> blocking "bridge" channels

    In some cases, it's desirable to have a channel where one end of the channel is async, and another end is blocking. For example, we might want to send messages to an async task from a context where we can't yield, or we might want to send messages from an async task to a dedicated background thread that can wait by blocking.

    It should be quite easy to add support for async <-> blocking communication due to the overall modular design of thingbuf's MPSC channels. We could add something like this:

    enum EitherWaiter {
       Async(core::task::Waker),
       Blocking(std::thread::Thread),
    }
    
    impl Notify for EitherWaiter {
        fn notify(self) {
            match self {
                Self::Async(waker), => waker.notify(),
                Self::Blocking(thread) => thread.notify(),
            }
        }
    
        #[inline]
        fn same(&self, other: &Self) -> bool {
            match (self, other) {
                (Self::Async(ref this), Self::Async(ref that)), => that.will_wake(this),
                (Self::Blocking(ref this), Self::Blocking(ref that)) => this.id() == that.id(),
                 _ => false,
            }
        }
    }
    

    and now, suddenly, a WaitQueue or WaitCell can contain both async Wakers and blocking Threads!

    Design Questions

    The main design question is whether this should be added to the existing channel types, or whether we should add a new channel type specifically for asynchronous <--> blocking communication. As I see it, this is roughly the tradeoff:

    Pros of adding bridging to the existing channels

    • The API would be much simpler. Currently we have separate async Sender/Receiver, async StaticSender/StaticReceiver, synchronous sync::Sender/sync::Receiver, and synchronous sync::StaticSender/sync::StaticReceiver...which is a lot of types. Adding a third bridge channel type would introduce a new sender, new receiver, and (presumably) also static variants, for four new types.

      On the other hand, if we decided to combine everything, we could even get rid of the separate synchronous channel types, and just have one channel with a receiver with async fn recv() and fn recv_blocking(), and a sender with async fn send() and fn send_blocking(). This would simplify the API significantly.

    • Fewer API types also means less code duplication. Adding a separate bridging channel would probably require duplicating some code from both the existing async and sync channel implementations.

    • More flexible user code. If users want to write code that (for example) takes a receiver capable of doing an async recv operation, but the senders might be sync or async, they would have to wrap the separate Receiver and BridgeReceiver types or be generic over them. If we just combined everything into one channel type, this wouldn't be necessary.

    Cons of the combined API

    • Enum dispatch overhead. AFAIK, this is the one really big downside of combining everything: whenever you wake something, you have to match on the enum and call the appropriate notify method. This is a small amount of additional runtime work you have to do on every wakeup. I don't know how severe a performance impact this would have, but it's impossible for it to not have some ooverhead over not having to do it. It's possible that the overhead is so small that it doesn't meaningfully effect our benchmarks, though. We should test this. If there's a performance impact that's noticeable, we might want to have a separate bridge API so that you only pay that cost if you're using it.

    • Enum discriminants. Similarly, adding the enum introduces a bit of space overhead; at least a byte per waiter (possibly more depending on padding & alignment). I severely doubt that niche optimization can find anywhere in an enum of Waker/Thread` to put the discriminant, so it's almost certainly making waiters a bit bigger. I'm not sure how much this matters, but users might not want to pay the space cost if they're not using it.

    • Maybe people actually want to have a type-level distinction between the different kinds of channels? I'm not really sure why, though.

    enhancement 
    opened by hawkw 1
Releases(v0.1.3)
  • v0.1.3(May 13, 2022)

  • v0.1.2(Apr 12, 2022)

  • v0.1.1(Mar 16, 2022)

  • v0.1.0(Mar 15, 2022)

    Initial release!

    Performance

    • mpsc: rewrite and optimize wait queue (#22) (8c882b0f)
    • mspc: replace bad VecDeque wait queue with intrusive list (#16) (23f4c96f)

    Features

    • add #[must_use] to constructors (#45) (0299a606)
    • add into_inner and Error impl to Full (#43) (527a6398)
    • add Deref and DerefMut impls to Ref types (#13) (6ebfe7b8, breaks #)
    • add nicer fmt::Debug impls (#4) (38cbad20)
    • StringBuf: add StringBuf type (856c1f6c)
    • ThingBuf: add pop_with and push_with (9192c603)
    • mpsc:
      • stick errors in their own module (3137b85e)
      • add std::error::Error impls (d5ac083b)
      • add methods to errors (d5bf3db0)
      • add support for statically-allocated MPSC channels (#23) (5b17c184, closes #17)
      • add waiting send/send_ref (#7) (76df064c)
      • make errors more like other mpscs (#5) (5e749ccc)
      • initial sync and async channel APIs (#2) (1c28c84f)
    • recycling: add customizable recycling policies (#33) (54e53534, closes #30)
    • thingbuf:
      • add no_std compatible StaticThingBuf (#1) (3b23f858)
      • hahahaha static storage works (e47cd7dc)

    Breaking Changes

    • add Deref and DerefMut impls to Ref types (#13) (6ebfe7b8, breaks #)

    Bug Fixes

    • ThingBuf:
      • fix backwards subtraction in len (caab6b23)
      • fix wrong increment in pop (0e53279c)
    • mpsc:
      • ensure un-received messages are dropped (#29) (c444e50b)
      • fix a deadlock in async send_ref (#20) (c58c6200)
    Source code(tar.gz)
    Source code(zip)
Owner
Eliza Weisman
systems witch
Eliza Weisman
A typemap for a set of known types optionally without heap allocation, and supporting iterating by traits

fixed_typemap docs.rs GitHub Sponsors Implements typemaps that support a lot of extra funcctionality using procedural macros. docs.rs has a lot more t

Austin Hicks 2 Dec 27, 2021
Unify your game sources in one place and aquire more of them, using modules made by the community.

Project Black Pearl Unify your game sources in one place by using modules made by the community. What is Project Black Pearl? Project Black Pearl (or

Project Black Pearl 8 Jan 15, 2023
Leetcode Solutions in Rust, Advent of Code Solutions in Rust and more

RUST GYM Rust Solutions Leetcode Solutions in Rust AdventOfCode Solutions in Rust This project demostrates how to create Data Structures and to implem

Larry Fantasy 635 Jan 3, 2023
Simple autoclicker written in Rust, to learn the Rust language.

RClicker is an autoclicker written in Rust, written to learn more about the Rust programming language. RClicker was was written by me to learn more ab

null 7 Nov 15, 2022
Rust programs written entirely in Rust

mustang Programs written entirely in Rust Mustang is a system for building programs built entirely in Rust, meaning they do not depend on any part of

Dan Gohman 561 Dec 26, 2022
Rust 核心库和标准库的源码级中文翻译,可作为 IDE 工具的智能提示 (Rust core library and standard library translation. can be used as IntelliSense for IDE tools)

Rust 标准库中文版 这是翻译 Rust 库 的地方, 相关源代码来自于 https://github.com/rust-lang/rust。 如果您不会说英语,那么拥有使用中文的文档至关重要,即使您会说英语,使用母语也仍然能让您感到愉快。Rust 标准库是高质量的,不管是新手还是老手,都可以从中

wtklbm 493 Jan 4, 2023
A library for extracting #[no_mangle] pub extern "C" functions (https://docs.rust-embedded.org/book/interoperability/rust-with-c.html#no_mangle)

A library for extracting #[no_mangle] pub extern "C" functions In order to expose a function with C binary interface for interoperability with other p

Dmitrii - Demenev 0 Feb 17, 2022
clone of grep cli written in Rust. From Chapter 12 of the Rust Programming Language book

minigrep is a clone of the grep cli in rust Minigrep will find a query string in a file. To test it out, clone the project and run cargo run body poem

Raunak Singh 1 Dec 14, 2021
Rust-blog - Educational blog posts for Rust beginners

pretzelhammer's Rust blog ?? I write educational content for Rust beginners and Rust advanced beginners. My posts are listed below in reverse chronolo

kirill 5.2k Jan 1, 2023
The ray tracer challenge in rust - Repository to follow my development of "The Raytracer Challenge" book by Jamis Buck in the language Rust

The Ray Tracer Challenge This repository contains all the code written, while step by implementing Ray Tracer, based on the book "The Ray Tracer Chall

Jakob Westhoff 54 Dec 25, 2022
Learn-rust-the-hard-way - "Learn C The Hard Way" by Zed Shaw Converted to Rust

Learn Rust The Hard Way This is an implementation of Zed Shaw's Learn X The Hard Way for the Rust Programming Language. Installing Rust TODO: Instruct

Ryan Levick 309 Dec 8, 2022
Learn to write Rust procedural macros [Rust Latam conference, Montevideo Uruguay, March 2019]

Rust Latam: procedural macros workshop This repo contains a selection of projects designed to learn to write Rust procedural macros — Rust code that g

David Tolnay 2.5k Dec 29, 2022
The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language.

rcc The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language. Compilers Language Co

null 2 Jan 17, 2022
Integra8 rust integration test framework Rust with a focus on productivity, extensibility, and speed.

integra8 Integra8 rust integration test framework Rust with a focus on productivity, extensibility, and speed. | This repo is in a "work in progress"

exceptional 3 Sep 26, 2022
Neofetch but in Rust (rust-toml-fetch)

rtfetch Configuration Recompile each time you change the config file logo = "arch.logo" # in src/assets. info = [ "", "", "<yellow>{host_n

Paolo Bettelini 6 Jun 6, 2022
Rust Sandbox [code for 15 concepts of Rust language]

Rust-Programming-Tutorial Rust Sandbox [code for 15 concepts of Rust language]. The first time I've been introduced to Rust was on January 2022, you m

Bek Brace 4 Aug 30, 2022
TypeRust - simple Rust playground where you can build or run your Rust code and share it with others

Rust playground Welcome to TypeRust! This is a simple Rust playground where you can build or run your Rust code and share it with others. There are a

Kirill Vasiltsov 28 Dec 12, 2022
Rust Imaging Library: A high-level Rust imaging crate.

ril Rust Imaging Library: A performant and high-level Rust imaging crate. Documentation • Crates.io • Discord What's this? This is a Rust crate design

Jay3332 18 Jan 5, 2023
In this repository you can find modules with code and comments that explain rust syntax and all about Rust lang.

Learn Rust What is this? In this repository you can find modules with code and comments that explain rust syntax and all about Rust lang. This is usef

Domagoj Ratko 5 Nov 5, 2022