One of the requirements for bool is that it only has the values 0x00 and 0x01, so there is a possible safety issue around having bool implement SharedMemCast.

Unfortunately, the dependency i chose for file system locks seems to be broken on MacOS.

The race happens when process 1 creates a file that is supposed to contain the shared memory mapping unique ID but process 2 ends up reading the file before the ID is written by process 1...

I will have to either find a new file locking crate or be OK with not fully supporting MacOS for now.

Fixes #17.

Hello! Thanks for the great crate. I want to use this for a project and I noticed that you needed a custom derive for the SharedMemCast trait. This is really useful for using this crate without having to write the unsafe impl, so I thought I would help out and contribute the macro. :)

The size of this PR is probably quite daunting, so I'll give you an overview of what I had to do so you understand each step:

1. All procedural macros need to go in a separate crate that is compiled before shared_memory is compiled. I called this crate shared_memory_derive and added it to the root directory of the project.
2. In order to re-export the SharedMemCast proc macro from src/lib.rs and make using the macro convenient, I needed to upgrade the crate to the 2018 edition. I didn't want my changes to be too obtrusive, so I only added the edition = 2018 line in the Cargo.toml file and made the most minimal changes to fix any errors that cropped up. You can always use cargo fix to finish the upgrade later if you feel the need to do that.
3. I copied all the metadata from the root Cargo.toml file into the shared_memory_derive/Cargo.toml file. I made sure that the versions of both crates are in sync. This can be useful to avoid confusion. Please make sure you publish shared_memory_derive before publishing shared_memory to avoid any problems.
4. I then went and created a comprehensive set of tests to make sure we never accidentally let through anything that should not be casted from shared memory. I'm using the trybuild crate for that. The tests in tests/ui are all meant to produce compiler errors and the tests in tests/run-pass are all meant to compile with no errors. See the CONTRIBUTING.md file I added for more details.
   • [ ] You should go through my tests and make sure all the cases I've allowed to pass are valid
   • [ ] Can you confirm whether or not zero-sized types are allowed to be cast from shared memory? (e.g. PhantomData<T>, (), [T; 0], unit structs, etc.) I've disabled them (by producing a compiler error) because it made the most sense to me that they should not ever be casted from shared memory. The same goes for empty enums which should never be initialized.
5. In order for these tests to pass, we need the SharedMemCast trait to be implemented for various sizes of fixed-size array. Rust has no type-level integer support just yet, so the solution that every crate uses is to have a bunch of impls for different sizes. I think we can cover a large number of use-cases by having impls for all sizes between 1 and 32, then the powers of 2 up to about 2 GB. See cast.rs for what I mean by that. I think these impls should be more than enough for most peoples' needs and we can always add more later.
6. Finally, after all these tests and impls were added, I implemented the actual custom derive code. This code actually isn't terribly complicated. I have attempted to comment it quite well so you understand the approach and the implementation. Please let me know if you have any questions.
7. After implementing that and getting the tests to pass, I updated the example in the examples/ directory to use #[derive] instead of unsafe impl. I also updated the docs to let people know that this feature exists. See below for a screenshot of that.
8. I modified the Travis CI script so that it uses the --all flag. This should ensure that the custom derive tests also run on CI.

These steps can be roughly seen in my commits as well if you prefer to go through things that way. Please don't hesitate to ask me any questions about all of this and do let me know if I've missed describing any part of what I did.

Thanks again for the great library! Looking forward to using it once these changes land and everything is published. :smile:

Documentation Screenshot

Only the docs for SharedMemCast were updated:

Сhanges

This fork has a feature rs2 that allows you to safely create shared memory through file locks. Example of common to all processes code:

SharedMem::open_linked(file_name)
    .or_else(|_|SharedMem::create_linked(file_name, LockType::RwLock, 4096))
    .or_else(|_|SharedMem::open_linked(file_name))
    .ok()

I was able to run the example where shared data is of primitive type e.g. u8 or usize. but when i tried to run the logic with struct or vec, the read side always returned segmentation fault:

write side:

{ let data_ptr = mutex.lock().unwrap(); let data = unsafe { &mut *(*data_ptr as *mut Vec) };

    //let sd : Entry = Entry { hashcode: 1, buff: vec![1, 2, 3, 4, 5]};
    let sd : Entry = Entry { hashcode: 1};
    serialize_into(data, &sd).unwrap();
}

read side:

{ let data_ptr = mutex.rlock().unwrap(); let data = unsafe { & *(*data_ptr as *const Vec) };

    eprintln!("{}", data[1]);

    match deserialize::<Entry>(data)
    {
        Ok(d) => eprint!("Data : {:?}", d),
        Err(e) =>  eprint!("Error deserializing data {}", e)
    }
}

any idea what is going on?

I am not able to use shared_memory lib and systemd_journal_logger lib together when I run binary file without debug symbols

For example

`use log::LevelFilter;

fn main() {
  systemd_journal_logger::init().unwrap();

  log::set_max_level( LevelFilter::Debug);

  log::warn!("A warning.");
  log::error!("An error");
}    `

And this dependencies

`[dependencies]
serde =  { version = "1.0", features = ["derive"] }
serde_json = "1.0"
log = "=0.4.14"
systemd-journal-logger = "=0.3.1"

everything works perfectly when running ./target/release/xyz

but if I include shared_memory It stops working

I checked in sm source code that implements a log crate, why dont use standard log api implementation ?| thanks

Cargo version: 1.53.0 (also tried with 1.55.0-nightly) Rustc version: 1.53.0 (also tried with 1.55.0-nightly) Crate version: 0.11.4

When it compile the crate it throw these errors:

error[E0164]: expected tuple struct or tuple variant, found associated function `nix::Error::Sys`
  --> /home/zacharie/.cargo/registry/src/github.com-1ecc6299db9ec823/shared_memory-0.11.4/src/unix.rs:87:13
   |
87 |         Err(nix::Error::Sys(Errno::EEXIST)) => return Err(ShmemError::MappingIdExists),
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `fn` calls are not allowed in patterns
   |
   = help: for more information, visit https://doc.rust-lang.org/book/ch18-00-patterns.html

error[E0164]: expected tuple struct or tuple variant, found associated function `nix::Error::Sys`
  --> /home/zacharie/.cargo/registry/src/github.com-1ecc6299db9ec823/shared_memory-0.11.4/src/unix.rs:88:13
   |
88 |         Err(nix::Error::Sys(e)) => return Err(ShmemError::MapCreateFailed(e as u32)),
   |             ^^^^^^^^^^^^^^^^^^ `fn` calls are not allowed in patterns
   |
   = help: for more information, visit https://doc.rust-lang.org/book/ch18-00-patterns.html

error[E0164]: expected tuple struct or tuple variant, found associated function `nix::Error::Sys`
   --> /home/zacharie/.cargo/registry/src/github.com-1ecc6299db9ec823/shared_memory-0.11.4/src/unix.rs:103:13
    |
103 |         Err(nix::Error::Sys(e)) => return Err(ShmemError::UnknownOsError(e as u32)),
    |             ^^^^^^^^^^^^^^^^^^ `fn` calls are not allowed in patterns
    |
    = help: for more information, visit https://doc.rust-lang.org/book/ch18-00-patterns.html

error[E0164]: expected tuple struct or tuple variant, found associated function `nix::Error::Sys`
   --> /home/zacharie/.cargo/registry/src/github.com-1ecc6299db9ec823/shared_memory-0.11.4/src/unix.rs:119:13
    |
119 |         Err(nix::Error::Sys(e)) => return Err(ShmemError::MapCreateFailed(e as u32)),
    |             ^^^^^^^^^^^^^^^^^^ `fn` calls are not allowed in patterns
    |
    = help: for more information, visit https://doc.rust-lang.org/book/ch18-00-patterns.html

error[E0164]: expected tuple struct or tuple variant, found associated function `nix::Error::Sys`
   --> /home/zacharie/.cargo/registry/src/github.com-1ecc6299db9ec823/shared_memory-0.11.4/src/unix.rs:135:13
    |
135 |         Err(nix::Error::Sys(e)) => return Err(ShmemError::MapOpenFailed(e as u32)),
    |             ^^^^^^^^^^^^^^^^^^ `fn` calls are not allowed in patterns
    |
    = help: for more information, visit https://doc.rust-lang.org/book/ch18-00-patterns.html

error[E0164]: expected tuple struct or tuple variant, found associated function `nix::Error::Sys`
   --> /home/zacharie/.cargo/registry/src/github.com-1ecc6299db9ec823/shared_memory-0.11.4/src/unix.rs:150:13
    |
150 |         Err(nix::Error::Sys(e)) => return Err(ShmemError::MapOpenFailed(e as u32)),
    |             ^^^^^^^^^^^^^^^^^^ `fn` calls are not allowed in patterns
    |
    = help: for more information, visit https://doc.rust-lang.org/book/ch18-00-patterns.html

error[E0164]: expected tuple struct or tuple variant, found associated function `nix::Error::Sys`
   --> /home/zacharie/.cargo/registry/src/github.com-1ecc6299db9ec823/shared_memory-0.11.4/src/unix.rs:166:13
    |
166 |         Err(nix::Error::Sys(e)) => return Err(ShmemError::MapOpenFailed(e as u32)),
    |             ^^^^^^^^^^^^^^^^^^ `fn` calls are not allowed in patterns
    |
    = help: for more information, visit https://doc.rust-lang.org/book/ch18-00-patterns.html

error: aborting due to 7 previous errors

Shmem currently behaves differently on Windows than Unix. Specifically, Shmem::set_owner has no effect on the underlying memory object on Windows, as the standard Windows shared file mapping doesn't allow persistence. This means that dropping a unique Shmem will destroy the shared memory even when Shmem::is_owner returns false. This clashes with POSIX behaviour, which persists the memory unless shm_unlink() is called and allows reopening even when all handles have previously been closed.

This PR unifies shared memory behaviour on Windows and Unix and adds a test to check for compliance. It follows the C++ Boost implementation of creating a file to back the shared memory which is renamed and marked for deletion when an owning Shmem drops, but persists otherwise. Shmem::set_owner stops being a no-op on Windows and now behaves as expected.

when trying to compile shared_memory 0.11.1 on Windows 10 I get this error.

error[E0432]: unresolved imports `winapi::shared::winerror`, `winapi::um::handleapi`, `winapi::um::winbase`
  --> C:\Users\domen\.cargo\registry\src\github.com-1ecc6299db9ec823\shared_memory-0.11.1\src\windows.rs:4:9
   |
4  |         winerror::ERROR_ALREADY_EXISTS,
   |         ^^^^^^^^ could not find `winerror` in `shared`
...
8  |         handleapi::{CloseHandle, INVALID_HANDLE_VALUE},
   |         ^^^^^^^^^ could not find `handleapi` in `um`
9  |         memoryapi::{MapViewOfFile, UnmapViewOfFile, VirtualQuery, FILE_MAP_READ, FILE_MAP_WRITE},
10 |         winbase::{CreateFileMappingA, OpenFileMappingA},
   |         ^^^^^^^ could not find `winbase` in `um`

Other information: windows 10 (build 17763) rustup 1.21.1 (7832b2ebe 2019-12-20) cargo 1.45.0-nightly (cb06cb269 2020-05-08) rustc 1.45.0-nightly (a74d1862d 2020-05-14)

At the moment, the number and kinds of locks and events is fixed when creating the shared memory. There are APIs (like channels) where it would be useful to be able to create these dynamically, allocating them into the regular shared memory rather than the metadata. Is this doable?

As suggested by @llogiq , mem_file would probably benefit from the features provided by https://travis-ci.org/

I will have to set up compilation on the supported targets at a minimum and maybe try to see how testing could be implemented.

Testing might prove to be a bit tricky since mem_file is a library aimed for multi-process environments and it sounds like a headache to do automated testing for that. Issues that come to mind:

• Granted that you can launch multiple separate processes in a single cargo test, how do you validate: -Memory is actually shared -Locking mechanism work properly (the two process cant both hold a mutex for example) -Things are getting cleaned up/closed properly after a non-crashing/non-frozen run

• In case of error for any of these testcases, your testing framework might have to deal with crashes processes, frozen processes waiting forever on locks, etc.. on a per-OS basis.

Updates the requirements on nix to permit the latest version.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Updates the requirements on clap to permit the latest version.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.