Introduces feature static which builds both hdf5 and netcdf from source. Adds CI for conda and static builds on the three major platforms.

Closes #29 Closes #46

@milesgranger suggested in #22 that we create an org for this repo or find another org willing to take ownership of it. I'm in support of this - my time is limited and I haven't been keeping up with Rust since I wrote the original version of this crate in 2015.

Any suggestions or opinions on the details here?

• Where exactly should we move it?
• Who wants to take on a maintainer role?

The global mutex is in no way ideal when reading/writing from multiple threads, and could be split into several (global/per file), or replaced by RWlocks.

This requires investigation into where netcdf does something thread unsafe, and limit the locking to this part. Should also investigate where HDF5 might be problematic

One could also integrate bindings to https://github.com/Parallel-NetCDF/PnetCDF, but this limits the formats to CLASSIC

https://github.com/Unidata/netcdf-c/issues/1373

Test concurrent access to netcdf file using varying number of threads and varying number of open files. At the moment opening several files should not have any effect since there is a library-global mutex.

This introduces a slightly larger test file (3.1 MB) retrieved from http://test.opendap.org/dap/data/nc/coads_climatology.nc

Run with:

$ cargo bench --tests conc

This PR makes the crate more ergonomic for users, and adds some missing functionality such as unlimited dimensions.

Removed:

• mutex from netcdf-sys
• direct access to internal structures

Changed:

• getters and setters for dimensions/variables/attributes
• funtions returns a netcdf::Result
• functions now take Option<&[usize]> for indexes/sizes
• ndarray is now feature gated
• tests produces files in a temporary directory

Added:

• can open from any Path-like item
• Deref and DerefMutfor the file makes adding and accessing variables/dimensions easier
• unlimited dimensions
• read from a memory slice (feature gated due to travis netcdf version)
• tests

Hello, I've been having this error thrown by rust-analyser since trying to use this crate. I'm not overly familiar with netcdf files so I'm not exactly sure what I need to install/ where to install it from. (honestly when I look up HDF5 I'm not sure which site to even open up haha)

The "crates.io" version of this package (0.1.0) is from 2015. And still the current version is 0.1.0

Maybe version in Cargo.toml should be increased? At least to 0.2.0? There are new methods such as 'Variable::values_at', etc. compared to version of 2015.

Hello,

I wasn't able to build netcdf-sys without renaming "netcdf_v4.3.3.1.h" to "netcdf.h", and without changing the gcc call in "netcdf-sys/build.rs" (to include "netcdf.h"). I've made a fork that I could compile, it does work but some units tests fail. I assume it's caused by my version of libnetcdf-dev. (1:4.1.3-7ubuntu2 amd64).

I'm both new to rust and to the netcdf file format :S, I really hope I'm not wasting your time. Best regards,

evomassiny

Hi,

I was wondering if you would be open to pull requests if I were to help write up some documentation. I figure it would be generated using rustdoc and be hosted using github-pages.

I'm new to Rust, but I thought I could at least do this much to help out. :smile:

Hello !

I'm trying the examples/ncdump.rs bin to read files generated from python by xarray.Dataset.to_netcdf : https://xarray.pydata.org/en/stable/generated/xarray.Dataset.to_netcdf.html

https://github.com/georust/netcdf/blob/ca0970edf8f28517319b0a08b010eba449c6ec94/examples/ncdump.rs#L71

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Netcdf(-45)', netcdf-0.5.2/src/variable.rs:138:56

The -45 being NC_EBADTYPE from https://github.com/georust/netcdf/blob/71b41fe/netcdf-sys/src/netcdf_const.rs#L109, I suppose.

I added a catch_unwind to get more details: it seems to fail on types coming from numpy: fixed-length sequence of chars (dtype('S1'), dtype('S4'), etc.)

Any idea on where to go from there ?

Thanks a lot !

Will close #13

Disclosure, I'm very new to interfacing Rust with C.

With that said, I've gotten as far as I can, and so far I see it, the test I've added should pass, but I get an error code 2 back from the nc_open_mem as suggested in the original issue. So I must be doing a silly thing here and there.

Willing to go further with a few hints. :+1:

Hello, thank you for the work on this project !

By any chance, is there a plan to add a CF time attribute reading/parsing to handle the datetime type ? I can't find anything on the docs nor on crates.io I think of something like Julia CFtime or Python CFtime

I believe it would be a great feature for the geophysical field

Look to hdf5-rust for how this could look like, where the user can specify arbtrary types and derive NcType for serialisation and deserialisation to netcdf files

When reading values from a variable, it should be possible to get a lazy-loading iterator over chunks.

Implementation details: new method: fn values_chunked(start, chunklength, &mut buffer) -> ChunkIterator { }

struct ChunkIterator { start: () buflen: () buffer: () }

Hi, I think both netcdf and hdf5 supports parallel reads. Glimpsing at the source of this project there seem to be some global locks. Is parallel netcdf supported?

I am interested in the raw xdr encoded data for a dap server. Is it possible to access the raw bytes through this library?