Measuring the execution speed of unit tests to track performance of cog3pio Python functions over time.

Using pytest-codspeed to do the benchmarking, with the help of https://github.com/CodSpeedHQ/action. Decorated test_read_geotiff with @pytest.mark.benchmark to see if the benchmarking works.

Note: Running the benchmarks on Python 3.12 to enable flame graph generation, available with pytest-codspeed>=2.0.0 - see https://docs.codspeed.io/features/trace-generation.

References:

• https://codspeed.io/blog/introducing-codspeed
• https://docs.codspeed.io/benchmarks/python#running-the-benchmarks-in-your-ci

Enable GeoTIFFs stored on remote urls (e.g. http, s3, azure, gcp, etc) to be read into numpy arrays.

Uses the object_store crate, which pulls data via an async API. Note that only the network read is async, the TIFF decoding (using the tiff crate) is still synchronous.

Usage (in Python):

import numpy as np
from cog3pio import read_geotiff

array: np.ndarray = read_geotiff(
    path="https://github.com/pka/georaster/raw/v0.1.0/data/tiff/float32.tif"
)
assert array.shape == (20, 20)
assert array.dtype == "float32"

TODO:

• [x] Add failing unit test reading a GeoTIFF via HTTP
• [x] Use object_store's parse_url function to handle remote URL reads
• [x] Ensure that local relative filepaths (e.g. data/float32.tif) work alongside remote URLs (e.g. https://somewhere.com/float32.tif)
• [x] Refactor to remove .unwrap() statements
• [x] Document in main README.md on new usage

Other things done in this PR out of necessity:

• Move to compiling on manylinux_2_28, therefore dropping compilation on i686 (x86) targets on Linux

References:

• Similar PR to read FlatGeobuf via object_store at https://github.com/geoarrow/geoarrow-rs/pull/494

Rust-based function for reading GeoTIFF files!

Uses the tiff crate for the I/O, ndarray crate to store the 2D array in Rust, and numpy crate to convert to numpy.ndarray in Python.

Note: This only works on single-band GeoTIFF files with float32 dtype for now

Usage:

In Rust:

use ndarray::Array2;
use std::fs::File;
use cog3pio::io::geotiff::read_geotiff;

let path: &str = "path/to/file.tif";
let file: File = File::open(path).expect("Cannot find GeoTIFF file");
let arr: Array2<f32> = read_geotiff(file).unwrap();
assert_eq!(arr.dim(), (20, 20));
assert_eq!(arr.mean(), Some(19.0));

In Python:

import numpy as np
from cog3pio import read_geotiff

array: np.ndarray = read_geotiff(path="georaster/data/tiff/float32.tif")
assert array.shape == (20, 20)
assert array.dtype == "float32"

TODO:

• [x] Initial implementation to allow reading single-band Float32 dtype GeoTIFF files
• [x] Improve documentation
• [x] Add unit tests in Python and Rust

TODO in future:

• [ ] Refactor to handle other dtypes
• [ ] Reconsider project layout?

References:

• https://stackoverflow.com/questions/76145217/how-to-transform-a-tiff-file-to-ndarray-in-rust
• https://en.anyquestion.info/q/how-to-transform-a-tiff-file-to-ndarray-in-rust
• https://github.com/image-rs/image-tiff/pull/216/files

Minimal configuration to run unit tests. Using pytest. Using GitHub Actions CI.

Created using maturin generate-ci --pytest github, with additional modifications. Also set pyo3's abi3-py310 feature.

Matrix builds are on:

• Linux (ubuntu-22.04) - targets: [x86_64, x86, aarch64, armv7, s390x, ppc64le]
• Windows (windows-2022) - targets: [x64, x86]
• macOS (macos-14) - targets: [x86_64, aarch64]

References:

• https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
• https://pyo3.rs/v0.20.3/features.html#the-abi3-pyxy-features

Created using maturin init --mixed following https://github.com/PyO3/maturin/blob/v1.4.0/guide/src/project_layout.md#mixed-rustpython-project, with some custom modifications.

TODO:

• [x] Include pyo3 crate as a dependency, set minimum Python version to 3.10 as per SPEC 0.
• [x] Add trove classifiers
• [x] Add .gitignore file
• [x] Repo description, roadmap and related crates