Progmem utility for the AVR architecture

Overview

avr-progmem

Progmem utilities for the AVR architectures.

This crate provides unsafe utilities for working with data stored in the program memory of an AVR micro-controller. Additionally, it defines a 'best-effort' safe wrapper struct ProgMem to simplify working with it.

This crate is implemented only in Rust and some short assembly, it does NOT depend on the avr-libc or any other C-library. However, due to the use of inline assembly, this crate may only be compiled using a nightly Rust compiler.

AVR Memory

This crate is specifically for AVR-base micro-controllers such as the Arduino Uno (and some other Arduino boards, but not all), which have a modified Harvard architecture which implies the strict separation of program code and data while having special instructions to read and write to program memory.

While, of course, all ordinary data is stored in the data domain where it is perfectly usable, the harsh constraints of most AVR processors make it very appealing to use the program memory (also referred to as progmem) for storing constant values. However, due to the Harvard design, those values are not usable with normal instructions (i.e. those emitted from normal Rust code). Instead, special instructions are required to load data from the program code domain, i.e. the lpm (load from program memory) instruction. And because there is no way to emit it from Rust code, this crate uses inline assembly to emit that instruction.

However, since a pointer into program code cannot be differentiated from a normal data pointer, it is entirely up to the programmer to ensure that these different 'pointer-types' are not accidentally mixed. In other words, this is unsafe in the context of Rust.

Loading Data from Program Memory

The first part of this crate simply provides a few functions (e.g. read_byte) to load constant data (i.e. a Rust static that is immutable) from the program memory into the data domain, so that sub-sequentially it is normal usable data, i.e. as owned data on the stack.

Because, as aforementioned, a simple *const u8 in Rust does not specify whether is lives in the program code domain or the data domain, all functions which simply load a given pointer from the program memory are inherently unsafe.

Notice that using a &u8 reference might make things rather worse than safe. Because keeping a pointer/reference/address into the program memory as Rust reference might easily cause it to be dereferenced, even in safe code. But since that address is only valid in the program code domain (and Rust doesn't know about it) it would illegally load the address from the data memory, causing undefined behavior!

Example

use avr_progmem::read_byte;

// This `static` must never be directly dereferenced/accessed!
// So a `let data: u8 = P_BYTE;` is **undefined behavior**!!!
/// Static byte stored in progmem!
#[link_section = ".progmem.data"]
static P_BYTE: u8 = b'A';

// Load the byte from progmem
// Here, it is sound, because due to the link_section it is indeed in the
// program code memory.
let data: u8 = unsafe { read_byte(&P_BYTE) };
assert_eq!(b'A', data);

The best-effort Wrapper

Since working with progmem data is inherently unsafe and rather difficult to do correctly, this crate introduces the best-effort 'safe' wrapper ProgMem, that is supposed to only wrap data in progmem, thus offering only functions to load its content using the progmem loading function. The latter are fine and safe, given that the wrapper type really contains data in the program memory. Therefore, to keep that invariant up, the constructor is unsafe.

Yet to make that also easier, this crate provides the progmem! macro (it has to be a macro), which will create a static variable in program memory for you and wrap it in the ProgMem struct. It will ensure that the static will be stored in the program memory by defining the #[link_section = ".progmem.data"] attribute on it. This makes the load functions on that struct sound and additionally prevents users to accidentally access that static directly, which, since it is in progmem, would be fundamentally unsound.

Example

use avr_progmem::progmem;

// It will be wrapped in the ProgMem struct and expand to:
// ```
// #[link_section = ".progmem.data"]
// static P_BYTE: ProgMem<u8> = unsafe { ProgMem::new(b'A') };
// ```
// Thus it is impossible for safe Rust to directly dereference/access it!
progmem! {
    /// Static byte stored in progmem!
    static progmem P_BYTE: u8 = b'A';
}

// Load the byte from progmem
// It is still sound, because the `ProgMem` guarantees us that it comes
// from the program code memory.
let data: u8 = P_BYTE.load();
assert_eq!(b'A', data);

Other Architectures

As mentioned before, this crate is specifically designed to be use with AVR-base micro-controllers. But since most of us don't write their programs on an AVR system but e.g. on x86 systems, and might want to test them there (well as far as it is possible), this crate also has a fallback implementation for all other architectures that are not AVR, falling back to a simple Rust static in the default data segment. And all the data loading functions will just dereference the pointed-to data, assuming that they just live in the default location.

This fallback is perfectly safe on x86 and friend, and should also be fine on all further architectures, otherwise normal Rust statics would be broken. However, it is an important point to know when for instance writing a library that is not limited to AVR.

Implementation Limitations

Aside from what has been already been covered, the current implementation has two further limitations.

First, since this crate uses an inline assembly loop on a 8-bit architecture, the loop counter only allows values up to 255. This means that not more that 255 bytes can be loaded at once with any of the methods of this crate. However, this only applies to a single continuous load operation, so for instance ProgMem<[u8;1024]>::load() will panic, but accessing such a big type in smaller chunks e.g. ProgMem<[u8;1024]>::load_sub_array::<[u8;128]>(512) is perfectly fine because the to be loaded type [u8;128] is only 128 bytes in size.

Second, since this crate only uses the lpm instruction, which is limited by a 16-bit pointer, this crate may only be used with data stored in the lower 64 kiB of program memory. Since this property has not be tested it is unclear whether it will cause a panic or right-up undefined behavior, so be very wary when working with AVR chips having more then 64 kiB of program memory. This second restriction, of course, dose not apply to non-AVR architectures.

License

Licensed under Apache License, Version 2.0 (LICENSE or https://www.apache.org/licenses/LICENSE-2.0).

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be licensed as above, without any additional terms or conditions.

Comments
  • add documentation about how to put a string in progmem

    add documentation about how to put a string in progmem

    I have spent the afternoon experimenting with how to put a string in progmem.

    Using

    #[cfg_attr(target_arch = "avr", link_section = ".progmem.data")]
    static MESSAGE3: &str = "apple";
    

    only puts the pointer and count of the slice in progmem. The actual characters still end up in RAM.

    It is possible to use something like

    #[cfg_attr(target_arch = "avr", link_section = ".progmem.data")]
    static MESSAGE6: &[u8; 9] = b"dai kenja";
    

    But if you try

    #[cfg_attr(target_arch = "avr", link_section = ".progmem.data")]
    static MESSAGE5: &[u8; 13] = b"dai 大賢者 kenja";
    

    you get an error:

    error: byte constant must be ASCII. Use a \xHH escape for a non-ASCII byte
    
    opened by mutantbob 22
  • Implement `IntoIterator` on `ProgMem<[T;N]>`

    Implement `IntoIterator` on `ProgMem<[T;N]>`

    While playing with the ili9341 crate I found their draw_raw_iter method took a parameter <I:IntoIterator<Item=u16>> and thought "ProgMem should do that".

    opened by mutantbob 2
  • convert from llvm_asm! to (I think) asm!

    convert from llvm_asm! to (I think) asm!

    The latest AVR template now uses channel = "nightly-2022-05-10". Unfortunately, that breaks any code that uses llvm_asm!

    error: cannot find macro `llvm_asm` in this scope
      --> /home/thoth/.cargo/registry/src/github.com-1ecc6299db9ec823/avr-progmem-0.2.0/src/raw.rs:84:4
       |
    84 |             llvm_asm!(
       |             ^^^^^^^^
    

    I think the avr-hal crates use feature flags enabled by build.rs to support the new rust : https://github.com/Rahix/avr-hal/commit/5d757696e338104622a0f89c51020b718bf48b62

    opened by mutantbob 2
  • add an `include_str!` example

    add an `include_str!` example

    I added the include_str! example we were talking about. I also removed some unsafe keywords that the compiler said were unnecessary. I know it has demanded the unsafe keyword in other instances of an unsafe function body, but I don't know all the subtleties.

    opened by mutantbob 1
  • Fix incompatibility with avr-binutils >= 2.30

    Fix incompatibility with avr-binutils >= 2.30

    Due to a change in the linker scripts supplied with avr-binutils in version 2.30 (2018-01-27), the bare .progmem section name is no longer recognized - one now has to use a section name matching the .progmem.* glob:

    --- a/ld/scripttempl/avr.sc
    +++ b/ld/scripttempl/avr.sc
    @@ -128,7 +128,7 @@ SECTIONS
         ${RELOCATING+ *libprintf_flt.a:*(.progmem.data)}
         ${RELOCATING+ *libc.a:*(.progmem.data)}
     
    -    ${RELOCATING+ *(.progmem*)}
    +    ${RELOCATING+ *(.progmem.*)}
         
         ${RELOCATING+. = ALIGN(2);}
    

    The relevant commit can be found here: https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=f726cc1cb69ab0e0303209d08effb10b99007080

    I also added two more commits in the pull-request: One to fix a compiler warning about dead code when the lpm-asm-loop feature is active and one which adds a cargo config + runner script for Arduino Uno. This allows running the example with just:

    cargo run --example uno-serial
    

    If you don't want those additional commits, please let me know, I can remove them again.

    opened by Rahix 1
  • Consider pinning `avr-hal` revision in dev-dependencies

    Consider pinning `avr-hal` revision in dev-dependencies

    Hey!

    You should probably pin the version of avr-hal in the dev-dependencies to a specific git revision. avr-hal is still quite early in development and newer versions might break the example code so this way anyone downloading the project and trying to compile the example in the future will build against the same version of avr-hal that you originally wrote it for:

    [dev-dependencies.arduino-uno]
    # It is not yet available via crates.io
    git = "https://github.com/Rahix/avr-hal.git"
    rev = "7337cd76cd96f2d27701b137396d94a06d3a501d"
    

    (7337cd76cd96 is the latest commit on master, you should probably check whether that still works and use an older rev instead if it does not ...)

    opened by Rahix 1
  • Change `ProgMem` from a wrapper by value to simple pointer wrapper.

    Change `ProgMem` from a wrapper by value to simple pointer wrapper.

    The motivation for this change is to have two statics: a hidden one that will store the actual value in progmem and a public facade static that the user can access by referencing it.

    TODOs:

    • [x] implement change
    • [x] update docs
    opened by Cryptjar 0
Owner
null
Register access crate for AVR microcontrollers

avr-device Auto-generated wrappers around registers for AVR microcontrollers. Usage Add the following to Cargo.toml: [dependencies.avr-device] version

Rahix 96 Nov 13, 2022
Hardware Abstraction Layer for AVR microcontrollers and common boards

avr-hal Hardware Abstraction Layer for AVR microcontrollers and common boards (for example Arduino). Based on the avr-device crate. This is a new vers

Rahix 734 Nov 30, 2022
AVR device definitions

avrd AVR device definitons in Rust. Documentation This crate exposes information about different AVR microcontrollers so it can be used pragmatically.

The AVR-Rust project 29 Nov 2, 2022
Functional testing framework for AVR binaries, powered by simavr.

Functional testing framework for AVR binaries, powered by simavr. tl;dr get your microcontroller's firmware black-box-tested in seconds!

Patryk Wychowaniec 14 Nov 16, 2022
Batch rename utility for developers

nomino Batch rename utility for developers How to install Pre-Compiled You can download a pre-compiled executable for Linux, MacOS and Windows operati

Navid 448 Nov 23, 2022
A small utility for tracking the change in opening and closing of issues in a GitHub repo

A small utility for tracking the change in opening and closing of issues in a GitHub repo. This tool can be used to build visualizations for issue triage over time with the hope of motivating closing more issues than are opened.

Ryan Levick 12 Sep 29, 2021
mdTranslation is a utility to prepare multi-lingual Markdown documents.

mdTranslation is a utility to prepare multi-lingual Markdown documents. There's also a mdBook preprocessor called mdbook-translation for

Charles Lew 15 Sep 14, 2022
ᎩᎦᎨᎢ (IPA: [gigagei]) is a random quote fetching console utility. Written in Rust.

gigagei ᎩᎦᎨᎢ (IPA: [gigagei]) is a random quote fetching console utility. Written in Rust. Installing Use latest pre-built binary from releases Buildi

veleth 10 Jun 17, 2022
Utility library to work with tuples.

Utility library to work with tuples.

René Kijewski 8 Sep 15, 2022
Devmode is a project management utility for developers.

Dev(mode) Dev(mode) is a project management utility for developers.

Eduardo Flores 14 Oct 6, 2022
A fast, multi-threaded line counting utility written in Rust.

xloc A fast, multi-threaded line counting utility written in Rust. What is xloc A drop in replacement for bash's wc -l. Your project has x lines of co

null 1 Nov 15, 2021
Provides utility functions to perform a graceful shutdown on an tokio-rs based service

tokio-graceful-shutdown IMPORTANT: This crate is in an early stage and not ready for production. This crate provides utility functions to perform a gr

null 57 Oct 28, 2022
A cli utility written in Rust that allows fetching all the labels of a project, save those as a YAML file

A cli utility written in Rust that allows fetching all the labels of a project, save those as a YAML file that you can easily edit or save as backup and apply a saved preset to new repositories.

Chevdor 4 May 5, 2022
Emoji-printer - Utility to convert strings with emoji shortcodes to strings with the emoji unicode

Emoji Printer Intro Utility to convert strings with emoji shortcodes (:sushi:) to strings with the emoji unicode ( ?? ) Install cargo add emoji-printe

Kyle Scully 2 Dec 30, 2021
Rs.aws-login - A command line utility to simplify logging into AWS services.

aws-login A command line utility to simplify logging into AWS accounts and services. $ aws-login use ? Please select a profile to use: › ❯ dev-read

Kevin Herrera 11 Oct 30, 2022
A rust-based version of the popular dnsgen python utility

ripgen A rust-based version of the popular dnsgen python utility. ripgen is split into two main parts: ripgen: A CLI utility that calls into ripgen_li

resync 189 Nov 17, 2022
Tons of extension utility functions for Rust

LazyExt Tons of extension utility functions for Rust. English | 简体中文 Status Name Status Crate Documents Introduction lazyext-slice Alpha Thousands of

Al Liu 3 Jan 17, 2022
Lupus is a utility to administer backups with future integration with rsync

Lupus is a utility to administer backups with future integration with rsync. Many other features are either included or planned such as chat bridges using rcon and or parsing the pipe output from programs/games.

null 3 Sep 19, 2022
Auto Fan Management Utility in Linux Systems for Monster Laptops

Auto Fan Management Utility in Linux Systems for Monster Laptops Monster Laptoplar için Linux Sistemlerde Oto Fan Yönetimi TR Monster laptoplar gömülü

null 2 Aug 22, 2022