Apologies for the large diff! Much of it is comments and tests, though.

These changes make the FASTA parser more flexible, and will allow us to handle FASTA files the way we need for SecureDNA (SDNA employees, refer to the "Notes" section in [[Alpha API endpoint]] on the internal wiki).

Changes

Adds 2 new settings: allow_preceding_comment and concatenate_headers. These are described in the doc comments, but briefly:

• allow_preceding_comment = true will ignore any content preceding the first header, like before. If false, it will emit that content as a headerless (header = "") record.
• concatenate_headers = true will glue successive headers with no content together with newlines. If false, it will use the old behavior and emit content = "" records. See the doc comment for a doctest that explains this.

The return value of FastaParser is now a FastaRecord with header and contents fields, instead of a 2-tuple. It's also grown a line_range field which gives the line numbers of the record (including header, 1-indexed, [start, end)).

Removes DnaFastaParser and SimpleFastaParser in favor of a single FastaParser type which is generic over FromStr types. Since String has an (infalliable) FromStr impl, SimpleFastaParser is now FastaParser<String>, and DnaFastaParser is now FastaParser<DnaSequence>. We didn't have this type before, but a hypothetical ProteinFastaParser is now FastaParser<ProteinSequence>, and you could even have FastaParser<u64> if you wanted that.

FASTA parsing is now state-machine-driven, which should make a non-allocating API easier to add in the future if we want, and also simplified the parsing code since it was getting hairy handling the new special cases in the old loop. Luckily most of @mkysel 's parsing logic was relatively straightforward to port into the state machine without too many changes besides handling the new settings.

The old tests are all redone to use the new API (and to test all 4 settings combinations, where it makes sense), and a bunch of new tests are added.

Misc changes:

parse now explicitly asks for a BufRead instead of taking Read and wrapping it internally in BufReader. I checked the ecosystem using SourceGraph and that seemed to be the way most Rust stuff works, so this brings our API in line with that.

There's a new parse_str method that takes advantage of &[u8] having a BufRead impl. That's part of the reason for the other change, too, since before that would have needlessly allocated a BufReader buffer which would bother me :-)

This Display impl converts the parsed record back into FASTA >header\ncontents\n format. Implementing Display automatically gets us ToString and .to_string().

(We want this because it's the easiest way to turn a Vec<FastaRecord> contained in an ELT back into a string to ASN.1-encode it.)

This provides a more polished version of the API shown in in #19, hopefully making it easier to work with DNA stored in more kinds of data structures. I've also included a benchmark for using this API to generate windows with fewer allocations, though it doesn't fully satisfy #20 because we still store AAs as &strs, preventing us from using Rust's standard windowing.

Closes #18.

Changes

Types

• Nucleotide no longer has N in it, and just represents one of ACTG.
• NucleotideAmbiguous represents ACTG or one of the 11 ambiguity codes WMRYSKBVDHN.
• There is a trait NucleotideLike for common behavior between the two, like "to ASCII" and "complement".
• Similarly, Codon (3x Nucleotide) and CodonAmbiguous (3x NucleotideAmbiguous) are different types now.
• The ambiguous types have possibilities() methods for iterating over the possible realizations.

Translation

• Ambiguity codes are properly handled instead of mapping them all to N.
• DnaSequence is generic over the type of the contained nucleotides. So, DnaSequence::<Nucleotide>::from_str(s) is our "strict mode", and DnaSequence::<NucleotideAmbiguous>::from_str(s) is the lax mode.

Tests

• I added a test test_dna_parses_strict, which verifies that this "strict mode" indeed only accepts "aAtTcCgG \t".
• I added a test test_translate_ambiguous, which verifies that TTRTTV maps to protein LX:

        // R means "A or G" and both {TTA,TTG} map to L (Leucine).
        // Thus, "TTR" should map to L.
        //
        // But V means "A or G or C", and TTC maps to F (Phenylalanine).
        // Thus, "TTV" is truly ambiguous and maps to X.
  

Updates the requirements on pyo3 to permit the latest version.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Updates to fasta parsing

• change DNAFastaParser to DnaFastaParser
• change &mut R to R
• handle BufReader internally, instead of passing a BufReader to parser

We don't want to publish yet; say so explicitly in Cargo.toml (this will also help in case we use automated release tools like cargo-release)

quickdna on  chore/no-publish [+1] > cargo publish --dry-run
error: `quickdna` cannot be published.
`package.publish` is set to `false` or an empty list in Cargo.toml and prevents publishing.

When writing NucleotideIter, I failed to realize that a common use-case was code generic on NucleotideLike, so it's only easy-to-use with concrete NucleotideLike types. Using it with a generic NucleotideLike requires adding a bunch of cumbersome non-obvious constraints, which in turn affects callers of your code. In order to make it convenient to use NucleotideIter in generic contexts I'm adding the ability to make the following inferences:

• Anything NucleotideLike is ToNucleotideLike (sadly, I can't also do this for &impl NucleotideLike types generically without specialization, but that's not a big deal in practice because we can work around it with .copied() iters)
• Any codons derived from NucleotideLike types may be passed to TranslationTable::to_fn callables.

Add python linter (black) to CI. The python test suite is already being run as part of the maturin Rust build.

I could not add pyright as there are a bunch of errors when invoked. It will be more work to enable pyright.

Right now we have a single Nucleotide type that supports A/T/C/G/N (N being "any of ATCG").

We'd like to move this type to be "strict" (only support ATCG), then have NucleotideAmbiguous which is any of the ambiguity codes (ATCGYRWSKMDVHBN).

They'd have the same enum values so "converting" a sequence would just be a zero-copy check.

Thought: assign ATCG the enum values {1,2,4,8}, and represent ambiguities as bitwise ORs.

Consideration: CodonIdx becomes 12-bit instead of 9-bit; the translation tables will now each have length 4096, up from 293. (But this is surely still small enough to keep one cached.)

This PR allows empty spaces (which get ignored) in DNA sequences.

Considerations:

• I did consider putting the check in nucleotides, but those should still error out. Its only DNA sequences that can contain nucleotide sequences with spaces in them.

Peptides accept empty spaces today and do not require a change. They also accept incorrect characters, but that is not the goal of this PR.

Looked into also adding automatic changelog generation, but we need to format our commits so they can be parsed.

cargo-release also did more in the past (it created a dev-version after the released version: v0.5.0-alpha after v0.4.0), but looks like now it's just the version bump + git tag.

…so that we can unambiguously use "strict" and "lax" to instead refer to whether we allow N nucleotides or not.

Instead, there is now a ::default() which disallows preceding file comments (i.e. treats them as sequences), but does concatenate headers. This is the parsing behavior we describe in our internal wiki. Any override of these defaults should be done explicitly when constructing the parser.

Right now a ProteinSequence is just an unchecked ASCII sequence.

We should make an AminoAcid type like Nucleotide that represents only the valid amino acids ABCDEFGHIKLMNPQRSTVWYZ, and then AminoAcidAmbiguous which includes ambiguity code X. See #18.

Thought: if we assign these by ASCII codes like enum AminoAcid { A = 65, B = 66, ... } then we can cast &[u8] to &[AminoAcid] after validation. But maybe that's a bit overzealous if we normally have to strip whitespace and uppercase the string anyway.

Consideration: What about . (deletion) and * (terminator)? Should we treat those as ambiguity codes or as something else?

Right now one of the biggest sources of allocations in synth_client is that DnaSequence allocates a new Vec sequence for every window. If we could just return a reference to the owned sequence that would be ideal. See #19.

Currently DnaSequence owns a Vec<Nucleotide>. But we'd also like to be able to have a DnaSequence backed by a borrowed &[Nucleotide]. Same for ProteinSequence.

Thought from @vgel: use generics, so DnaSequence<Vec<Nucleotide>> vs DnaSequence<&[Nucleotide]>, and then maybe nice aliases for these. Maybe even make DnaSequence<[Nucleotide; N]> work. This is what SmallVec does.

Alternatively, follow the Rust tradition of two typenames for borrowed vs. owned, like str vs String and Path vs PathBuf.

As Martin pointed out in #14 , we should have dependabot and a lockfile for the Python bindings part of the crate, since that produces a built artifact (quickdna.so). But since we don't want that for the Rust part which is meant to be consumed as a regular Rust library (https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html), we probably want two separate crates that aren't in a workspace so that only one has a lockfile, or something like that.