Fwumious Wabbit, fast on-line machine learning toolkit written in Rust

Looking forward to a moment when there will be some way to test interoperation with Java - to use at least predict part in Java using JNI/JNA or something similar, the same as we currently use VW maven component (https://github.com/VowpalWabbit/vowpal_wabbit/tree/master/java).

A Vec<u8>'s buffer is only guaranteed to be 1-aligned, but it's used here in code assuming it's 4-aligned. That might work, depending on what the allocator does, but it should be checked to avoid UB.

(Alternatively it could over-allocate the buffer and do something like align_to, but I didn't want to make broad changes.)

• parameterized training data generator to simplify running many ad hoc experiments if required
• added a bash script that trains fw on generated data, stores the outputs, computes relevant classification metrics (on training data, intentionally) and alerts the user if something is off --- including properties of prediction files, as well as classification capabilities vs. random must be in good shape/of quality. Current experiments indicate that random vs. fw balanced accuracy on training data ((sensitivity + specificity) / 2) difference is more than 0.2, hence this is the current margin considered for the test to pass.
• added an action that runs this for each commit, so we know where things went south if that's the case

The plan is to include this as a git action so that for each new binary we have a few learning-level sanity checks conducted. Currently, we included the main properties we are most interested in with each new version; more can be added should the need arise.

Hi there, I've recently started my rust journey and was hoping you'd be open for some ornamental changes recommended by clippy.

This PR replaces match ... Some(...) blocks with the more idiomatic if let.

• fix a small bug with not accepting "w" as a valid character for a namespace
• allow for transformations of transformed namespaces
• implement Combine transformer

Add binning basic support with a few default binners implemented.

This allows for any kind of transformations of float values before making them categorical features to be used in LR/FFM.

1. Instead of running the benchmark for 10 million train & test examples each - running only 1 million (should be indicative enough and will run more quickly)
2. Benchmarking scripts run the benchmark only on fw, not on both fw and vw

(Reposting #61 under a different branch name.)

Hi there, I've recently started my rust journey and was hoping you'd be open for some ornamental changes recommended by clippy.

This PR replaces match ... Some(...) blocks with the more idiomatic if let.

while investigating some FW crashes due to segmentation fault, caused by malloc trying to allocate a block and complaining that the block CRC isn't the same as when it was freed, I ran valgrind (on our reproducible setup offline) and got a hint that something bad is going on in some lines in parser.rs the only thing I could imagine happening is the "buf" pointer somehow becoming invalid, it seems this can happen if the output_buffer vec grows and is reallocated. got rid of it - and the valgrind complaints went away, as well as the crashes. still not sure exactly about the scenario though - because the vector is preallocated generously on startup, so we might want to continue looking into the input as there may be something fishy going on there. WDYT?

A) Introduce two new parameters --linear (to be used instead of --interaction and --keep) and --ffm_field_verbose (to be used instead of --ffm_field)

This now allows for passing feature names / namespaces as full namespace names as found in vw_namespace_map.csv.

It's a first step to unlock more flexible namespace definitions in input files.

B) Implement multi-byte namespaces in vw_namespaces_map.csv and when parsing vw files

LMK what you think - discussed this with @flaunderg - currently we publish artifacts internally by explicitly triggering a build from Jenkins. the produced artifact is put in artifactory at fw-/fw-<branch_name>- - and that's when we set a git tag "fw-<branch_name>-version" in the repo.

we suggest that - only when creating a new tag for "main" branch, two additional things will happen:

1. the version.rs file will be overwritten, with the auto-incremented version (current is 0.1, so next is 0.2 etc.)
2. the benchmark will be run and BENCHMARK.md and benchmark_results.png will be committed

so - when someone builds (or if we choose to publish artifacts from "main") - we'll be able to tell the binary version, and not just have to rely on commit # (which we can also add to the version info as with vw, btw). this way there will be less potential for conflict when we merge branches where we already published artifacts to try them out - the version will always be taken from main. if main was promoted and you pull - you'll get the updated version.

implement ability to specify specific field interaction parameters the feature gets turned on with --ffm_interaction_matrix then field weights can be expressed by --ffm_interaction field_id_1:field_id_2:weight

weight 0 means the interaction is fully masked out. default is weight 1.0 everywhere field ids are sequential ids of fields as passed by field declaration parameters

WARNING: since fields are not named, this means that any change to order of field declarations requires careful adjustment of interactions too...

ping @adischw