All implementations of Cost::gradient simply multiply a gradient by gradient_error_by_coefficent. I.e. with the currently implemented Cost functions, we could just drop the gradient_error_by_coefficent variable from the interface and let the caller of Cost::gradient multiply with such a value.

Of course, if it were to be expected that some implementations of the cost function would use that parameter in a non-multiplicative way, having it remain in the interface might be a reasonable choice.

• Mathematically sound
• Easier code (subjective, but we do need less loops)
• Sharing of intermediate results which would otherwise recalculated for each coefficient
• Will help avoid dynamic allocations in future. (First step to get rid of Vec in training states)

vikos supports logistic and linear regression. Actually we could change it to train any given GLM model (https://en.wikipedia.org/wiki/Generalized_linear_model).

Such models are of the type y=g^-1(X*a+b)

Mostly tutorial-related changes (formatting according to the rust guidelines, added missing bits and pieces to the tutorial text).

Also spell out "Stochastic Gradient Descent" for SGD in every instance it is mention in the docs.

The model trait now supports a target types different from f64. This extensions of the trait has been used to support a OneVsRest model in the library. The iris sample has therefore been stripped of its custom model implementation and uses the one from the library instead.

This branch splits the old Model trait into a new Model trait for accessing the coefficients and an Expert trait for modeling the relation between that coefficients and the target.

Motivation: This branch started out with the humble goal to remove the std::marker::PhantomData hack in model::Constant which has been required for type inference to kick in, so that the user does not have to specify the feature type explicitly. However the only way to get rid of it without breaking type inference, has been to split the Model trait as stated above. Despite born out of technical necessity to please the compiler, I found that this split has intriguing consequences for the domain of ML: Consider the urban legend about the barometer question. Niels Bohr was supposedly asked to determine the height of the building using a barometer. Among others he suggested:

1. Throwing the barometer from a building
2. Marking off the number of barometer lengths vertically along the emergency staircase
3. Measuring the pressure difference between ground and roof Let's say we'd have to set of measurements for 1,2 and 3. We can write three Experts for the same set of coefficients (in that case the height of the building). The generalization still works even if not all experts use all coefficients (e.g. if the length of the barometer would be unknown. We would when need to learn it, but only 2 is influenced by this).

On a less philosophical note this branch allowed to implement the Model trait directly for f64 and get rid of Constant. Once the target type is generic it will allow the same classifier to implement an Expert returning a crisp prediction (true | false) or a probability (f64)

resolves #17 by treating truth(Y), model(M), cost(c) and teacher(alias algorithm) as generic types in Traits. Before this pull request YMCA types are modelled as associated types in traits and generic types in implementations. This also removed the usage of std::marker::PhantomData in all but one places in the library

blue-yonder/vikos now has a Chat Room on Gitter

@HolgerPeters has just created a chat room. You can visit it here: https://gitter.im/vikos-optimization/Lobby.

This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.

Happy chatting.

PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

I created this PR as a PR to the base of the cost-trait branch, so that the diff is a bit easier and so that we can discuss how we want to refactor the gradient function's dependency on the derivative of the model (see Issue #10).

I refactored Cost to have a default implementation of the gradient function, and added a function outer_derivative for the derivative of the cost function when deriving with respect of the model-prediction.

Currently, the Cost trait can be defined by just specifying fn gradient(prediction, truth, gradient_by_error_coefficient). Probably, a library user will also want to be able to obtain absolute costs as a complement to the gradient. I think this is feasible, since I cannot imagine a cost function where we can give the gradient, but not an absolute value.

Another benefit of this would be, that at the moment when implementing the Cost trait, the actual cost function is not documented in code (in doc comments at best). So when reading https://github.com/blue-yonder/vikos/blob/master/src/cost.rs#L38 it is not obvious what the cost function actually calculates.

For most world applications we need to persist our expertises, so we can reuse them without training. Most likely using rustserialize, but I did not have a closer look at the library yet

At the moment, vikos is fitting its models by online algorithms, one event at a time. This is quite similar to vowpal wabbit.

Vowpal Wabbit (VW) project is a fast out-of-core learning system sponsored by Microsoft Research and (previously) Yahoo! Research. Source

Vowpal wabbit is also fitting Logistic or Linear Regression Models. It would be interesting to compare both projects in terms of speed or design decisions.

Now vikos is able to fit arbitrary glm models.

It would be nice if one adds the link functions for custom glm models such as Normal, Exponential, and so on.

You can find an overview here: Table custom link functions

What I am lacking is a description of the selling points of this package.

It should answer the question: "What is Vikos doing better than other RUST ML libraries"?

(It seems to be that data is processed in a stream fashion, one sample at a time)

Currently, in the teacher trait, Teacher<M>::teach_event<Y, C>(&self, &mut Self::Training, &mut M, &C, &M::Input, Y) has two mutable arguments, the associated training type and model. My question is, whether we could rewrite this into a function with immutable arguments, that returns new values for these arguments. This way we might be able to rewrite learn_history's loop over the history as a folding operation (reduction).

I received some verbal feedback that it would be nice to have a real short example in the readme. An elevator pitch made code snippet. It had also been remarked that the README should highlight the existence of a tutorial