Collection of Optimization algorithm in Rust

Overview

rustimization

A rust optimization library which includes L-BFGS-B and Conjugate Gradient algorithm.

Documentation

The simplest way to use these optimization algorithm is to use the Funcmin class.

extern crate rustimization;
use rustimization::minimizer::Funcmin;
fn test(){
    let f = |x: &Vec<f64>| { (x[0]+4.0).powf(2.0)};
    let g = |x: &Vec<f64>| {vec![2.0*(x[0]+4.0)]};
    let mut x = vec![40.0f64];
    {
        //you must create a mutable object
        let mut fmin = Funcmin::new(&mut x,&f,&g,"cg");
        fmin.minimize();
    }
    println!("{:?}",x);
}

Output

[-4.000000000000021]

here Funcmin constructor takes four parameters first one is initial estimation x second and third one is the function f and the derivative g of the function respectively and forth one is the algorithm you want to use. Currently two algorithms available "cg" and "lbfgsb" if you want more parameter tuning you can use the classes of the algorithm such as for Lbbfgsb_minimizer class

Example

let f = |x:&Vec<f64>|{ (x[0]+4.0).powf(2.0)};
    let g = |x:&Vec<f64>|{vec![2.0*(x[0]+4.0)]};
    let mut x = vec![40.0f64];
    {
        //creating lbfgsb object. here it takes three parameter
        let mut fmin = Lbfgsb::new(&mut x,&f,&g);
        //seting upper and lower bound first parameter is the index and second one is value
        fmin.set_upper_bound(0,100.0);
        fmin.set_lower_bound(0,10.0);
        //set verbosity. higher value is more verbosity. the value is -1<= to <=101
        fmin.set_verbosity(101);
        //start the algorithm
        fmin.minimize();
    }
    println!("{:?}",x);

Output

RUNNING THE L-BFGS-B CODE

           * * *

Machine precision = 2.220D-16
 N =            1     M =            5

 L =  1.0000D+01

X0 =  4.0000D+01

 U =  1.0000D+02

At X0         0 variables are exactly at the bounds

At iterate    0    f=  1.93600D+03    |proj g|=  3.00000D+01


ITERATION     1

---------------- CAUCHY entered-------------------
 There are            1   breakpoints 

Piece      1 --f1, f2 at start point  -7.7440D+03  7.7440D+03
Distance to the next break point =   3.4091D-01
Distance to the stationary point =   1.0000D+00
 Variable             1   is fixed.
Cauchy X =  
      1.0000D+01

---------------- exit CAUCHY----------------------

           0  variables are free at GCP            1
 LINE SEARCH           0  times; norm of step =    30.000000000000000     

At iterate    1    f=  1.96000D+02    |proj g|=  0.00000D+00

 X =  1.0000D+01

 G =  2.8000D+01

           * * *

Tit   = total number of iterations
Tnf   = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip  = number of BFGS updates skipped
Nact  = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F     = final function value

           * * *

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F
    1      1      2      1     0     1   0.000D+00   1.960D+02

 X =  1.0000D+01
  F =   196.00000000000000     

CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL            

 Cauchy                time 1.570E-04 seconds.
 Subspace minimization time 0.000E+00 seconds.
 Line search           time 1.800E-05 seconds.

 Total User time 9.330E-04 seconds.

convergence!

Requirements

To use this library you must have gfortran installed in your pc

  • for windows use fortran compiler provided by mingw or TDM-GCC
  • for linux you can use the package manager to install gfortran
  • for Mac os you can install it form here or here

The orginal L-BFGS-B fortran subroutine is distributed under BSD-3 license. To know more about the condition to use this fortran routine please go here.

To know more about the condition to use the Conjugate Gradient Fortran routine please go here

References

  1. R. H. Byrd, P. Lu and J. Nocedal. A Limited Memory Algorithm for Bound Constrained Optimization, (1995), SIAM Journal on Scientific and Statistical Computing , 16, 5, pp. 1190-1208.
  2. C. Zhu, R. H. Byrd and J. Nocedal. L-BFGS-B: Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization (1997), ACM Transactions on Mathematical Software, Vol 23, Num. 4, pp. 550 - 560.
  3. J.L. Morales and J. Nocedal. L-BFGS-B: Remark on Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization (2011), to appear in ACM Transactions on Mathematical Software.
  4. J. C. Gilbert and J. Nocedal. Global Convergence Properties of Conjugate Gradient Methods for Optimization, (1992) SIAM J. on Optimization, 2, 1.
You might also like...
Rust implementation of real-coded GA for solving optimization problems and training of neural networks
Rust implementation of real-coded GA for solving optimization problems and training of neural networks

revonet Rust implementation of real-coded genetic algorithm for solving optimization problems and training of neural networks. The latter is also know

Mathematical optimization in pure Rust

argmin A pure Rust optimization framework This crate offers a numerical optimization toolbox/framework written entirely in Rust. It is at the moment p

Rust implementation of real-coded GA for solving optimization problems and training of neural networks
Rust implementation of real-coded GA for solving optimization problems and training of neural networks

revonet Rust implementation of real-coded genetic algorithm for solving optimization problems and training of neural networks. The latter is also know

Image optimization using Rust and Vips 🦀

Huffman Image optimization using Rust and Libvips. Requirements You must have the following packages installed before getting started Rust Vips pkg-co

Salty and Sweet one-line Rust Runtime Optimization Library

SAS SAS (Salty-And-Sweet) is an one-line Rust runtime optimization library. Features NUMA-aware rayon: numa feature should be enabled If you have 1 NU

Linearly Constrained Separable Optimization

LCSO (Linearly Constrained Separable Optimization) Authors The original algorithms here were developed by Nicholas Moehle, and this project was author

TP - Optimization of the energy consumption of a Matrix Multiplication

Optimization de la consommation: Multiplication de matrices Nous allons travailler sur la multiplication de matrices denses. C’est un algorithme class

A pure, low-level tensor program representation enabling tensor program optimization via program rewriting

Glenside is a pure, low-level tensor program representation which enables tensor program optimization via program rewriting, using rewriting frameworks such as the egg equality saturation library.

OptFrame is an optimization framework focused in metaheuristic techniques

optframe-rust Welcome to OptFrame project in Rust. What is OptFrame? OptFrame is an optimization framework focused in metaheuristic techniques, develo

A standalone Aleo prover build upon snarkOS and snarkVM, with multi-threading optimization

Aleo Light Prover Introduction A standalone Aleo prover build upon snarkOS and snarkVM, with multi-threading optimization. It's called "light" because

A library and application for lossless, format-preserving, two-pass optimization and repair of Vorbis data, reducing its size without altering any audio information.
A library and application for lossless, format-preserving, two-pass optimization and repair of Vorbis data, reducing its size without altering any audio information.

OptiVorbis A library and application for lossless, format-preserving, two-pass optimization and repair of Vorbis data, reducing its size without alter

A blazingly fast compiling & optimization tool for CosmWasm smart contracts.
A blazingly fast compiling & optimization tool for CosmWasm smart contracts.

cw-optimizoor A blazingly fast alternative to CosmWasm/rust-optimizer for compiling & optimizing CW smart contracts. It's primarily meant to speed up

Rust-nlp is a library to use Natural Language Processing algorithm with RUST

nlp Rust-nlp Implemented algorithm Distance Levenshtein (Explanation) Jaro / Jaro-Winkler (Explanation) Phonetics Soundex (Explanation) Metaphone (Exp

Rust-algorithm-club - Learn algorithms and data structures with Rust

Rust Algorithm Club 🚧 🚧 This repo is under construction. Most materials are written in Chinese. Check it out here if you are able to read Chinese. W

A genetic algorithm for bechmark problems, written to learn Rust.

Genetic Algorithm A genetic algorithm in Rust for the following benchmark problems: Ackley Griewangk Rastrigin Rosenbrock Schwefel Sphere Usage: Insta

Genetic Algorithm library in Rust

RsGenetic Summary and Features RsGenetic is a framework for executing genetic algorithms in Rust. It is designed to have a simple but modular API. Exa

A Rust implementation of the Zopfli compression algorithm.

Zopfli in Rust This is a reimplementation of the Zopfli compression tool in Rust. I have totally ignored zopflipng. More info about why and how I did

A complete harfbuzz's shaping algorithm port to Rust

rustybuzz rustybuzz is a complete harfbuzz's shaping algorithm port to Rust. Matches harfbuzz v2.7.0 Why? Because you can add rustybuzz = "*" to your

Multilingual implementation of RAKE algorithm for Rust

RAKE.rs The library provides a multilingual implementation of Rapid Automatic Keyword Extraction (RAKE) algorithm for Rust. How to Use Append rake to

Comments
  • error while loading shared libraries: libcgfam.so: cannot open shared object file: No such file or directory

    error while loading shared libraries: libcgfam.so: cannot open shared object file: No such file or directory

    I am getting the following error in one of my projects. It seems to be a new issue with more recent version of Cargo.

    Steps to reproduce

    Create a new project

    cargo new rustimization_bug
    

    Add rustimization==0.1.1 as a dependency in Cargo.

    Create src/main.rs as follows

    extern crate rustimization;                                                     
                                                                                    
    use rustimization::minimizer::Funcmin;                                          
                                                                                    
    fn main() {                                                                     
                                                                                    
        let f = |x : &Vec<f64>| {                                                   
            0.0f64                                                                  
        };                                                                          
        let g = |x : &Vec<f64>| {                                                   
            vec![0.0f64]                                                            
        };                                                                          
                                                                                    
        let mut x = Vec::new();                                                     
        let mut fmin = Funcmin::new(&mut x, &f, &g, "cg");                          
        fmin.max_iteration(10);                                                     
        fmin.minimize();                                                            
                                                                                    
        println!("Hello, world!");                                                  
    }                                                                               
    

    Build the system with cargo build and the running the executable creates the following error

    -> % target/debug/rustimization_bug
    target/debug/rustimization_bug: error while loading shared libraries: libcgfam.so: cannot open shared object file: No such file or directory
    

    This works with cargo run

    -> % cargo run
    warning: unused variable: `x`
     --> src/main.rs:7:14
      |
    7 |     let f = |x : &Vec<f64>| {
      |              ^ help: if this is intentional, prefix it with an underscore: `_x`
      |
      = note: `#[warn(unused_variables)]` on by default
    
    warning: unused variable: `x`
      --> src/main.rs:10:14
       |
    10 |     let g = |x : &Vec<f64>| {
       |              ^ help: if this is intentional, prefix it with an underscore: `_x`
    
    warning: `rustimization_bug` (bin "rustimization_bug") generated 2 warnings
        Finished dev [unoptimized + debuginfo] target(s) in 0.00s
         Running `target/debug/rustimization_bug`
    
     IFLAG= -3
     IMPROPER INPUT PARAMETERS (N IS NOT POSITIVE)
    Hello, world!
    

    Environment

    -> % lsb_release -a
    No LSB modules are available.
    Distributor ID:	Ubuntu
    Description:	Ubuntu 22.04 LTS
    Release:	22.04
    Codename:	jammy
    -> % cargo --version
    cargo 1.61.0 (a028ae4 2022-04-29)
    
    opened by jmccrae 0
  • l-bfgs-s `f` & `g` might be mutable

    l-bfgs-s `f` & `g` might be mutable

    According to the fortran code

    c     f is a double precision variable.
    c       On first entry f is unspecified.
    c       On final exit f is the value of the function at x.
    c
    c     g is a double precision array of dimension n.
    c       On first entry g is unspecified.
    c       On final exit g is the value of the gradient at x.
    

    It is not clear whether f & g are not modified in the subroutine.

    opened by colinfang 0
  • LBFGS-B number of variable metric corrections cannot be changed

    LBFGS-B number of variable metric corrections cannot be changed

    Calling set_matric_correction() leads to a segfault since the wa variable is not reallocated to handle the new size.

    Perhaps the constructor could be refactored into a builder pattern?

    opened by yongqli 0
  • approx_fprime_helper

    approx_fprime_helper

    Hi, I'm trying to port a python project to Rust and I'm using your library as a replacement to scipy.optimize. rustimization seems to be working ok but there's something I don't like. In scipy/optimize/lbfgsb.py, we can find this

        if jac is None:
            def func_and_grad(x):
                f = fun(x, *args)
                g = _approx_fprime_helper(x, fun, epsilon, args=args, f0=f)
                return f, g
    

    which is to say 1) use this if g wasn't provided 2) use the last value of f() when calling g(). Using your library, doing 1) is possible, but it's impossible to do 2). It's quite important if f() is expensive.

    I already coded _approx_fprime_helper and I'm pretty sure I can code this myself in your library. My question is more: is this project still alive? Do you accept PR?

    opened by nilgoyette 3
Owner
Naushad Karim
Naushad Karim
BLAS bindings for Rust

RBLAS Rust bindings and wrappers for BLAS (Basic Linear Algebra Subprograms). Overview RBLAS wraps each external call in a trait with the same name (b

Michael Yang 77 Oct 8, 2022
gmp bindings for rust

Documentation The following functions are intentionally left out of the bindings: gmp_randinit (not thread-safe, obsolete) mpz_random (not thread-safe

Bartłomiej Kamiński 37 Nov 5, 2022
Rust wrapper for ArrayFire

Arrayfire Rust Bindings ArrayFire is a high performance library for parallel computing with an easy-to-use API. It enables users to write scientific c

ArrayFire 696 Dec 30, 2022
Scientific Computing Library in Rust

SciRust Scientific computing library written in Rust programming language. The objective is to design a generic library which can be used as a backbon

In Digits 242 Dec 16, 2022
Statistical computation library for Rust

statrs Current Version: v0.13.0 Should work for both nightly and stable Rust. NOTE: While I will try to maintain backwards compatibility as much as po

Michael Ma 384 Dec 27, 2022
The write-once-run-anywhere GPGPU library for Rust

The old version of Emu (which used macros) is here. Overview Emu is a GPGPU library for Rust with a focus on portability, modularity, and performance.

Caleb Winston 1.5k Dec 30, 2022
A neural network model that can approximate any non-linear function by using the random search algorithm for the optimization of the loss function.

random_search A neural network model that can approximate any non-linear function by using the random search algorithm for the optimization of the los

ph04 2 Apr 1, 2022
Suite for automatically testing algorithm questions from the Polish Algorithm Olympiad.

oisuite Your number #1 tool to managing your algo questions! This software only works on UNIX-based operating systems (macOS, Linux, BSD, etc.) Projec

null 3 Nov 25, 2021
An implementation of the A-Star pathfinding algorithm tailored for traversing a bespoke collection of weighted hexagons.

An implementation of the A-Star pathfinding algorithm tailored for traversing a bespoke collection of weighted hexagons. It's intended to calculate the most optimal path to a target hexagon where you are traversing from the centre of one hexagon to the next along a line orthogonal to a hexagon edge

null 19 Dec 11, 2022
The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language.

rcc The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language. Compilers Language Co

null 2 Jan 17, 2022