Skip to content

Latest commit

 

History

History
74 lines (61 loc) · 1.82 KB

README.md

File metadata and controls

74 lines (61 loc) · 1.82 KB

Description

Provides simple and lightweight implementations of model validation and hyperparameter optimization for Julia.

Installation

Run the following code to install the package:

] add https://github.com/triepels/CrossValidation.jl

Get Started

You have to define functions fit! and loss for your model type.

julia> import CrossValidation: fit!, loss

Function fit! takes a model and fits it on data based on some optional fitting arguments:

julia> function fit!(model::MyModel, data)
           # Code to fit model...
       end

Function loss estimates how well the model performs on (out-of-sample) data:

julia> function loss(model::MyModel, data)
           # Code to evalute loss of the model...
       end

Features

Model validation based on various resample methods:

julia> validate(KFold(data, 10)) do train
            return fit!(MyModel(; args), train)
       end

Hyperparameter optimization using various optimizers:

julia> sp = space(a = DiscreteUniform(1:10))
julia> sha((x) -> MyModel(; x...), sp, FixedSplit(data, 0.8), Budget{:arg}(100))

Model validation with hyperparameter optimization:

julia> validate(KFold(data, 10)) do train
           args = brute((x) -> MyModel(; x...), sp, FixedSplit(train, 0.8))
           return fit!(MyModel(; args...), train)
       end

Resample methods

The following resample methods are available:

  • Fixed Split (Holdout)
  • Random Split (Holdout)
  • Leave-One-Out
  • K-Fold
  • Forward Chaining
  • Sliding Window

Optimizers

The following optimizers are available:

  • Grid Search
  • Random Search
  • Hill-Climbing (HC)
  • Successive Halving (SHA)
  • Hyperband
  • Simulated Annealing and Successive Halving (SASHA)

Note

This package is not yet stable. Future releases might be subject to breaking changes.