Merge pull request #82 from wilsonrljr/v0.2.0

V0.2.0
wilsonrljr · Jun 17, 2022 · 74cffce · 74cffce
2 parents 740e9b9 + 1e1aaa3
commit 74cffce
Show file tree

Hide file tree

Showing 207 changed files with 7,940 additions and 34,149 deletions.
diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml
@@ -1,6 +1,6 @@
 # These are supported funding model platforms
 
-github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
+github: [wilsonrljr]
 patreon: sysidentpy
 open_collective: # Replace with a single Open Collective username
 ko_fi: # Replace with a single Ko-fi username

diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml
@@ -15,7 +15,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.7, 3.8, 3.9]
+        python-version: ["3.7", "3.8", "3.9", "3.10"]
 
     steps:
     - uses: actions/checkout@v2

diff --git a/CHANGELOG b/CHANGELOG
@@ -5,6 +5,83 @@ File for tracking changes in SysIdentPy
 Changes in SysIdentPy
 =====================
 
+v0.2.0
+------
+
+CONTRIBUTORS
+~~~~~~~~~~~~
+
+- wilsonrljr
+
+CHANGES
+~~~~~~~
+- The update **v0.2.0**  has been released with additional feature, minor API changes and fixes. 
+
+- MAJOR: Many new features for General Estimators
+    - Now the user can build General NARX models with Fourier basis function.
+    - The user can choose which basis they want by importing it from sysidentpy.basis_function. Check the notebooks with examples of how to use it.
+    - Now it is possible to build General NAR models. The user just need to pass model_type="NAR" to build NAR models.
+    - Now it is possible to build General NFIR models. The user just need to pass model_type="NFIR" to build NAR models.
+    - Now it is possible to run n-steps ahead prediction using General Estimators. Until now only infinity-steps ahead were allowed. Now the users can set any steps they want.
+    - Polynomial and Fourier are supported for now. New basis functions will be added in next releases.
+    - No need to pass the number of inputs anymore.
+    - Improved docstring.
+    - Fixed minor grammatical and spelling mistakes.
+    - many under the hood changes.
+
+- MAJOR: Many new features for NARX Neural Network
+    - Now the user can build Neural NARX models with Fourier basis function.
+    - The user can choose which basis they want by importing it from sysidentpy.basis_function. Check the notebooks with examples of how to use it.
+    - Now it is possible to build Neural NAR models. The user just need to pass model_type="NAR" to build NAR models.
+    - Now it is possible to build Neural NFIR models. The user just need to pass model_type="NFIR" to build NAR models.
+    - Now it is possible to run n-steps ahead prediction using Neural NARX. Until now only infinity-steps ahead were allowed. Now the users can set any steps they want.
+    - Polynomial and Fourier are supported for now. New basis functions will be added in next releases.
+    - No need to pass the number of inputs anymore.
+    - Improved docstring.
+    - Fixed minor grammatical and spelling mistakes.
+    - many under the hood changes.
+
+- Major: Support for old methods removed.
+    - Now the old sysidentpy.PolynomialNarmax is not available anymore. All the old features are included in the new API with a lot of new features and performance improvements.
+
+- API Change (new): sysidentpy.general_estimators.ModelPrediction
+    - ModelPrediction class was adapted to support General Estimators as a stand-alone class.
+    - predict: base method for prediction. Support infinity_steps ahead, one-step ahead and n-steps ahead prediction and any basis function.
+    - _one_step_ahead_prediction: Perform the 1-step-ahead prediction for any basis function.
+    - _n_step_ahead_prediction: Perform the n-step-ahead prediction for polynomial basis.
+    - _model_prediction: Perform the infinity-step-ahead prediction for polynomial basis.
+    - _narmax_predict: wrapper for NARMAX and NAR models.
+    - _nfir_predict: wrapper for NFIR models.
+    - _basis_function_predict: Perform the infinity-step-ahead prediction for basis functions other than polynomial.
+    - basis_function_n_step_prediction: Perform the n-step-ahead prediction for basis functions other than polynomial.
+
+- API Change (new): sysidentpy.neural_network.ModelPrediction
+    - ModelPrediction class was adapted to support Neural NARX as a stand-alone class.
+    - predict: base method for prediction. Support infinity_steps ahead, one-step ahead and n-steps ahead prediction and any basis function.
+    - _one_step_ahead_prediction: Perform the 1-step-ahead prediction for any basis function.
+    - _n_step_ahead_prediction: Perform the n-step-ahead prediction for polynomial basis.
+    - _model_prediction: Perform the infinity-step-ahead prediction for polynomial basis.
+    - _narmax_predict: wrapper for NARMAX and NAR models.
+    - _nfir_predict: wrapper for NFIR models.
+    - _basis_function_predict: Perform the infinity-step-ahead prediction for basis functions other than polynomial.
+    - basis_function_n_step_prediction: Perform the n-step-ahead prediction for basis functions other than polynomial.
+
+- API Change: Fit method for Neural NARX revamped.
+    - No need to convert the data to tensor before calling Fit method anymore.
+
+API Change: Keyword and positional arguments
+    - Now users have to provide parameters with their names, as keyword arguments, instead of positional arguments. This is valid for every model class now.
+
+- API Change (new): sysidentpy.utils.narmax_tools
+    - New functions to help user getting useful information to build model. Now we have the regressor_code helper function to help to build neural NARX models.
+
+- DOC: Improved Basic Steps notebook with new details about the prediction function.
+- DOC: NARX Neural Network notebook was updated following the new api and showing new features.
+- DOC: General Estimators notebook was updated following the new api and showing new features.
+- DOC: Fixed minor grammatical and spelling mistakes, including Issues #77 and #78.
+- DOC: Fix issues related to html on Jupyter notebooks examples on documentation.
+
+
 v0.1.9
 ------
 

diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
@@ -3,7 +3,7 @@ Contributing
 
 SysIdentPy is intended to be a community project, hence all contributions are welcome!
 
-Sugestions and Bug reporting
+Suggestions and Bug reporting
 ----------------------------
 There exist many possible use cases in System Identification field
 and we can not test all scenarios without your help! If you find any

diff --git a/README.md b/README.md
@@ -20,13 +20,13 @@
 SysIdentPy is a Python module for System Identification using **NARMAX** models built on top of **numpy** and is distributed under the 3-Clause BSD license.
 
 # Note
-The update **v0.1.7**  has been released with major changes and additional features (Fourier basis function, NAR and NFIR models, possibility to select the lag of the residues for Extended Least Squares algorithm and many more).
+The update **v0.2.0**  has been released with major changes and additional features (Fourier basis function, NAR and NFIR models, n-steps ahead prediction for both General Estimators and Neural NARX and more).
 
-There are several API modifications and you will need to change your code to have the new (and upcoming) features.
+There are API modifications and you will need to change your code to have the new (and upcoming) features.
 
 Check the examples of how to use the new version in the [documentation page](<http://sysidentpy.org/notebooks.html>).
 
-For more details, please see the [changelog](<http://sysidentpy.org/changelog/v0.1.7.html>).
+For more details, please see the [changelog](<http://sysidentpy.org/changelog/v0.2.0.html>).
 
 # Documentation
 
@@ -36,21 +36,23 @@ For more details, please see the [changelog](<http://sysidentpy.org/changelog/v0
 
 ## SysIdentPy now support NARX Neural Network and General estimators, e.g., sklearn estimators and Catboost.
 
-### Exemples
+### Examples
 ```python
 from torch import nn
 import numpy as np
 import pandas as pd
 import matplotlib.pyplot as plt
-from sysidentpy.metrics import mean_squared_error
+from sysidentpy.metrics import root_relative_squared_error
 from sysidentpy.utils.generate_data import get_siso_data
 
 
 # Generate a dataset of a simulated dynamical system
-x_train, x_valid, y_train, y_valid = get_siso_data(n=1000,
-                                                   colored_noise=False,
-                                                   sigma=0.001,
-                                                   train_percentage=80)
+x_train, x_valid, y_train, y_valid = get_siso_data(
+  n=1000,
+  colored_noise=False,
+  sigma=0.001,
+  train_percentage=80
+)
 ```
 
 #### Building Polynomial NARX models with FROLS algorithm
@@ -76,6 +78,7 @@ model = FROLS(
 )
 model.fit(X=x_train, y=y_train)
 yhat = model.predict(X=x_valid, y=y_valid)
+rrse = root_relative_squared_error(y_valid, yhat)
 print(rrse)
 r = pd.DataFrame(
 	results(
@@ -101,6 +104,10 @@ plot_residues_correlation(data=x1e, title="Residues", ylabel="$x_1e$")
 #### NARX Neural Network
 ```python
 from sysidentpy.neural_network import NARXNN
+from sysidentpy.basis_function import Polynomial
+from sysidentpy.utils.plotting import plot_residues_correlation, plot_results
+from sysidentpy.residues.residues_correlation import compute_residues_autocorrelation
+from sysidentpy.residues.residues_correlation import compute_cross_correlation
 
 class NARX(nn.Module):
     def __init__(self):
@@ -118,30 +125,42 @@ class NARX(nn.Module):
         z = self.lin3(z)
         return z
 
+basis_function=Polynomial(degree=1)
+
 narx_net = NARXNN(
   net=NARX(),
   ylag=2,
   xlag=2,
+  basis_function=basis_function,
+  model_type="NARMAX",
   loss_func='mse_loss',
   optimizer='Adam',
   epochs=200,
   verbose=False,
   optim_params={'betas': (0.9, 0.999), 'eps': 1e-05} # optional parameters of the optimizer
 )
 
-train_dl = narx_net.data_transform(x_train, y_train)
-valid_dl = narx_net.data_transform(x_valid, y_valid)
-narx_net.fit(train_dl, valid_dl)
-yhat = narx_net.predict(x_valid, y_valid)
-ee, ex, extras, lam = narx_net.residuals(x_valid, y_valid, yhat)
-narx_net.plot_result(y_valid, yhat, ee, ex)
+narx_net.fit(X=x_train, y=y_train)
+yhat = narx_net.predict(X=x_valid, y=y_valid)
+plot_results(y=y_valid, yhat=yhat, n=1000)
+ee = compute_residues_autocorrelation(y_valid, yhat)
+plot_residues_correlation(data=ee, title="Residues", ylabel="$e^2$")
+x1e = compute_cross_correlation(y_valid, yhat, x_valid)
+plot_residues_correlation(data=x1e, title="Residues", ylabel="$x_1e$")
 ```
 ![neural](/examples/figures/narx_network.png)
 
 #### Catboost-narx
 ```python
-from sysidentpy.general_estimators import NARX
 from catboost import CatBoostRegressor
+from sysidentpy.general_estimators import NARX
+from sysidentpy.basis_function import Polynomial
+from sysidentpy.utils.plotting import plot_residues_correlation, plot_results
+from sysidentpy.residues.residues_correlation import compute_residues_autocorrelation
+from sysidentpy.residues.residues_correlation import compute_cross_correlation
+
+
+basis_function=Polynomial(degree=1)
 
 catboost_narx = NARX(
   base_estimator=CatBoostRegressor(
@@ -150,13 +169,17 @@ catboost_narx = NARX(
     depth=6),
   xlag=2,
   ylag=2,
+  basis_function=basis_function,
   fit_params={'verbose': False}
 )
 
-catboost_narx.fit(x_train, y_train)
-yhat = catboost_narx.predict(x_valid, y_valid)
-ee, ex, extras, lam = catboost_narx.residuals(x_valid, y_valid, yhat)
-catboost_narx.plot_result(y_valid, yhat, ee, ex)
+catboost_narx.fit(X=x_train, y=y_train)
+yhat = catboost_narx.predict(X=x_valid, y=y_valid)
+plot_results(y=y_valid, yhat=yhat, n=1000)
+ee = compute_residues_autocorrelation(y_valid, yhat)
+plot_residues_correlation(data=ee, title="Residues", ylabel="$e^2$")
+x1e = compute_cross_correlation(y_valid, yhat, x_valid)
+plot_residues_correlation(data=x1e, title="Residues", ylabel="$x_1e$")
 ```
 ![catboost](/examples/figures/catboost_narx.png)
 
@@ -167,7 +190,7 @@ The following is the Catboost performance without the NARX configuration.
 
 ```python
 
-def plot_results(yvalid, yhat):
+def plot_results_tmp(y_valid, yhat):
     _, ax = plt.subplots(figsize=(14, 8))
     ax.plot(y_valid[:200], label='Data', marker='o')
     ax.plot(yhat[:200], label='Prediction', marker='*')
@@ -183,17 +206,17 @@ catboost = CatBoostRegressor(
   depth=6
 )
 catboost.fit(x_train, y_train, verbose=False)
-plot_results(y_valid, catboost.predict(x_valid))
+plot_results_tmp(y_valid, catboost.predict(x_valid))
 ```
 ![catboost](/examples/figures/catboost.png)
 
-The examples directory has several Jupyter notebooks presenting basic tutorials of how to use the package and some specific applications of sysidentpy. Try it out!
+The examples directory has several Jupyter notebooks with tutorials of how to use the package and some specific applications of sysidentpy. Try it out!
 
 # Requirements
 
 SysIdentPy requires:
 
-- Python (>= 3.6)
+- Python (>= 3.7)
 - NumPy (>= 1.5.0) for all numerical algorithms
 - Matplotlib >= 1.5.2 for static plotting and visualizations
 - Pytorch (>=1.7.1) for building feed-forward neural networks

diff --git a/docs/_images/air_passenger_benchmark_13_1.png b/docs/_images/air_passenger_benchmark_13_1.png
diff --git a/docs/_images/aols_4_0.png b/docs/_images/aols_4_0.png
diff --git a/docs/_images/basic_steps_17_0.png b/docs/_images/basic_steps_17_0.png
diff --git a/docs/_images/basic_steps_17_1.png b/docs/_images/basic_steps_17_1.png
diff --git a/docs/_images/basic_steps_17_2.png b/docs/_images/basic_steps_17_2.png
diff --git a/docs/_images/basic_steps_19_1.png b/docs/_images/basic_steps_19_1.png
diff --git a/docs/_images/basic_steps_23_1.png b/docs/_images/basic_steps_23_1.png
diff --git a/docs/_images/extended_least_squares_12_1.png b/docs/_images/extended_least_squares_12_1.png
diff --git a/docs/_images/extended_least_squares_14_1.png b/docs/_images/extended_least_squares_14_1.png
diff --git a/docs/_images/f_16_benchmark_15_0.png b/docs/_images/f_16_benchmark_15_0.png
diff --git a/docs/_images/f_16_benchmark_17_1.png b/docs/_images/f_16_benchmark_17_1.png
diff --git a/docs/_images/f_16_benchmark_7_1.png b/docs/_images/f_16_benchmark_7_1.png
diff --git a/docs/_images/f_16_benchmark_8_1.png b/docs/_images/f_16_benchmark_8_1.png
diff --git a/docs/_images/general_estimators_10_0.png b/docs/_images/general_estimators_10_0.png
diff --git a/docs/_images/general_estimators_13_1.png b/docs/_images/general_estimators_13_1.png
diff --git a/docs/_images/general_estimators_13_2.png b/docs/_images/general_estimators_13_2.png
diff --git a/docs/_images/general_estimators_13_3.png b/docs/_images/general_estimators_13_3.png
diff --git a/docs/_images/general_estimators_15_1.png b/docs/_images/general_estimators_15_1.png
diff --git a/docs/_images/general_estimators_15_2.png b/docs/_images/general_estimators_15_2.png
diff --git a/docs/_images/general_estimators_15_3.png b/docs/_images/general_estimators_15_3.png
diff --git a/docs/_images/general_estimators_17_1.png b/docs/_images/general_estimators_17_1.png
diff --git a/docs/_images/general_estimators_17_2.png b/docs/_images/general_estimators_17_2.png
diff --git a/docs/_images/general_estimators_17_3.png b/docs/_images/general_estimators_17_3.png
diff --git a/docs/_images/general_estimators_19_1.png b/docs/_images/general_estimators_19_1.png
diff --git a/docs/_images/general_estimators_19_2.png b/docs/_images/general_estimators_19_2.png
diff --git a/docs/_images/general_estimators_19_3.png b/docs/_images/general_estimators_19_3.png
diff --git a/docs/_images/general_estimators_9_0.png b/docs/_images/general_estimators_9_0.png
diff --git a/docs/_images/identification_of_an_electromechanical_system_4_1.png b/docs/_images/identification_of_an_electromechanical_system_4_1.png
diff --git a/docs/_images/identification_of_an_electromechanical_system_8_2.png b/docs/_images/identification_of_an_electromechanical_system_8_2.png
diff --git a/docs/_images/identification_of_an_electromechanical_system_8_3.png b/docs/_images/identification_of_an_electromechanical_system_8_3.png
diff --git a/docs/_images/identification_of_an_electromechanical_system_8_4.png b/docs/_images/identification_of_an_electromechanical_system_8_4.png
diff --git a/docs/_images/metamss_2_1.png b/docs/_images/metamss_2_1.png
diff --git a/docs/_images/metamss_5_1.png b/docs/_images/metamss_5_1.png
diff --git a/docs/_images/metamss_5_2.png b/docs/_images/metamss_5_2.png
diff --git a/docs/_images/metamss_5_3.png b/docs/_images/metamss_5_3.png
diff --git a/docs/_images/metamss_6_1.png b/docs/_images/metamss_6_1.png
diff --git a/docs/_images/multiple_inputs_example_10_1.png b/docs/_images/multiple_inputs_example_10_1.png
diff --git a/docs/_images/multiple_inputs_example_11_1.png b/docs/_images/multiple_inputs_example_11_1.png
diff --git a/docs/_images/narx_neural_network_19_1.png b/docs/_images/narx_neural_network_19_1.png
diff --git a/docs/_images/narx_neural_network_19_2.png b/docs/_images/narx_neural_network_19_2.png
diff --git a/docs/_images/narx_neural_network_19_3.png b/docs/_images/narx_neural_network_19_3.png
diff --git a/docs/_images/narx_neural_network_21_1.png b/docs/_images/narx_neural_network_21_1.png
diff --git a/docs/_images/narx_neural_network_21_2.png b/docs/_images/narx_neural_network_21_2.png
diff --git a/docs/_images/narx_neural_network_21_3.png b/docs/_images/narx_neural_network_21_3.png