Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CHF benchmark using synthetic data, paper finalized fixes too #63

Merged
merged 20 commits into from
Aug 23, 2024

Conversation

myerspat
Copy link
Collaborator

@myerspat myerspat commented Aug 22, 2024

PR Description

This PR includes the CHF benchmark, described in the benchmark at docs/source/benchmarks/chf.ipynb and in the docstring of pyMAISE.datasets.load_chf(). This PR also includes some quality-of-life fixes before paper publication. These are listed below.

Closes: #49
Closes: #51

What changes were made?

  • CHF benchmark, data, and load function at pyMAISE.datasets.load_chf()
  • Added tensorflow.keras.Input layer for building DNNs in pyMAISE.methods.nnHyperModel.build()
  • Added notebook output clearing function in pyMAISE.utils.display._try_clear(), and used this function throughout pyMAISE routine
  • Added progress printing using tqdm in pyMAISE.PostProcessor and pyMAISE.CVTuner
  • Fixed pyMAISE.PostProcessor.confusion_matrix() and included both number and percentage, handles binary and multiclass cases
  • Removed MSE in pyMAISE.PostProcessor.metrics() and added mean absolute percentage error (MAPE) and root mean squared percentage error (RMSPE)
  • Changed pyMAISE.PostProcessor.validation_plot() to scatter plot
  • Added pretty-printing function (pyMAISE.PostProcessor.print_model()) for readable NN hyperparameters
  • Move dependence of keras to tensorflow.keras only
  • Reran all notebooks
  • Added network plotting at pyMAISE.PostProcessor.nn_network_plot()

Reviewers: @mradaideh

@myerspat myerspat requested a review from mradaideh August 22, 2024 13:42
@myerspat myerspat self-assigned this Aug 22, 2024
@mradaideh
Copy link
Collaborator

Thanks Patrick for the work. I did go through my list and yours and they cross in many points. I will repeat those points below since they are not explicitly mentioned in your report, but sorry if they were already embedded:

1- pip install tensorflow[and-cuda] by default when we install pymaise via pip or when we get via github.
2- Install via source, add this option for quicker install without having to clone: pip install git+https://github.com/myerspat/pyMAISE.git
3- add model.summary() for keras networks by default to show all layers and nodes. This will also show total number of parameters to estimate in the network.
4- Add Keras network plotter but this does not need to be default and can be invoked as an option in the Postprocessor class.
5- You have already addressed that transform issue in xarray. I hope you can explain it somewhere in the API or in a notebook. Also, it might worth adding to your development notes (I keep forgetting them) is to add a notebook example that shows tuning on smaller dataset and then postprocess on full data which can be used to demonstrate these nuances.

Aside from that, I look forward to pip install the new version.

@mradaideh
Copy link
Collaborator

Also forgot to add for the printing of NN structure, you may add the number of nodes for the input and output layers and test the printer with LSTM and CNN to see if it consistent.

@myerspat myerspat merged commit efc4672 into develop Aug 23, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants