Allow different metrics as objective function and add a stopping criteria for the calibration

[CyrilThebault]

• I updated the calculate_sim_stats.py script to be able to calculate different metric as objective function (e.g. NSE, KGE, RMSE, MAE).
• I added the stopping_criteria.py script in order to stop the calibration process if a stopping criteria is reached (typically if the metrics doesn't improve anymore over the few last saves). This script is called in save_best.sh. I also added some lines to check if the 'OstQuit.txt' file is in the folder and remove it if TRUE to be able to restart the calibration process without any issue due to a previous run that would have been stopped with this new feature.
• I updated the control_active files to consider these changes

[andywood]

Are the changes related to stopping criteria backward compatible? ie, if a user doesn't care to include that in their calibration workflow, is there an easy option to skip it?

[CyrilThebault]

Hi Andy, You're right, I need to add an argument in the control_active file to decide whether or not to use the stopping criterion. I can make this change quickly and easily. I added this feature because DDS was design "to scale the search to the user-specified number of maximum function evaluations and thus has no other stopping criteria" (see Section 2.2, Tolson and Shoemaker, 2007). In the Ostrich documentation it is specified: 3.5. Aborting an Ostrich Run An OSTRICH run can be aborted by creating a file named OstQuit.txt and placing it in the working directory where the corresponding ostIn.txt file is located. OSTRICH will detect the presence of the OstQuit.txt file during the next iteration of search and will exit gracefully. So what my script does is calculate the criterion and create the OstQuit.txt file once the condition is true. Thus, this script is specific to Ostrich and I haven't adapted it for python DDS (demo 2). Have a nice day, Cyril

…

________________________________ De : Andy Wood ***@***.***> Envoyé : mercredi 14 août 2024 22:50 À : CH-Earth/summa_calib ***@***.***> Cc : Cyril Thébault ***@***.***>; Author ***@***.***> Objet : Re: [CH-Earth/summa_calib] Allow different metrics as objective function and add a stopping criteria for the calibration (PR #8) [△EXTERNAL] Are the changes related to stopping criteria backward compatible? ie, if a user doesn't care to include that in their calibration workflow, is there an easy option to skip it? Also, is that possibly already a functionality of Ostrich? — Reply to this email directly, view it on GitHub<#8 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BG33KGJ46X24ZGDWMISFIYTZRQXSTAVCNFSM6AAAAABMQ7O2B2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJQGYYDAOJQGQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>