Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Polish analysis of a multiclosure test #1982

Merged
merged 153 commits into from
Oct 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
153 commits
Select commit Hold shift + click to select a range
753292a
Init analysis
andreab1997 Feb 20, 2024
948f1bd
Add other functions
andreab1997 Feb 20, 2024
e1477a3
Add other functions again
andreab1997 Feb 20, 2024
d09b869
Init analysis
andreab1997 Feb 20, 2024
d6472a0
Add other functions
andreab1997 Feb 20, 2024
c2a5cdf
Add other functions again
andreab1997 Feb 20, 2024
f97b21c
Remove ipdbs and make sure multiclosure_inconsistent work
comane Jun 10, 2024
f12f26c
Add multiclosure comparefits
andreab1997 Feb 22, 2024
c78bfed
Add sklearn to pyproject
andreab1997 Feb 26, 2024
1bee235
Scikit added to conda
comane Jun 10, 2024
f637063
added PCA bias variance example to validphys2/examples
comane Mar 5, 2024
abd1d50
l2 condition number plot
comane Mar 5, 2024
e55bde3
added l2 condition number plot
comane Mar 5, 2024
50eaf37
removed unused functions pca_covmat, dataset_fits_gaussian_parameters
comane Mar 5, 2024
e182381
updated multiclosure analysis with l2 plots
comane Mar 5, 2024
c7d2e9a
variance pdf has to be taken from the same fits
comane Mar 5, 2024
74d7aa3
use np.nan and set default EVR to 0.99
comane Mar 5, 2024
5650418
cleaned bias variance table
comane Mar 5, 2024
56a000b
added docs to func + removed sklearn preprocessing as unused
comane Mar 5, 2024
e432d22
added single point dataset to example report
comane Mar 5, 2024
2dd06b8
added bootstrap of bias distribution
comane Mar 11, 2024
fc78ecb
added rbv plots as function of lambda
comane Mar 11, 2024
4e5acdf
added ratio bias variance to multi closure report
comane Mar 12, 2024
fb0e0ee
forgot ratio_bias_variance template
comane Mar 12, 2024
7e88307
fmt = 'o' for ratio bias variance plot
comane Mar 12, 2024
054fa44
added functions for computing Rbv using old definition + runcard with…
comane Mar 15, 2024
812e0a9
Piazza pulita 1: removed old plotting functions that were never used
comane Mar 17, 2024
e6f2ae3
Piazza pulita 2: removed principal_components_variance_distribution_d…
comane Mar 17, 2024
3326831
moved single data colorbar plots to multiclosure.py/_output.py
giovannidecrescenzo Mar 17, 2024
9c96b92
scikit-dep >=1.4.1 in pyproject file
comane Apr 11, 2024
ece05ff
use mean theory covmat for PCA -> gives more stable values of variance
comane Apr 22, 2024
e0f57f3
rewrote functions using new variance definition (covmat computed by a…
comane Jun 10, 2024
35435d8
removed sklearn dep from conda recipe meta file
comane Apr 23, 2024
212bed4
removed variancepdf as unused
comane Jun 10, 2024
6b813e4
added check_multifit_replicas check
comane Jun 10, 2024
415fcc6
use plotting dataset labels for rbv vs lambda titles
comane Jun 10, 2024
1847f2b
added hlines for rbv = 1
comane Jun 10, 2024
aa18c92
added bootstrapped_internal_multiclosure_dataset_loader for tuple of …
comane Jun 11, 2024
b787bde
bootstrap of PCA regularised multiclosure tests
comane Jun 11, 2024
2a9fd0f
bootstrap for internal_data_loadeer objecgs
comane Jun 11, 2024
904f89c
added bootstrapped_principal_components_bias_variance_dataset
comane Jun 12, 2024
9c6dc04
added bootstrap table to report
comane Jun 12, 2024
9c1c2f6
changed defaults of bootstrap
comane Jun 12, 2024
0968863
added title for single data point in latex mode
Jun 13, 2024
bf01c56
Add PCA on corr matrix on full dataset
andreab1997 Jun 18, 2024
c25aa55
fixed inconsistency with single data point and suggest different way …
giovannidecrescenzo Jun 19, 2024
4a3d02a
added delta plots
giovannidecrescenzo Jun 19, 2024
6b4b42e
use consistent bootstrap def and separate table datasets from table data
comane Jun 21, 2024
9acfad5
fixed PCA of correlation matrix for full dataset
comane Jun 21, 2024
79ccd7e
compute rbv scan using bootstrap uncertainty quantification
comane Jun 21, 2024
81fbd23
added full data bootstrapped table
comane Jun 25, 2024
9f950ea
slight change in def of delta hist
Jun 26, 2024
e07ad97
added definition of delta in line with eq. 2.22
comane Jun 26, 2024
333972f
added definition of delta in line with eq. 2.22
comane Jun 26, 2024
4ff4e90
added rbv scan for full dataset
comane Jun 26, 2024
4396181
added bootstrapped xi indicator function for full dataset
comane Jun 27, 2024
865cad3
removed unused import from vp_multiclosure.py script
comane Jul 24, 2024
390f50c
removed new lines
comane Jul 24, 2024
b0c6f3b
removed unused variables
comane Jul 24, 2024
63d9c7f
_covmats as array instead of list of arrays
comane Jul 24, 2024
7a3cc99
added eigendecomposition function
comane Jul 24, 2024
4c062cc
removed compareinconsistent closure templates
comane Oct 4, 2024
fba46de
removed vp_multiclosure module
comane Oct 4, 2024
d08d5ee
report template for multiclosure inconsistent tests
comane Oct 4, 2024
dfa63e0
removed vp-multiclosure from pyproject
comane Oct 4, 2024
42540bf
added and removed extra lines
comane Oct 4, 2024
9b2b255
removed extra line from congig
comane Oct 4, 2024
618e893
polished multiclosure.fits_normed_dataset_central_delta function
comane Oct 17, 2024
4595637
added meaningful description to single data point mapper
comane Oct 17, 2024
12f2f5d
update author to lazy person
comane Oct 17, 2024
dfceacb
loop over reps
comane Oct 17, 2024
6ba6bd1
Lazy Person
comane Oct 22, 2024
a1bff4c
removed multiclosure_inconsistent and moved to multiclosure
comane Oct 22, 2024
a63064a
removed multiclosure_inconsistent from __init__ file
comane Oct 22, 2024
4c5c612
addressed revision comments
comane Oct 22, 2024
25037d0
Init analysis
andreab1997 Feb 20, 2024
322c3d9
Add other functions
andreab1997 Feb 20, 2024
575b3bc
Add other functions again
andreab1997 Feb 20, 2024
20a35e3
Init analysis
andreab1997 Feb 20, 2024
223e2e1
Add other functions
andreab1997 Feb 20, 2024
dfdc425
Add other functions again
andreab1997 Feb 20, 2024
ba43562
Remove ipdbs and make sure multiclosure_inconsistent work
comane Jun 10, 2024
6c714c3
Add multiclosure comparefits
andreab1997 Feb 22, 2024
2890a4d
Add sklearn to pyproject
andreab1997 Feb 26, 2024
807ddd3
Scikit added to conda
comane Jun 10, 2024
ae53886
added PCA bias variance example to validphys2/examples
comane Mar 5, 2024
f4d3455
l2 condition number plot
comane Mar 5, 2024
f15f1b7
added l2 condition number plot
comane Mar 5, 2024
79bcef5
removed unused functions pca_covmat, dataset_fits_gaussian_parameters
comane Mar 5, 2024
0bd1a13
updated multiclosure analysis with l2 plots
comane Mar 5, 2024
55e5943
variance pdf has to be taken from the same fits
comane Mar 5, 2024
708136e
use np.nan and set default EVR to 0.99
comane Mar 5, 2024
52cad8f
cleaned bias variance table
comane Mar 5, 2024
63a2927
added docs to func + removed sklearn preprocessing as unused
comane Mar 5, 2024
c13b1cd
added single point dataset to example report
comane Mar 5, 2024
ec272a1
added bootstrap of bias distribution
comane Mar 11, 2024
004eec2
added rbv plots as function of lambda
comane Mar 11, 2024
356802d
added ratio bias variance to multi closure report
comane Mar 12, 2024
178ba5e
forgot ratio_bias_variance template
comane Mar 12, 2024
2ccb619
fmt = 'o' for ratio bias variance plot
comane Mar 12, 2024
af45c36
added functions for computing Rbv using old definition + runcard with…
comane Mar 15, 2024
0b04959
Piazza pulita 1: removed old plotting functions that were never used
comane Mar 17, 2024
56f8b0b
Piazza pulita 2: removed principal_components_variance_distribution_d…
comane Mar 17, 2024
733eda3
moved single data colorbar plots to multiclosure.py/_output.py
giovannidecrescenzo Mar 17, 2024
db65f79
scikit-dep >=1.4.1 in pyproject file
comane Apr 11, 2024
8b0602c
use mean theory covmat for PCA -> gives more stable values of variance
comane Apr 22, 2024
ca4f3b3
rewrote functions using new variance definition (covmat computed by a…
comane Jun 10, 2024
2b5db21
removed sklearn dep from conda recipe meta file
comane Apr 23, 2024
5bb7525
removed variancepdf as unused
comane Jun 10, 2024
e238d97
added check_multifit_replicas check
comane Jun 10, 2024
d9a7358
use plotting dataset labels for rbv vs lambda titles
comane Jun 10, 2024
a7c8126
added hlines for rbv = 1
comane Jun 10, 2024
fa82725
added bootstrapped_internal_multiclosure_dataset_loader for tuple of …
comane Jun 11, 2024
89a73b2
bootstrap of PCA regularised multiclosure tests
comane Jun 11, 2024
aeba820
bootstrap for internal_data_loadeer objecgs
comane Jun 11, 2024
80fb245
added bootstrapped_principal_components_bias_variance_dataset
comane Jun 12, 2024
198a757
added bootstrap table to report
comane Jun 12, 2024
36ccfd4
changed defaults of bootstrap
comane Jun 12, 2024
7f138eb
added title for single data point in latex mode
Jun 13, 2024
cf20ed1
Add PCA on corr matrix on full dataset
andreab1997 Jun 18, 2024
5d990b8
fixed inconsistency with single data point and suggest different way …
giovannidecrescenzo Jun 19, 2024
02ed9af
added delta plots
giovannidecrescenzo Jun 19, 2024
9eb99fa
use consistent bootstrap def and separate table datasets from table data
comane Jun 21, 2024
43f7e5f
fixed PCA of correlation matrix for full dataset
comane Jun 21, 2024
9775271
compute rbv scan using bootstrap uncertainty quantification
comane Jun 21, 2024
51b2e1d
added full data bootstrapped table
comane Jun 25, 2024
65848f0
slight change in def of delta hist
Jun 26, 2024
2b89c63
added definition of delta in line with eq. 2.22
comane Jun 26, 2024
998c652
added definition of delta in line with eq. 2.22
comane Jun 26, 2024
e56cfce
added rbv scan for full dataset
comane Jun 26, 2024
b4153a3
added bootstrapped xi indicator function for full dataset
comane Jun 27, 2024
38164d1
removed unused import from vp_multiclosure.py script
comane Jul 24, 2024
a659461
removed new lines
comane Jul 24, 2024
4fd4523
removed unused variables
comane Jul 24, 2024
1a912ab
_covmats as array instead of list of arrays
comane Jul 24, 2024
1f2912b
added eigendecomposition function
comane Jul 24, 2024
c6c0e07
removed compareinconsistent closure templates
comane Oct 4, 2024
366cd03
removed vp_multiclosure module
comane Oct 4, 2024
95270ed
report template for multiclosure inconsistent tests
comane Oct 4, 2024
153c2c6
removed vp-multiclosure from pyproject
comane Oct 4, 2024
308f528
added and removed extra lines
comane Oct 4, 2024
403944f
removed extra line from congig
comane Oct 4, 2024
1407fd3
polished multiclosure.fits_normed_dataset_central_delta function
comane Oct 17, 2024
d567175
added meaningful description to single data point mapper
comane Oct 17, 2024
1939417
update author to lazy person
comane Oct 17, 2024
ab2cbd3
loop over reps
comane Oct 17, 2024
7cc4eec
Lazy Person
comane Oct 22, 2024
7f8423f
removed multiclosure_inconsistent and moved to multiclosure
comane Oct 22, 2024
403e354
removed multiclosure_inconsistent from __init__ file
comane Oct 22, 2024
d83d620
addressed revision comments
comane Oct 22, 2024
6356c74
avoid rewriting columns twice
comane Oct 22, 2024
12ba99e
Merge branch '240305_multict_analysis' of https://github.com/NNPDF/nn…
comane Oct 22, 2024
6c4558b
avoid double column definition
comane Oct 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions validphys2/examples/pca_bias_variance_report.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
meta:
title: PCA report for bias and variance of multiclosure fits.
author: Lazy Person
keywords: [multiclosure, PCA, bias, variance]


dataset_inputs:
- {dataset: HERA_NC_251GEV_EP-SIGMARED, variant: legacy}
- {dataset: HERA_NC_300GEV_EP-SIGMARED, variant: legacy}
- {dataset: HERA_NC_318GEV_EP-SIGMARED, variant: legacy}
- {dataset: HERA_CC_318GEV_EM-SIGMARED, variant: legacy}


theoryid: 200
use_cuts: internal
use_t0: True
t0pdfset: 210223-mw-000_fakepdf


explained_variance_ratio: 0.99

##### Bootstrap parameters
n_fit_max: 2
n_fit: 10

n_rep_max: 100
n_rep: 60

n_boot_multiclosure: 100
rng_seed_mct_boot: 1


fits:
- 25_5_2023_19_47_5_dis_pt1_mnc_commit_4d5d473c_filterseed_415295
- 25_5_2023_19_49_38_dis_pt1_mnc_commit_4d5d473c_filterseed_120750

template_text: |

PCA report for bias and variance of multiclosure fits.
------------------------------------------------------

## Table of bias and variance
{@table_bias_variance_datasets@}


## Bootstrapped table for full dataset
{@bootstrapped_table_bias_variance_data@}

## L2 condition number
{@plot_l2_condition_number@}


actions_:
- report(main=true)
comane marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 2 additions & 0 deletions validphys2/src/validphys/closuretest/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

module containing all actions specific to closure test
"""

from validphys.closuretest.closure_plots import *
from validphys.closuretest.closure_results import *
from validphys.closuretest.multiclosure import *
Expand All @@ -11,3 +12,4 @@
from validphys.closuretest.multiclosure_pdf_output import *
from validphys.closuretest.multiclosure_preprocessing import *
from validphys.closuretest.multiclosure_pseudodata import *
from validphys.closuretest.inconsistent_closuretest.multiclosure_inconsistent_output import *
Loading