Set up repo for 2023 analysis and further fixes #8
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces several important updates to the configuration, calibration, and selection for the top tagging scale factor and working point analyses, with a focus on extending support to 2023 campaigns, fixing jet calibration, including the jetId selection, and fixing file system and workflow configuration. Also, columnflow and the cmsdb are updated.
Campaign and Configuration Expansion
analysis_sf.py) and working point (analysis_wp.py) configurations, including standard, limited, and medium-limited file sets. This includes un-commenting and enabling previously disabled code, and updating the number of limited files from 1 to 2 to also test merging tasks properly.add_configlogic to recognize 2023 as an implemented year, allowing the new campaigns to be processed without raising errors.Calibration and Jet Handling Improvements
topsf/calibration/default.pyto use explicit JEC/JER calibrators (jec_ak4,jer_ak4, etc.) instead of the genericjets_ak4/jets_ak8. The previous implementation only ran one of thejetscalibrators as the subclassing wasn't working as expected within columnflow. In addition, MET phi calibrations are now included also for Run 3 analyses as the needed corrections are now available.Configuration and Workflow Enhancements
The following changes came about when issues with writing on dcache were present in these analyses but not in the the mtt analysis. Therefor the
law.cfgfiles of these analyses were aligned further and lead to successful writing on the remote filesystem again (with occasional gfal2 errors).law.cfg, and expanded the list of file systems for output, improving compatibility and performance for distributed workflows.Dataset and Subprocess List Updates
Dependency Updates
cmsdbandcolumnflowto newer commits, ensuring compatibility with the above changes.Summary
The current version of the code to be merged in this PR now includes the setup for the derivation of cut based top tagging scale factors and working points for the data taking eras
22preEE,22postEE,23preBPix, and23postBPix. As far as I'm aware, all calibration and selections steps are in place, allowing for rather quick creation of the necessary histograms for fitting. The combine fitting tasks are still not fixed and need to be called with separate scripts.Next, I'm planning to restructure the way the configs are written by using different scripts for the different aspects needed: datasets, selection parameters, corrections, SF, etc. This will make the configs easier to maintain as there's ideally one place to adjust things which should be easier to find. The necessary changes won't be part of this PR though and will be their own PR.