Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NGT] Implement NanoAOD ntuples production for HLT Muon objects #47069

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Parsifal-2045
Copy link
Contributor

PR description:

This PR allows to produce, on-demand, ntuples in NanoAOD format for all HLT muon objects, including L1TkMuons and all the new collections introduced by #46897.
The production is triggered automatically when running .777 or .778 workflows. Otherwise, it can be triggered by adding NANO:@MUHLT as a step in any step2 cmsDriver command, together with the addition of NANOAODSIM in both datatier and eventcontent.

PR validation:

No physics changes are foreseen.
I've tested on the usual 29834 and 29850 workflows also in their .777 and .778 flavours, in all cases the workflows run and ntuples are produced correctly
.0 workflows require to manually trigger the nutples production using, for example:

cmsDriver.py step2 \
-s DIGI:pdigi_valid,L1TrackTrigger,L1,L1P2GT,DIGI2RAW,HLT:@relvalRun4,NANO:@MUHLT \
--conditions auto:phase2_realistic_T33 \
--datatier GEN-SIM-DIGI-RAW,NANOAODSIM \
--eventcontent FEVTDEBUGHLT,NANOAODSIM \
--geometry ExtendedRun4D110 \
--era Phase2C17I13M9 \
--filein file:step1.root \
--fileout file:step2.root \
-n 10

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2025

cms-bot internal usage

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2025

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47069/43241

  • Found files with invalid states:
    • DPGAnalysis/MuonTools/python/hlt_nano_phase2_muon_trig_cff.py:

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2025

A new Pull Request was created by @Parsifal-2045 for master.

It involves the following packages:

  • Configuration/EventContent (operations)
  • Configuration/PyReleaseValidation (upgrade, pdmv)
  • DPGAnalysis/MuonTools (xpog)
  • PhysicsTools/NanoAOD (xpog)

@AdrianoDee, @Moanwar, @antoniovilela, @cmsbuild, @davidlange6, @DickyChant, @fabiocos, @ftorrresd, @hqucms, @mandrenguyen, @miquork, @rappoccio, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@AnnikaStein, @Martin-Grunewald, @battibass, @fabiocos, @gpetruc, @makortel, @missirol, @mmusich, @slomeo this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@mmusich
Copy link
Contributor

mmusich commented Jan 9, 2025

type ngt

@ftorrresd
Copy link
Contributor

enable nano

@cmsbuild cmsbuild added the ngt label Jan 9, 2025
@ftorrresd
Copy link
Contributor

please test

@@ -702,6 +702,7 @@ def SwapKeepAndDrop(l):
phase2_muon.toModify(FEVTDEBUGHLTEventContent,
outputCommands = FEVTDEBUGHLTEventContent.outputCommands + [
'keep recoMuons_muons1stStep_*_*',
'keep *_hltL2OfflineMuonSeeds_*_*',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need to be saved regardless or can it be (for the time being) gated through the phase2L2AndL3Muons modifier?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's actually the other way around. The collection is not produced when using the phase2L2AndL3Muons procModifier, but I decided to produce ntuples also for the both the current muon seeds collections, namely the "offline" one and the one from L1Tk Muons (at least for now)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The collection is not produced when using the phase2L2AndL3Muons procModifier,

I then suggest to apply the reverse modifier, that way it will be easier to remove once it's re-absorbed.

Copy link
Contributor Author

@Parsifal-2045 Parsifal-2045 Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The collection is not produced when using the phase2L2AndL3Muons procModifier,

I then suggest to apply the reverse modifier, that way it will be easier to remove once it's re-absorbed.

What's the correct syntax for that? I now have (phase2Muon and not phase2L3AndL3Muons).toModify...

Copy link
Contributor

@mmusich mmusich Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the syntax for that?

(phase2Muon & ~phase2L2AndL3Muons).toModify(...)

(OTOH, I am not formally a reviewer of this PR, so I let the @cms-sw/operations-l2 decide what's best -- incidentally ever increasing the size of FEVTDEBUGHLTEventContent will inevitably lead to operation issues when trying to run big MC samples production )

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2025

+1

Size: This PR adds an extra 60KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26edf9/43701/summary.html
COMMIT: f94c6ee
CMSSW: CMSSW_15_0_X_2025-01-09-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47069/43701/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 3 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3818730
  • DQMHistoTests: Total failures: 79
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3818631
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 214 log files, 184 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially removed 212 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 21
  • DQMHistoTests: Total histograms compared: 74922
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 74922
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 20 files compared)
  • Checked 105 log files, 60 edm output root files, 21 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.001 3.076 3.076 0.000 ( +0.0% ) 5.99 5.02 +19.2% 2.551 2.550
2500.002 3.192 3.192 0.000 ( +0.0% ) 5.38 4.49 +19.9% 2.983 2.978
2500.003 3.133 3.133 0.000 ( +0.0% ) 5.62 4.65 +21.0% 2.961 2.962
2500.011 1.644 1.644 0.000 ( +0.0% ) 9.17 7.63 +20.1% 2.618 2.635
2500.012 2.184 2.184 0.000 ( +0.0% ) 5.58 4.69 +19.1% 2.804 2.812
2500.013 1.999 1.999 0.000 ( +0.0% ) 7.65 6.43 +19.0% 2.723 2.725
2500.021 0.022 0.022 0.000 ( +0.0% ) 2.02 1.46 +38.4% 2.604 2.595
2500.022 0.022 0.022 0.000 ( +0.0% ) 1.93 1.41 +37.0% 2.594 2.594
2500.023 0.022 0.022 0.000 ( +0.0% ) 1.84 1.40 +31.5% 2.460 2.467
2500.024 0.022 0.022 0.000 ( +0.0% ) 1.52 1.17 +29.7% 2.695 2.694
2500.031 0.035 0.035 0.000 ( +0.0% ) 1.76 1.36 +29.6% 2.661 2.650
2500.032 0.036 0.036 0.000 ( +0.0% ) 1.75 1.34 +30.6% 2.612 2.612
2500.033 0.037 0.037 0.000 ( +0.0% ) 1.68 1.25 +34.2% 2.704 2.703
2500.034 0.036 0.036 0.000 ( +0.0% ) 1.67 1.26 +32.8% 2.683 2.677
2500.101 2.803 2.803 0.000 ( +0.0% ) 13.39 11.94 +12.1% 2.624 2.636
2500.111 1.462 1.462 0.000 ( +0.0% ) 27.04 23.01 +17.5% 2.319 2.332
2500.112 1.882 1.882 0.000 ( +0.0% ) 19.17 19.47 -1.5% 2.396 2.400
2500.131 0.750 0.750 0.000 ( +0.0% ) 32.52 28.24 +15.1% 1.506 1.513
2500.201 2.637 2.637 0.000 ( +0.0% ) 12.09 9.66 +25.1% 2.198 2.203
2500.211 1.805 1.805 0.000 ( +0.0% ) 23.55 19.93 +18.2% 2.391 2.391
2500.212 2.202 2.202 0.000 ( +0.0% ) 17.20 16.18 +6.3% 2.482 2.481
2500.221 2.037 2.037 0.000 ( +0.0% ) 10.53 10.18 +3.4% 2.121 2.123
2500.222 3.440 3.440 0.000 ( +0.0% ) 10.79 9.38 +15.0% 2.214 2.215
2500.223 9.406 9.406 0.000 ( +0.0% ) 3.72 3.13 +18.8% 2.289 2.292
2500.224 6.265 6.265 0.000 ( +0.0% ) 1.30 1.07 +21.9% 2.277 2.220
2500.225 6.312 6.312 0.000 ( +0.0% ) 1.21 1.00 +21.3% 2.486 2.445
2500.226 3.134 3.134 0.000 ( +0.0% ) 12.00 10.44 +15.0% 2.211 2.204
2500.227 1.442 1.442 0.000 ( +0.0% ) 22.20 18.32 +21.2% 1.445 1.440
2500.228 3.921 3.921 0.000 ( +0.0% ) 8.64 6.90 +25.2% 2.301 2.301
2500.231 1.455 1.455 0.000 ( +0.0% ) 21.36 17.17 +24.4% 2.290 2.291
2500.232 2.461 2.461 0.000 ( +0.0% ) 19.46 14.87 +30.8% 2.396 2.394
2500.233 4.953 4.953 0.000 ( +0.0% ) 5.87 4.62 +27.2% 2.471 2.473
2500.234 3.841 3.841 0.000 ( +0.0% ) 1.68 1.38 +21.7% 2.425 2.184
2500.235 3.872 3.872 0.000 ( +0.0% ) 1.57 1.29 +21.9% 2.617 2.368
2500.236 2.251 2.251 0.000 ( +0.0% ) 20.76 15.77 +31.6% 2.385 2.391
2500.237 1.018 1.018 0.000 ( +0.0% ) 33.07 27.27 +21.3% 1.452 1.453
2500.238 2.443 2.443 0.000 ( +0.0% ) 16.28 13.61 +19.6% 2.479 2.473
2500.241 9.404 9.404 0.000 ( +0.0% ) 7.05 5.47 +28.9% 1.932 1.925
2500.242 10.331 10.331 0.000 ( +0.0% ) 1.60 1.35 +18.5% 1.728 1.726
2500.243 2.712 2.712 0.000 ( +0.0% ) 15.34 11.54 +32.9% 1.070 1.065
2500.244 486.016 486.016 0.000 ( +0.0% ) 1.09 0.89 +22.9% 1.684 1.685
2500.245 826.413 826.413 0.000 ( +0.0% ) 1.45 1.13 +28.2% 1.667 1.690
2500.901 1.778 1.778 0.000 ( +0.0% ) 43.33 33.45 +29.5% 1.444 1.442
2500.902 1.628 1.628 0.000 ( +0.0% ) 45.30 37.83 +19.8% 1.336 1.334
2500.911 14.041 14.041 0.000 ( +0.0% ) 8.02 6.90 +16.2% 1.091 1.081
2500.912 0.199 0.729 -0.530 ( -72.7% ) 3.04 1.93 +57.7% 0.851 0.846
2500.913 0.110 0.110 0.000 ( +0.0% ) 2.57 1.99 +29.3% 0.851 0.850

@Moanwar
Copy link
Contributor

Moanwar commented Jan 12, 2025

+Upgrade

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants