[NGT] Implement new seeding module for HLT Standalone Muon seeding and streamline HLT L3 Tracker Muon reconstruction #46897

Parsifal-2045 · 2024-12-09T13:36:52Z

PR description:

This PR implements a new Standalone (L2) Muon seeding strategy that fully utilizes the L1 Tracker Muons (L1TkMu) information.
L3 Tracker Muon reconstruction is also streamlined, also taking advantage of the improved L1TkMu, to reduce redundancy.
The full scope of the changes was described in the HLT Upgrade meeting on 8/10/2024, while the details on the integration were presented in the meeting on 3/12/2024.

The new L2 seeder directly matches the stubs used to produce each L1TkMu with segments in DTs/CSCs. All matched segments are assigned to the seed, together with the pT measurement from the L1TkMu. This approach produces a single collection of seeds.
On the other hand, the current implementation produces two collections of seeds (from L1TkMu and "offline") that require matching and produces roughly a factor of 5 more seeds in ZMM simulated events with 200 PU.
The changes implemented in the seeding module results in a significant improvement in timing (about 42%) with mostly compatible efficiency.

The streamlining of the L3 reconstruction hinges on the idea of performing either the Inside-Out or the Outside-In L3 Tracker Muon reconstruction first, to then execute a second pass only for candidates that require it (i.e. either they were not reconstructed in the first pass or the quality of the track was not good enough). This approach starts from either reconstruction path (flexible in case of future changes or detectors' requirements) to produce a filtered collection of L3 tracks: the filter uses some of the same criteria used in the HLT Muon ID step (e.g. number of hits in the pixel, number of hits in the tracker, normalized chi2, ...). The filtered L3 Tracks are geometrically matched to either L1TkMu (Outside-In first) or Standalone Muons (Inside-Out first):

if the match is successful, the track is considered good enough and passes to the next phase;
if the match is unsuccessful, the lower-level object will be reused to seed the second reconstruction path.
From the current results, the Inside-Out first approach seems to perform better, with more than 80% of the events in a ZMM sample at 200 PU requiring a single reconstruction path, to be compared with about 20% for the Outside-In first approach.
In any case, the streamlined L3 reconstruction reduces the fakerate in the merged (L3 IO + L3 OI) tracks collection as well as timing, while maintaining compatible physics perfomance until and including the final HLT Muon ID objects. All results, validation plots and ntuples are available here

The changes have been implemented using two new process modifiers:

phase2L2AndL3Muons implements the new Standalone Muon seeding and executes the L3 Tracker Muon reconstruction Inside-Out first;
phase2L3MuonsOIFirst to be used on top of the previous one, swaps the L3 Tracker Muon reconstruction from Inside-Out first to Outside-In first

These process modifiers have been included in a set of workflows targeting SingleMu, TTbar, and ZMM samples:

.777 includes the phase2L2AndL3Muons
.778 includes the phase2L2AndL3Muons and the phase2L3MuonsOIFirst process modifiers

PR validation:

Validation builds on top of #46860 introducing specific validation paths for the new objects (e.g. filtered collections, objects to reuse). Physics performance mostly compatible with the current implementation. Tested on CMSSW_14_2_0_pre3 RelVals (TTbar and ZMM) (not pre4 due to those being limited to 35 PU) as well as private production with workflows 29834(.0, .777, .778) (TTbar) and 29850 (.0, .777, .778) (ZMM).
Please test on Phase 2 workflows using also .777 and .778 modifiers

cmsbuild · 2024-12-09T13:37:17Z

cms-bot internal usage

cmsbuild · 2024-12-09T13:39:30Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46897/42945

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Add run3 tracking low pu era #33532, ECAL RecHit producer Alpaka migration #46453, Removing CUDA/gpu from Pixel code configs and dropping all CUDA wfs #46853

cmsbuild · 2024-12-09T13:39:59Z

A new Pull Request was created by @Parsifal-2045 for master.

It involves the following packages:

Configuration/EventContent (operations)
Configuration/ProcessModifiers (operations)
Configuration/PyReleaseValidation (pdmv, upgrade)
DataFormats/MuonSeed (reconstruction)
HLTrigger/Configuration (hlt)
RecoMuon/L2MuonSeedGenerator (hlt, reconstruction)
RecoMuon/L3TrackFinder (hlt, reconstruction)
Validation/RecoMuon (dqm)

@AdrianoDee, @Martin-Grunewald, @Moanwar, @antoniovagnerini, @antoniovilela, @cmsbuild, @davidlange6, @DickyChant, @fabiocos, @jfernan2, @mandrenguyen, @miquork, @mmusich, @rappoccio, @rseidita, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@CeliaFernandez, @Fedespring, @HuguesBrun, @Martin-Grunewald, @SohamBhattacharya, @VourMa, @abbiendi, @andrea21z, @bellan, @calderona, @cericeci, @fabiocos, @jhgoh, @makortel, @missirol, @mmusich, @rociovilar, @rovere, @silviodonato, @slomeo, @trocino this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

mmusich · 2024-12-09T14:02:24Z

test parameters:

workflow_opts= -w upgrade
workflow = 29834.0, 29834.777, 29834.778, 29850.0, 29850.777, 29850.778
enable = hlt_p2_integration, hlt_p2_timing

mmusich

These are just cosmetic comments from a quick transversal look.
I'll go through the dense C++ changes in the coming days.

A general request for clarification.
What is the plan to re-absorb phase2L3MuonsOIFirst and phase2L2AndL3Muons process modifiers in the mainstream configuration?

The question is not really for the PR proponent but I have a feeling we're diverging with the options for the phase-2 menu. @rovere @VourMa

...Configuration/python/HLT_75e33/modules/hltIter2Phase2L3FromL1TkMuonPixelSeedsFiltered_cfi.py

...figuration/python/HLT_75e33/modules/hltPhase2L3FromL1TkMuonPixelTracksTrackingRegions_cfi.py

HLTrigger/Configuration/python/HLT_75e33/modules/hltPhase2L3MuonMerged_cfi.py

RecoMuon/L2MuonSeedGenerator/src/Phase2L2MuonSeedCreator.cc

RecoMuon/L2MuonSeedGenerator/src/Phase2L2MuonSeedCreator.h

HLTrigger/Configuration/python/HLT_75e33/modules/hltPhase2L3MuonFilter_cfi.py

DataFormats/MuonSeed/src/classes_def.xml

RecoMuon/L2MuonSeedGenerator/src/Phase2L2MuonSeedCreator.h

HLTrigger/Configuration/python/HLT_75e33/modules/hltPhase2L3MuonFilter_cfi.py

HLTrigger/Configuration/python/HLT_75e33/modules/hltPhase2L3MuonMerged_cfi.py

mmusich · 2024-12-09T16:22:33Z

RecoMuon/L2MuonSeedGenerator/src/Phase2L2MuonSeedCreator.cc

+      mat[1][1] = 0.05 * 0.05;                                                     // sigma^2(lambda)
+      mat[2][2] = 0.2 * 0.2;                                                       // sigma^2(phi)
+      mat[3][3] = 20. * 20.;                                                       // sigma^2(x_transverse)
+      mat[4][4] = 20. * 20.;                                                       // sigma^2(y_transverse)


where are all these numbers coming from?

The propagation to the various muon stations have remained basically unchanged from the current L2 MuonSeedGenerator (seeded by L1Tk Muons)

should we consider storing these numbers in a single place for both modules? or is the old seed generator going to be discontinued?

The aim is for this new module to replace the current "legacy" seeding strategy. However, it's currently under a process modifier as a soft introduction while some slight inefficiencies are ironed out and its performance is assessed thoroughly. In any case, in the end, there should be a single Standalone Muon seed from L1Tk Muons generator

RecoMuon/L2MuonSeedGenerator/src/Phase2L2MuonSeedCreator.cc

HLTrigger/Configuration/python/HLT_75e33/modules/hltPhase2L3MuonMerged_cfi.py

HLTrigger/Configuration/python/HLT_75e33/modules/hltPhase2L3OISeedsFromL2Muons_cfi.py

mmusich · 2024-12-09T19:03:46Z

RecoMuon/L2MuonSeedGenerator/src/Phase2L2MuonSeedCreator.cc

+      continue;
+    }
+    float eta = l1TkMuRef->phEta();
+    float theta = 2 * std::atan(std::exp(-eta));


this pattern together with the usage of ThreeVector from CLHEP smells.
What about something possibly more efficient reducing the usage of trig functions along the lines of 6272d52 ?

The latest commit should address this. It's still computing theta (only once per L1TkMu) since it's used for the stub-segment matching, but everything has been refactored to result in fewer overall "taxing" computations

- Takes full advantage of the Phase 2 L1 Tracker Muon information - Direct match between L1 stubs and segments in DTs/CSCs - Extract stubs from L1Tk Muon, only checks segments in the same chamber as the stub to match - Match based on dphi, dtheta and number of hits in the segment - In the barrel, if no match is found in a given station, a rough extrapolation is attempted from the closest station with a match - At most one seed per L1Tk Muon produced in a single seeding step, with the possibility to extend to exotic signatures

- execute either Inside-Out or Outside-In reconstruction first, resorting to the second pass only for candidates that require it - Tracks filtered by quality using some of the same criteria as HLT Muon ID - L1/L2 objects matched with bad L3 tracks are reused to seed the second reconstruction step - Changes implemented via 2 procModifiers for IO first (default) and OI first reconstruction - Implemented changes in SingleMu, TTbar, and ZMM workflows with .777 and .778 subfixes for IO and OI first reconstruction, respectively - Change 2026 to Run4 in Phase 2 workflows

cmsbuild · 2024-12-10T18:30:01Z

+1

Size: This PR adds an extra 52KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-56c0a9/43358/summary.html
COMMIT: 37b3b6f
CMSSW: CMSSW_15_0_X_2024-12-10-1100/el8_amd64_gcc12
Additional Tests: HLT_P2_INTEGRATION,HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46897/43358/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.
HLT P2 Timing: chart

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3510017
DQMHistoTests: Total failures: 475
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3509522
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 202 log files, 172 edm output root files, 46 DQM output files
TriggerResults: found differences in 1 / 44 workflows

mmusich · 2024-12-11T14:25:56Z

The new L2 seeder directly matches the stubs used to produce each L1TkMu with segments in DTs/CSCs.

when I try to run the phase-2 integration tests with the command hltPhase2UpgradeIntegrationTests --procModifiers phase2L2AndL3Muons I am getting crashes of the kind:

----- Begin Fatal Exception 11-Dec-2024 15:14:38 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 0
   [1] Running path 'HLT_PFHT330PT30_QuadPFPuppiJet_75_60_45_40_TriplePFPuppiBTagDeepFlavour_2p4'
   [2] Calling method for module Phase2L2MuonSeedCreator/'hltL2MuonSeedsFromL1TkMuon'
Exception Message:
RefCore: A request to resolve a reference to a product of type 'std::vector<l1t::MuonStub>' with ProductID '1:379'
can not be satisfied because the product cannot be found.
Probably the branch containing the product is not stored in the input file.
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

this means that at the moment we're not saving in the phase2 GEN-SIM-DIGI-RAW data-tier the needed collection for the seeding after running the steps L1TrackTrigger,L1,L1P2GT,DIGI2RAW .

The tests generates the RAW data on the fly via

cmsDriver.py TTbar_14TeV_TuneCP5_cfi -s GEN,SIM,DIGI:pdigi_valid,L1TrackTrigger,L1,L1P2GT,DIGI2RAW -n 10 --conditions auto:phase2_realistic_T33 --beamspot DBrealisticHLLHC --datatier GEN-SIM-DIGI-RAW --eventcontent FEVTDEBUG --geometry ExtendedRun4D110 --era Phase2C17I13M9 --relval 9000,100 --fileout hlt_test_configs/step1.root --nThreads 1 --customise SLHCUpgradeSimulations/Configuration/aging.customise_aging_1000 --python_filename hlt_test_configs/TTbar_GEN_SIM_step.py

should the process modifier trigger the saving of the collection in the FEVTDEBUG content as well?

Parsifal-2045 · 2024-12-11T15:41:56Z

The new L2 seeder directly matches the stubs used to produce each L1TkMu with segments in DTs/CSCs.

when I try to run the phase-2 integration tests with the command hltPhase2UpgradeIntegrationTests --procModifiers phase2L2AndL3Muons I am getting crashes of the kind:

----- Begin Fatal Exception 11-Dec-2024 15:14:38 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 0
   [1] Running path 'HLT_PFHT330PT30_QuadPFPuppiJet_75_60_45_40_TriplePFPuppiBTagDeepFlavour_2p4'
   [2] Calling method for module Phase2L2MuonSeedCreator/'hltL2MuonSeedsFromL1TkMuon'
Exception Message:
RefCore: A request to resolve a reference to a product of type 'std::vector<l1t::MuonStub>' with ProductID '1:379'
can not be satisfied because the product cannot be found.
Probably the branch containing the product is not stored in the input file.
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

this means that at the moment we're not saving in the phase2 GEN-SIM-DIGI-RAW data-tier the needed collection for the seeding after running the steps L1TrackTrigger,L1,L1P2GT,DIGI2RAW .

The tests generates the RAW data on the fly via

cmsDriver.py TTbar_14TeV_TuneCP5_cfi -s GEN,SIM,DIGI:pdigi_valid,L1TrackTrigger,L1,L1P2GT,DIGI2RAW -n 10 --conditions auto:phase2_realistic_T33 --beamspot DBrealisticHLLHC --datatier GEN-SIM-DIGI-RAW --eventcontent FEVTDEBUG --geometry ExtendedRun4D110 --era Phase2C17I13M9 --relval 9000,100 --fileout hlt_test_configs/step1.root --nThreads 1 --customise SLHCUpgradeSimulations/Configuration/aging.customise_aging_1000 --python_filename hlt_test_configs/TTbar_GEN_SIM_step.py

should the process modifier trigger the saving of the collection in the FEVTDEBUG content as well?

I assumed that the stubs were already saved in the L1 Trigger Event content, which was the case. However, their name seems to have changed from l1tTkStubsGmt to l1tStubsGmt. Updating the name in L1Trigger/Configuration/L1Trigger_EventContent_cff.py solves the issue for me. Now the question is should I add this name change to the PR?

mmusich · 2024-12-11T15:43:20Z

Now the question is should I add this name change to the PR?

I think so. @cms-sw/l1-l2 FYI

cmsbuild · 2024-12-11T15:49:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46897/42973

Found files with invalid states:
- RecoMuon/L3TrackFinder/interface/phase2HLTMuonSelectorForL3.h:
  - Added: 9d7592f
  - Deleted: 37b3b6f
- RecoMuon/L3TrackFinder/src/phase2HLTMuonSelectorForL3.cc:
  - Added: 9d7592f
  - Deleted: 37b3b6f
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Add run3 tracking low pu era #33532, ECAL RecHit producer Alpaka migration #46453, Removing CUDA/gpu from Pixel code configs and dropping all CUDA wfs #46853

cmsbuild · 2024-12-11T15:49:48Z

Pull request #46897 was updated. @AdrianoDee, @Martin-Grunewald, @Moanwar, @aloeliger, @antoniovagnerini, @antoniovilela, @cmsbuild, @davidlange6, @DickyChant, @epalencia, @fabiocos, @jfernan2, @mandrenguyen, @miquork, @mmusich, @rappoccio, @rseidita, @srimanob, @subirsarkar can you please check and sign again.

mmusich · 2024-12-11T16:00:37Z

I am running other (private) tests in parallel -- in case you want to start main tests feel free to do it @Parsifal-2045

Parsifal-2045 · 2024-12-11T16:01:01Z

@cmsbuild please test

cmsbuild · 2024-12-12T21:35:37Z

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-56c0a9/43387/summary.html
COMMIT: af19b58
CMSSW: CMSSW_15_0_X_2024-12-11-0800/el8_amd64_gcc12
Additional Tests: HLT_P2_INTEGRATION,HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46897/43387/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.
HLT P2 Timing: chart

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3510017
DQMHistoTests: Total failures: 453
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3509544
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 202 log files, 172 edm output root files, 46 DQM output files
TriggerResults: found differences in 1 / 44 workflows

cmsbuild added this to the CMSSW_15_0_X milestone Dec 9, 2024

cmsbuild added reconstruction-pending dqm-pending hlt-pending operations-pending pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending code-checks-pending changes-dataformats labels Dec 9, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Dec 9, 2024

cmsbuild mentioned this pull request Dec 9, 2024

Removing CUDA/gpu from Pixel code configs and dropping all CUDA wfs #46853

Open

mmusich reviewed Dec 9, 2024

View reviewed changes

Moanwar reviewed Dec 9, 2024

View reviewed changes

DataFormats/MuonSeed/src/classes_def.xml Show resolved Hide resolved

cmsbuild mentioned this pull request Dec 9, 2024

ECAL RecHit producer Alpaka migration #46453

Open

mmusich reviewed Dec 9, 2024

View reviewed changes

Parsifal-2045 added 4 commits December 10, 2024 11:07

Implement validation for new Phase 2 muon workflows

2eab640

Formatting muon sequences

4762785

Parsifal-2045 force-pushed the Phase2Muons branch from a979b05 to 4762785 Compare December 10, 2024 10:11

cmsbuild removed the code-checks-approved label Dec 10, 2024

cmsbuild added the tests-started label Dec 10, 2024

cmsbuild added operations-approved tests-approved and removed operations-pending tests-started labels Dec 10, 2024

Update L1 Muon stubs naming to save them in FEVT

af19b58

cmsbuild added l1-pending operations-pending tests-pending code-checks-pending and removed operations-approved tests-approved code-checks-approved labels Dec 11, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Dec 11, 2024

cmsbuild added tests-started and removed tests-pending labels Dec 11, 2024

cmsbuild added operations-approved tests-approved and removed operations-pending tests-started labels Dec 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[NGT] Implement new seeding module for HLT Standalone Muon seeding and streamline HLT L3 Tracker Muon reconstruction #46897

[NGT] Implement new seeding module for HLT Standalone Muon seeding and streamline HLT L3 Tracker Muon reconstruction #46897

Parsifal-2045 commented Dec 9, 2024

cmsbuild commented Dec 9, 2024 •

edited

Loading

cmsbuild commented Dec 9, 2024

cmsbuild commented Dec 9, 2024

mmusich commented Dec 9, 2024

mmusich left a comment •

edited

Loading

mmusich Dec 9, 2024

Parsifal-2045 Dec 10, 2024

mmusich Dec 10, 2024

Parsifal-2045 Dec 10, 2024

mmusich Dec 9, 2024

Parsifal-2045 Dec 10, 2024

cmsbuild commented Dec 10, 2024

mmusich commented Dec 11, 2024 •

edited

Loading

Parsifal-2045 commented Dec 11, 2024 •

edited

Loading

mmusich commented Dec 11, 2024 •

edited

Loading

cmsbuild commented Dec 11, 2024

cmsbuild commented Dec 11, 2024

mmusich commented Dec 11, 2024

Parsifal-2045 commented Dec 11, 2024

cmsbuild commented Dec 12, 2024

[NGT] Implement new seeding module for HLT Standalone Muon seeding and streamline HLT L3 Tracker Muon reconstruction #46897

Are you sure you want to change the base?

[NGT] Implement new seeding module for HLT Standalone Muon seeding and streamline HLT L3 Tracker Muon reconstruction #46897

Conversation

Parsifal-2045 commented Dec 9, 2024

PR description:

PR validation:

cmsbuild commented Dec 9, 2024 • edited Loading

cmsbuild commented Dec 9, 2024

cmsbuild commented Dec 9, 2024

mmusich commented Dec 9, 2024

mmusich left a comment • edited Loading

Choose a reason for hiding this comment

mmusich Dec 9, 2024

Choose a reason for hiding this comment

Parsifal-2045 Dec 10, 2024

Choose a reason for hiding this comment

mmusich Dec 10, 2024

Choose a reason for hiding this comment

Parsifal-2045 Dec 10, 2024

Choose a reason for hiding this comment

mmusich Dec 9, 2024

Choose a reason for hiding this comment

Parsifal-2045 Dec 10, 2024

Choose a reason for hiding this comment

cmsbuild commented Dec 10, 2024

Comparison Summary

mmusich commented Dec 11, 2024 • edited Loading

Parsifal-2045 commented Dec 11, 2024 • edited Loading

mmusich commented Dec 11, 2024 • edited Loading

cmsbuild commented Dec 11, 2024

cmsbuild commented Dec 11, 2024

mmusich commented Dec 11, 2024

Parsifal-2045 commented Dec 11, 2024

cmsbuild commented Dec 12, 2024

Comparison Summary

cmsbuild commented Dec 9, 2024 •

edited

Loading

mmusich left a comment •

edited

Loading

mmusich commented Dec 11, 2024 •

edited

Loading

Parsifal-2045 commented Dec 11, 2024 •

edited

Loading

mmusich commented Dec 11, 2024 •

edited

Loading