Patatrack integration - Pixel workflows (12/N) (#31854)

* Add workflows for Riemann fit and GPU (cms-patatrack#20) * add `riemannFit` and `gpu` modifiers and workflows for Riemann fit and GPU modules * switch GPU modules with `gpu` modifier * Add a DQM sequence for pixel-only tracking (cms-patatrack#23) * add a DQM sequence for pixel-only tracking * add pixelTrackingOnlyDQM sequences to the pixelTrackingOnly workflows * add pixelTrackingOnlyDQM sequences to the riemannFit and gpu workflows * Synchronise with CMSSW_10_1_0_pre2 * Synchronise with CMSSW_10_1_0_pre3 * Synchronise with CMSSW_10_1_0 * Synchronise with CMSSW_10_2_0_pre1 * Synchronise with CMSSW_10_2_0_pre2 * Synchronise with CMSSW_10_2_0_pre3 * Synchronise with CMSSW_10_2_0_pre4 * Synchronise with CMSSW_10_2_0_pre5 * Port the Riemann fit to CUDA (cms-patatrack#60) - the CPU Riemann fit works using all combinations between the 2 booleans: `useErrors` and `useMultipleScattering`; - the standalone version of the GPU Riemann fit has been updated in order to explore all possibilities among the 2 booleans above: all of them work and produce identical results up to 1e-5 precision (the default one, 1e-6 fails when enabling multiScattering, most likely due to matrix inversions); - the GPU version of the Riemann fit within CMSSW works, with 1 fit assigned to each thread, with 32 threads/warps, all dynamically computed. Things that needs a "hack": - limit the "dynamic" size of Eigen matrices to at most, 4x4, which is just fine for quadruplets. Using anything wider will cause errors which I *believe* is related to the stack size of threads on the GPU; - cast matrices to be inverted to 4x4 (was done before the previous point: will revert it back and see if that's still needed or not, but I believe it is); this was done in order to "specialize" the `invert()` call to something that is "natively" supported by Eigen on GPU (that brought in also few `__host__` `__device__` here and there in Eigen); - fix the alignment of the `struct` holding the results of the fit, since its size was different on GPU and CPU, causing an annoying off-by-one effect. * Synchronise with CMSSW_10_2_0_pre6 * update relval input strings (102X) * Synchronise with CMSSW_10_2_0 * adding NANO step to 2018 MC workflows * Merge pull request #24024 from prebello/from-CMSSW_10_2_X_2018-07-24-2300 update relval input strings (102X) * Synchronise with CMSSW_10_2_2_patch1 * activate the new SiStripCalSiStripCalBiasScan in the matrix test * Add pixel tracking workflows for data (cms-patatrack#144) * add HEM-15-16 relval workflows * introduce RelVal for hybrid emulation (with ZS10 repacking),HLT-like zero-suppression (with normal ZS repacking),RECO of 2015 PbPb VirginRaw data * add 3 production workflows for premix2018 * Merge pull request #24550 from zhenhu/add-HEM-15-16-relvals_102X [10.2.X] add HEM-15-16 relval workflows * Merge pull request #24618 from zhenhu/addPremixProdWfs_102X backport [10.2.X] add 3 production workflows for premix2018 * Synchronise with CMSSW_10_2_5 * Synchronise with CMSSW_10_2_6 * Synchronise with CMSSW_10_4_0_pre2 * Synchronise with CMSSW_10_4_0_pre3 * Synchronise with CMSSW_10_4_0_pre4 * Synchronise with CMSSW_10_4_0 * Change GPU workflow numbering: .7->.51, .8->.52, .9->.53 (cms-patatrack#259) * Synchronise with CMSSW_10_5_0_pre1 * Synchronise with CMSSW_10_5_0_pre2 * Synchronise with CMSSW_10_6_0_pre2 * Rework the Riemann fit and broken line fit (cms-patatrack#338) Merge the Riemann and broken line fits into single configurable pixel n-tuplet fitter, and extend it to work with up to 5 hits. Mmake the broken line fit the default algorithm. Try both triplets and quadruplets in the pixel "hole". Limit pT used to compute the multple scattering. Use the inline Cholesky decomposition. Generic clean up and improvements. * Synchronise with CMSSW_10_6_0 * Synchronise with CMSSW_11_0_0_pre7 * Fix broken merge in Configuration/PyReleaseValidation * Synchronise with CMSSW_11_0_0_pre11 * Update and extend the Patatrack workflows Update the definition of the Patatrack workflows to avoid conflicts in CMSSW_11_0_0_pre12: - renumber to avoid conflicts with #27392 - reimplement the definition following #28281 - add Z->mumu 2018 realistic workflows - add TTbar and Z->mumu 2021 realistic workflows - move the data workflows to 2018D, HLTPhysics and JetHT * Synchronise with CMSSW_11_0_0_pre13 * Backport: add ECAL-only workflows for 2018 data (#30105) * Add tracking-only and pixel tracking-only workflows for Run 3 (#30338) * Backport: add ECAL-only and HCAL-only workflows for MC and data (#30350) Backport #30105: add ECAL-only workflows for data. Backport #30136: add HCAL-only workflows for MC and data. * Synchronise with CMSSW_11_2_0_pre2 * Update the RelVal workflows and the CPU customisation (cms-patatrack#549) Update the RelVal workflows and the CPU customisation: - change the .501 workflow to run the full Patatrack pixel track reconstruction on CPU - add a customisation to run the Patatrack reconstruction with triplets, on CPU and GPU - add the .505 and .506 workflows to reconstruct triplets, on CPU and GPU Co-authored-by: Andrea Bocci <[email protected]> * Synchronise with CMSSW_11_2_0_pre8 * Synchronise with CMSSW_11_2_0_pre9 * Synchronise with CMSSW_11_2_0_pre10 * Backport: add RecoFakeHLT steps to 2018 Patatrack WFs to enable GPU modifiers (#32333) The 2018 GPU matrix WFs (10824.502, 10824.512, 10824.522, 10842.502) produced the same cmsDriver configurations as their CPU counterparts. This PR fixes this by setting the gpu modifier also for the RecoFakeHLT step. * Synchronise with CMSSW_11_3_0_pre1 * Synchronise with CMSSW_11_3_X_2020-12-24-2300 * Document the new Pixel workflows Co-authored-by: Matti Kortelainen <[email protected]> Co-authored-by: Marco Rovere <[email protected]> Co-authored-by: Patricia <[email protected]> Co-authored-by: Francesco <[email protected]> Co-authored-by: cmsbuild <[email protected]> Co-authored-by: mmusich <[email protected]> Co-authored-by: zhen <[email protected]> Co-authored-by: Vincenzo Innocente <[email protected]> Co-authored-by: Mariarosaria D'Alfonso <[email protected]> Co-authored-by: AdrianoDee <[email protected]> Co-authored-by: Thomas Reis <[email protected]>
cms-sw · Apr 12, 2021 · d6fb636 · d6fb636
1 parent 807000a
commit d6fb636
Show file tree

Hide file tree

Showing 5 changed files with 77 additions and 23 deletions.
diff --git a/Configuration/PyReleaseValidation/README.md b/Configuration/PyReleaseValidation/README.md
@@ -31,6 +31,14 @@ The offsets currently in use are:
 * 0.3: 0.1 + 0.2
 * 0.4: LowPU tracking era, `Run2_2017_trackingLowPU`
 * 0.5: Pixel tracking only + 0.1
+* 0.501: Patatrack, pixel only quadruplets, on CPU
+* 0.502: Patatrack, pixel only quadruplets, on GPU
+* 0.505: Patatrack, pixel only triplets, on CPU
+* 0.506: Patatrack, pixel only triplets, on GPU
+* 0.511: Patatrack, ECAL only CPU
+* 0.512: Patatrack, ECAL only GPU
+* 0.521: Patatrack, HCAL only CPU
+* 0.522: Patatrack, HCAL only GPU
 * 0.6: HE Collapse (old depth segmentation for 2018)
 * 0.7: trackingMkFit modifier
 * 0.8: BPH Parking (Run-2)
@@ -52,10 +60,4 @@ The offsets currently in use are:
 * 0.911: DD4Hep
 * 0.101: Phase-2 aging, 1000fb-1
 * 0.103: Phase-2 aging, 3000fb-1
-* 0.501: Patatrack, pixel only CPU
-* 0.502: Patatrack, pixel only GPU
-* 0.511: Patatrack, ECAL only CPU
-* 0.512: Patatrack, ECAL only GPU
-* 0.521: Patatrack, HCAL only CPU
-* 0.522: Patatrack, HCAL only GPU
 * 0.9001: Sonic Triton
diff --git a/Configuration/PyReleaseValidation/python/relval_2017.py b/Configuration/PyReleaseValidation/python/relval_2017.py
@@ -1,4 +1,3 @@
-
 # import the definition of the steps and input files:
 from  Configuration.PyReleaseValidation.relval_steps import *
 
@@ -24,16 +23,16 @@
 #        (HE collapse: TTbar, TTbar PU, TTbar design)
 #        (ParkingBPH: TTbar)
 #        (TTbar PU with JME NanoAOD)
-#        (Patatrack pixel-only: ZMM - on CPU)
-#        (Patatrack pixel-only: TTbar - on CPU)
+#        (Patatrack pixel-only: ZMM - on CPU: quadruplets, triplets)
+#        (Patatrack pixel-only: TTbar - on CPU: quadruplets, triplets)
 #        (Patatrack ECAL-only: TTbar - on CPU)
 #        (Patatrack HCAL-only: TTbar - on CPU)
 #   2021 (DD4HEP: TTbar, ZMM)
 #        (ele guns 10, 35, 1000; pho guns 10, 35; mu guns 1, 10, 100, 1000, QCD 3TeV, QCD Flat)
 #        (ZMM, TTbar, ZEE, MinBias, TTbar PU, TTbar PU premix, ZEE PU, TTbar design, GluGluTo2Jets, GluGluTo2Jets PU)
 #        (TTbar trackingOnly, pixelTrackingOnly, trackingMkFit, trackdnn)
-#        (Patatrack pixel-only: ZMM - on CPU)
-#        (Patatrack pixel-only: TTbar - on CPU)
+#        (Patatrack pixel-only: ZMM - on CPU: quadruplets, triplets)
+#        (Patatrack pixel-only: TTbar - on CPU: quadruplets, triplets)
 #        (Patatrack ECAL-only: TTbar - on CPU)
 #        (Patatrack HCAL-only: TTbar - on CPU)
 #        (TTbar 0T, TTbar PU 0T)
@@ -51,16 +50,16 @@
            10824.6,11024.6,11224.6,
            10824.8,
            11024.15,
-           10842.501,
-           10824.501,
+           10842.501,10842.505,
+           10824.501,10824.505,
            10824.511,
            10824.521,
            11634.911, 11650.911,
            11601.0,11602.0,11603.0,11604.0,11605.0,11606.0,11607.0,11608.0,11609.0,11630.0,11643.0,
            11650.0,11634.0,11646.0,11640.0,11834.0,11834.99,11846.0,12034.0,11725.0,11925.0,
            11634.1,11634.5,11634.7,11634.91,
-           11650.501,
-           11634.501,
+           11650.501,11650.505,
+           11634.501,11634.505,
            11634.511,
            11634.521,
            11634.24,11834.24,

diff --git a/Configuration/PyReleaseValidation/python/relval_gpu.py b/Configuration/PyReleaseValidation/python/relval_gpu.py
@@ -14,21 +14,29 @@
 #just define all of them
 
 #WFs to run in IB:
-# mc 2018   (Patatrack pixel-only: ZMM - on GPU, both CPU and GPU, auto)
-#           (Patatrack pixel-only: TTbar - on GPU, both CPU and GPU, auto)
+# mc 2018   (Patatrack pixel-only quadruplets: ZMM - on GPU, both CPU and GPU, auto)
+#           (Patatrack pixel-only triplets: ZMM - on GPU, both CPU and GPU, auto)
+#           (Patatrack pixel-only quadruplets: TTbar - on GPU, both CPU and GPU, auto)
+#           (Patatrack pixel-only triplets: TTbar - on GPU, both CPU and GPU, auto)
 #           (Patatrack ECAL-only: TTbar - on GPU, both CPU and GPU, auto)
 #           (Patatrack HCAL-only: TTbar - on GPU, both CPU and GPU, auto)
-# mc 2021   (Patatrack pixel-only: ZMM - on GPU, both CPU and GPU, auto)
-#           (Patatrack pixel-only: TTbar - on GPU, both CPU and GPU, auto)
+# mc 2021   (Patatrack pixel-only quadruplets: ZMM - on GPU, both CPU and GPU, auto)
+#           (Patatrack pixel-only triplets: ZMM - on GPU, both CPU and GPU, auto)
+#           (Patatrack pixel-only quadruplets: TTbar - on GPU, both CPU and GPU, auto)
+#           (Patatrack pixel-only triplets: TTbar - on GPU, both CPU and GPU, auto)
 #           (Patatrack ECAL-only: TTbar - on GPU, both CPU and GPU, auto)
 #           (Patatrack HCAL-only: TTbar - on GPU, both CPU and GPU, auto)
 numWFIB = [
            10842.502, # 10842.503,10842.504,
+           10842.506, # 10842.507,10842.508,
            10824.502, # 10824.503,10824.504,
+           10824.506, # 10824.507,10824.508,
            10824.512, # 10824.513,10824.514,
            10824.522, # 10824.523,10824.524,
            11650.502, # 11650.503,11650.504,
+           11650.506, # 11650.507,11650.508,
            11634.502, # 11634.503,11634.504,
+           11634.506, # 11634.507,11634.508,
            11634.512, # 11634.513,11634.514,
            11634.522, # 11634.523,11634.524
         ]

diff --git a/Configuration/PyReleaseValidation/python/relval_steps.py b/Configuration/PyReleaseValidation/python/relval_steps.py
@@ -2186,8 +2186,11 @@ def gen2021HiMix(fragment,howMuch):
                   '--era'          :'Run2_2016'
                   }
 
-step3_pixelNtupleFit = {
-    '--procModifiers': 'pixelNtupleFit',
+step3_pixel_ntuplet_cpu = {
+    '--customise': 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksSoAonCPU'
+}
+step3_pixel_triplets = {
+    '--customise': 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksForTriplets'
 }
 step3_gpu = {
     '--procModifiers': 'gpu',
@@ -2320,8 +2323,11 @@ def gen2021HiMix(fragment,howMuch):
 steps['RECODR2_2018reHLT_Prompt']=merge([{'--conditions':'auto:run2_data'},steps['RECODR2_2018reHLT']])
 steps['RECODR2_2018reHLT_ZBPrompt']=merge([{'--conditions':'auto:run2_data','-s':'RAW2DIGI,L1Reco,RECO,EI,PAT,ALCA:SiStripCalZeroBias+SiStripCalMinBias+TkAlMinBias+EcalESAlign,DQM:@rerecoZeroBias+@ExtraHLT+@miniAODDQM'},steps['RECODR2_2018reHLT']])
 steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']=merge([{'-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,DQM:@pixelTrackingOnlyDQM'},steps['RECODR2_2018reHLT_Prompt']])
-steps['RECODR2_2018reHLT_Patatrack_PixelOnlyCPU']=merge([step3_pixelNtupleFit, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])
+steps['RECODR2_2018reHLT_Patatrack_PixelOnlyCPU']=merge([step3_pixel_ntuplet_cpu, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])
 steps['RECODR2_2018reHLT_Patatrack_PixelOnlyGPU']=merge([step3_gpu, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])
+steps['RECODR2_2018reHLT_Patatrack_PixelOnlyTripletsCPU']=merge([step3_pixel_ntuplet_cpu, step3_pixel_triplets, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])
+steps['RECODR2_2018reHLT_Patatrack_PixelOnlyTripletsGPU']=merge([step3_gpu, step3_pixel_triplets, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])
+
 steps['RECODR2_2018reHLT_ECALOnlyCPU']=merge([{'-s': 'RAW2DIGI:RawToDigi_ecalOnly,RECO:reconstruction_ecalOnly,DQM:@ecalOnly'},steps['RECODR2_2018reHLT_Prompt']])
 steps['RECODR2_2018reHLT_ECALOnlyGPU']=merge([step3_gpu, steps['RECODR2_2018reHLT_ECALOnlyCPU']])
 steps['RECODR2_2018reHLT_HCALOnlyCPU']=merge([{'-s': 'RAW2DIGI:RawToDigi_hcalOnly,RECO:reconstruction_hcalOnly,DQM:@hcalOnly+@hcal2Only'},steps['RECODR2_2018reHLT_Prompt']])

diff --git a/Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py b/Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py
@@ -437,7 +437,26 @@ def condition_(self, fragment, stepList, key, hasHarvest):
     '-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,VALIDATION:@pixelTrackingOnlyValidation,DQM:@pixelTrackingOnlyDQM',
     '--datatier': 'GEN-SIM-RECO,DQMIO',
     '--eventcontent': 'RECOSIM,DQM',
-    '--procModifiers': 'pixelNtupleFit'
+    '--customise' : 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksSoAonCPU'
+}
+
+upgradeWFs['PatatrackPixelOnlyTripletsCPU'] = UpgradeWorkflowPatatrack_PixelOnlyCPU(
+    steps = [
+        'Reco',
+        'HARVEST',
+        'RecoGlobal',
+        'HARVESTGlobal',
+    ],
+    PU = [],
+    suffix = 'Patatrack_PixelOnlyTripletsCPU',
+    offset = 0.505,
+)
+
+upgradeWFs['PatatrackPixelOnlyTripletsCPU'].step3 = {
+    '-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,VALIDATION:@pixelTrackingOnlyValidation,DQM:@pixelTrackingOnlyDQM',
+    '--datatier': 'GEN-SIM-RECO,DQMIO',
+    '--eventcontent': 'RECOSIM,DQM',
+    '--customise' : 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksSoAonCPU,RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksForTriplets'
 }
 
 class UpgradeWorkflowPatatrack_PixelOnlyGPU(UpgradeWorkflowPatatrack):
@@ -471,6 +490,26 @@ def condition_(self, fragment, stepList, key, hasHarvest):
     '--procModifiers': 'gpu'
 }
 
+upgradeWFs['PatatrackPixelOnlyTripletsGPU'] = UpgradeWorkflowPatatrack_PixelOnlyGPU(
+    steps = [
+        'Reco',
+        'HARVEST',
+        'RecoGlobal',
+        'HARVESTGlobal',
+    ],
+    PU = [],
+    suffix = 'Patatrack_PixelOnlyTripletsGPU',
+    offset = 0.506,
+)
+
+upgradeWFs['PatatrackPixelOnlyTripletsGPU'].step3 = {
+    '-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,VALIDATION:@pixelTrackingOnlyValidation,DQM:@pixelTrackingOnlyDQM',
+    '--datatier': 'GEN-SIM-RECO,DQMIO',
+    '--eventcontent': 'RECOSIM,DQM',
+    '--procModifiers': 'gpu',
+    '--customise': 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksForTriplets'
+}
+
 class UpgradeWorkflowPatatrack_ECALOnlyCPU(UpgradeWorkflowPatatrack):
     def setup_(self, step, stepName, stepDict, k, properties):
         if 'Reco' in step: