Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patatrack integration - Pixel workflows (12/N) #31854

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
c5edb5c
Add workflows for Riemann fit and GPU (cms-patatrack#20)
makortel Feb 28, 2018
3cf0122
Add a DQM sequence for pixel-only tracking (cms-patatrack#23)
fwyzard Mar 1, 2018
f407613
Synchronise with CMSSW_10_1_0_pre2
fwyzard Mar 6, 2018
3de9e02
Synchronise with CMSSW_10_1_0_pre3
fwyzard Mar 27, 2018
647f0f1
Synchronise with CMSSW_10_1_0
fwyzard Apr 1, 2018
4493d8e
Synchronise with CMSSW_10_2_0_pre1
fwyzard Apr 23, 2018
854fd07
Synchronise with CMSSW_10_2_0_pre2
fwyzard Apr 29, 2018
9e378a0
Synchronise with CMSSW_10_2_0_pre3
fwyzard May 8, 2018
e3d495c
Synchronise with CMSSW_10_2_0_pre4
fwyzard May 29, 2018
29253b8
Synchronise with CMSSW_10_2_0_pre5
fwyzard Jun 7, 2018
6a8201a
Port the Riemann fit to CUDA (cms-patatrack#60)
rovere Jun 18, 2018
1bb47e0
Synchronise with CMSSW_10_2_0_pre6
fwyzard Jul 4, 2018
9f344c3
update relval input strings (102X)
prebello Jul 25, 2018
c0fc8df
Synchronise with CMSSW_10_2_0
fwyzard Aug 1, 2018
b065c6e
adding NANO step to 2018 MC workflows
fabozzi Aug 1, 2018
f2d0677
Merge pull request #24024 from prebello/from-CMSSW_10_2_X_2018-07-24-…
cmsbuild Aug 7, 2018
a3f910f
Synchronise with CMSSW_10_2_2_patch1
fwyzard Aug 13, 2018
d7e9678
activate the new SiStripCalSiStripCalBiasScan in the matrix test
mmusich Aug 23, 2018
7d124f3
Add pixel tracking workflows for data (cms-patatrack#144)
makortel Sep 2, 2018
bcb8b6d
add HEM-15-16 relval workflows
zhenhu Sep 10, 2018
d5deb5a
introduce RelVal for hybrid emulation (with ZS10 repacking),HLT-like …
mmusich Sep 6, 2018
81a4775
add 3 production workflows for premix2018
zhenhu Sep 18, 2018
22adc00
Merge pull request #24550 from zhenhu/add-HEM-15-16-relvals_102X
cmsbuild Sep 25, 2018
4e54925
Merge pull request #24618 from zhenhu/addPremixProdWfs_102X
cmsbuild Sep 25, 2018
d85ec89
Synchronise with CMSSW_10_2_5
fwyzard Sep 27, 2018
4365090
Synchronise with CMSSW_10_2_6
fwyzard Oct 31, 2018
0ba9da4
Synchronise with CMSSW_10_4_0_pre2
fwyzard Nov 14, 2018
79ca731
Synchronise with CMSSW_10_4_0_pre3
fwyzard Dec 7, 2018
77bbe4f
Synchronise with CMSSW_10_4_0_pre4
fwyzard Dec 14, 2018
27fbbb1
Synchronise with CMSSW_10_4_0
fwyzard Jan 14, 2019
dee131a
Change GPU workflow numbering: .7->.51, .8->.52, .9->.53 (cms-patatra…
makortel Jan 24, 2019
9b0b9b4
Synchronise with CMSSW_10_5_0_pre1
fwyzard Jan 28, 2019
9996ba2
Synchronise with CMSSW_10_5_0_pre2
fwyzard Feb 21, 2019
f6cf09a
Synchronise with CMSSW_10_6_0_pre2
fwyzard Mar 26, 2019
7ba38ea
Rework the Riemann fit and broken line fit (cms-patatrack#338)
VinInn Jan 29, 2019
e23f135
Synchronise with CMSSW_10_6_0
fwyzard May 15, 2019
e8f7151
Synchronise with CMSSW_11_0_0_pre7
fwyzard Sep 12, 2019
200d611
Fix broken merge in Configuration/PyReleaseValidation
fwyzard Oct 9, 2019
9d53109
Synchronise with CMSSW_11_0_0_pre11
fwyzard Nov 4, 2019
5ca473e
Update and extend the Patatrack workflows
fwyzard Nov 29, 2019
4214362
Synchronise with CMSSW_11_0_0_pre13
fwyzard Nov 30, 2019
fe5563a
Backport: add ECAL-only workflows for 2018 data (#30105)
fwyzard Jun 4, 2020
451963e
Add tracking-only and pixel tracking-only workflows for Run 3 (#30338)
fwyzard Jun 23, 2020
1be02c2
Backport: add ECAL-only and HCAL-only workflows for MC and data (#30350)
mariadalfonso Jun 5, 2020
a3fae5d
Synchronise with CMSSW_11_2_0_pre2
fwyzard Jul 12, 2020
678102d
Update the RelVal workflows and the CPU customisation (cms-patatrack#…
AdrianoDee Oct 12, 2020
6d6b8c9
Synchronise with CMSSW_11_2_0_pre8
fwyzard Oct 23, 2020
1fcf499
Synchronise with CMSSW_11_2_0_pre9
fwyzard Nov 16, 2020
b650af6
Synchronise with CMSSW_11_2_0_pre10
fwyzard Nov 27, 2020
737b3fd
Backport: add RecoFakeHLT steps to 2018 Patatrack WFs to enable GPU m…
thomreis Dec 1, 2020
58077e2
Synchronise with CMSSW_11_3_0_pre1
fwyzard Dec 24, 2020
fde200b
Synchronise with CMSSW_11_3_X_2020-12-24-2300
fwyzard Dec 24, 2020
1bb67a8
Synchronise with CMSSW_11_3_0_pre5
fwyzard Mar 23, 2021
cceab86
Document the new Pixel workflows
fwyzard Apr 9, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions Configuration/PyReleaseValidation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,14 @@ The offsets currently in use are:
* 0.3: 0.1 + 0.2
* 0.4: LowPU tracking era, `Run2_2017_trackingLowPU`
* 0.5: Pixel tracking only + 0.1
* 0.501: Patatrack, pixel only quadruplets, on CPU
* 0.502: Patatrack, pixel only quadruplets, on GPU
* 0.505: Patatrack, pixel only triplets, on CPU
* 0.506: Patatrack, pixel only triplets, on GPU
* 0.511: Patatrack, ECAL only CPU
* 0.512: Patatrack, ECAL only GPU
* 0.521: Patatrack, HCAL only CPU
* 0.522: Patatrack, HCAL only GPU
* 0.6: HE Collapse (old depth segmentation for 2018)
* 0.7: trackingMkFit modifier
* 0.8: BPH Parking (Run-2)
Expand All @@ -52,10 +60,4 @@ The offsets currently in use are:
* 0.911: DD4Hep
* 0.101: Phase-2 aging, 1000fb-1
* 0.103: Phase-2 aging, 3000fb-1
* 0.501: Patatrack, pixel only CPU
* 0.502: Patatrack, pixel only GPU
* 0.511: Patatrack, ECAL only CPU
* 0.512: Patatrack, ECAL only GPU
* 0.521: Patatrack, HCAL only CPU
* 0.522: Patatrack, HCAL only GPU
* 0.9001: Sonic Triton
17 changes: 8 additions & 9 deletions Configuration/PyReleaseValidation/python/relval_2017.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

# import the definition of the steps and input files:
from Configuration.PyReleaseValidation.relval_steps import *

Expand All @@ -24,16 +23,16 @@
# (HE collapse: TTbar, TTbar PU, TTbar design)
# (ParkingBPH: TTbar)
# (TTbar PU with JME NanoAOD)
# (Patatrack pixel-only: ZMM - on CPU)
# (Patatrack pixel-only: TTbar - on CPU)
# (Patatrack pixel-only: ZMM - on CPU: quadruplets, triplets)
# (Patatrack pixel-only: TTbar - on CPU: quadruplets, triplets)
# (Patatrack ECAL-only: TTbar - on CPU)
# (Patatrack HCAL-only: TTbar - on CPU)
# 2021 (DD4HEP: TTbar, ZMM)
# (ele guns 10, 35, 1000; pho guns 10, 35; mu guns 1, 10, 100, 1000, QCD 3TeV, QCD Flat)
# (ZMM, TTbar, ZEE, MinBias, TTbar PU, TTbar PU premix, ZEE PU, TTbar design)
# (TTbar trackingOnly, pixelTrackingOnly, trackingMkFit, trackdnn)
# (Patatrack pixel-only: ZMM - on CPU)
# (Patatrack pixel-only: TTbar - on CPU)
# (Patatrack pixel-only: ZMM - on CPU: quadruplets, triplets)
# (Patatrack pixel-only: TTbar - on CPU: quadruplets, triplets)
# (Patatrack ECAL-only: TTbar - on CPU)
# (Patatrack HCAL-only: TTbar - on CPU)
# (TTbar 0T, TTbar PU 0T)
Expand All @@ -51,16 +50,16 @@
10824.6,11024.6,11224.6,
10824.8,
11024.15,
10842.501,
10824.501,
10842.501,10842.505,
10824.501,10824.505,
10824.511,
10824.521,
11634.911, 11650.911,
11601.0,11602.0,11603.0,11604.0,11605.0,11606.0,11607.0,11608.0,11609.0,11630.0,11643.0,
11650.0,11634.0,11646.0,11640.0,11834.0,11834.99,11846.0,12024.0,
11634.1,11634.5,11634.7,11634.91,
11650.501,
11634.501,
11650.501,11650.505,
11634.501,11634.505,
11634.511,
11634.521,
11634.24,11834.24,
Expand Down
16 changes: 12 additions & 4 deletions Configuration/PyReleaseValidation/python/relval_gpu.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,21 +14,29 @@
#just define all of them

#WFs to run in IB:
# mc 2018 (Patatrack pixel-only: ZMM - on GPU, both CPU and GPU, auto)
# (Patatrack pixel-only: TTbar - on GPU, both CPU and GPU, auto)
# mc 2018 (Patatrack pixel-only quadruplets: ZMM - on GPU, both CPU and GPU, auto)
# (Patatrack pixel-only triplets: ZMM - on GPU, both CPU and GPU, auto)
# (Patatrack pixel-only quadruplets: TTbar - on GPU, both CPU and GPU, auto)
# (Patatrack pixel-only triplets: TTbar - on GPU, both CPU and GPU, auto)
# (Patatrack ECAL-only: TTbar - on GPU, both CPU and GPU, auto)
# (Patatrack HCAL-only: TTbar - on GPU, both CPU and GPU, auto)
# mc 2021 (Patatrack pixel-only: ZMM - on GPU, both CPU and GPU, auto)
# (Patatrack pixel-only: TTbar - on GPU, both CPU and GPU, auto)
# mc 2021 (Patatrack pixel-only quadruplets: ZMM - on GPU, both CPU and GPU, auto)
# (Patatrack pixel-only triplets: ZMM - on GPU, both CPU and GPU, auto)
# (Patatrack pixel-only quadruplets: TTbar - on GPU, both CPU and GPU, auto)
# (Patatrack pixel-only triplets: TTbar - on GPU, both CPU and GPU, auto)
# (Patatrack ECAL-only: TTbar - on GPU, both CPU and GPU, auto)
# (Patatrack HCAL-only: TTbar - on GPU, both CPU and GPU, auto)
numWFIB = [
10842.502, # 10842.503,10842.504,
10842.506, # 10842.507,10842.508,
10824.502, # 10824.503,10824.504,
10824.506, # 10824.507,10824.508,
10824.512, # 10824.513,10824.514,
10824.522, # 10824.523,10824.524,
11650.502, # 11650.503,11650.504,
11650.506, # 11650.507,11650.508,
11634.502, # 11634.503,11634.504,
11634.506, # 11634.507,11634.508,
11634.512, # 11634.513,11634.514,
11634.522, # 11634.523,11634.524
]
Expand Down
12 changes: 9 additions & 3 deletions Configuration/PyReleaseValidation/python/relval_steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -2186,8 +2186,11 @@ def gen2021HiMix(fragment,howMuch):
'--era' :'Run2_2016'
}

step3_pixelNtupleFit = {
'--procModifiers': 'pixelNtupleFit',
step3_pixel_ntuplet_cpu = {
'--customise': 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksSoAonCPU'
}
step3_pixel_triplets = {
'--customise': 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksForTriplets'
}
step3_gpu = {
'--procModifiers': 'gpu',
Expand Down Expand Up @@ -2320,8 +2323,11 @@ def gen2021HiMix(fragment,howMuch):
steps['RECODR2_2018reHLT_Prompt']=merge([{'--conditions':'auto:run2_data'},steps['RECODR2_2018reHLT']])
steps['RECODR2_2018reHLT_ZBPrompt']=merge([{'--conditions':'auto:run2_data','-s':'RAW2DIGI,L1Reco,RECO,EI,PAT,ALCA:SiStripCalZeroBias+SiStripCalMinBias+TkAlMinBias+EcalESAlign,DQM:@rerecoZeroBias+@ExtraHLT+@miniAODDQM'},steps['RECODR2_2018reHLT']])
steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']=merge([{'-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,DQM:@pixelTrackingOnlyDQM'},steps['RECODR2_2018reHLT_Prompt']])
steps['RECODR2_2018reHLT_Patatrack_PixelOnlyCPU']=merge([step3_pixelNtupleFit, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])
steps['RECODR2_2018reHLT_Patatrack_PixelOnlyCPU']=merge([step3_pixel_ntuplet_cpu, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])
steps['RECODR2_2018reHLT_Patatrack_PixelOnlyGPU']=merge([step3_gpu, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])
steps['RECODR2_2018reHLT_Patatrack_PixelOnlyTripletsCPU']=merge([step3_pixel_ntuplet_cpu, step3_pixel_triplets, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])
steps['RECODR2_2018reHLT_Patatrack_PixelOnlyTripletsGPU']=merge([step3_gpu, step3_pixel_triplets, steps['RECODR2_2018reHLT_Prompt_pixelTrackingOnly']])

steps['RECODR2_2018reHLT_ECALOnlyCPU']=merge([{'-s': 'RAW2DIGI:RawToDigi_ecalOnly,RECO:reconstruction_ecalOnly,DQM:@ecalOnly'},steps['RECODR2_2018reHLT_Prompt']])
steps['RECODR2_2018reHLT_ECALOnlyGPU']=merge([step3_gpu, steps['RECODR2_2018reHLT_ECALOnlyCPU']])
steps['RECODR2_2018reHLT_HCALOnlyCPU']=merge([{'-s': 'RAW2DIGI:RawToDigi_hcalOnly,RECO:reconstruction_hcalOnly,DQM:@hcalOnly+@hcal2Only'},steps['RECODR2_2018reHLT_Prompt']])
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -453,7 +453,26 @@ def condition_(self, fragment, stepList, key, hasHarvest):
'-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,VALIDATION:@pixelTrackingOnlyValidation,DQM:@pixelTrackingOnlyDQM',
'--datatier': 'GEN-SIM-RECO,DQMIO',
'--eventcontent': 'RECOSIM,DQM',
'--procModifiers': 'pixelNtupleFit'
'--customise' : 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksSoAonCPU'
}

upgradeWFs['PatatrackPixelOnlyTripletsCPU'] = UpgradeWorkflowPatatrack_PixelOnlyCPU(
steps = [
'Reco',
'HARVEST',
'RecoGlobal',
'HARVESTGlobal',
],
PU = [],
suffix = 'Patatrack_PixelOnlyTripletsCPU',
offset = 0.505,
)

upgradeWFs['PatatrackPixelOnlyTripletsCPU'].step3 = {
'-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,VALIDATION:@pixelTrackingOnlyValidation,DQM:@pixelTrackingOnlyDQM',
'--datatier': 'GEN-SIM-RECO,DQMIO',
'--eventcontent': 'RECOSIM,DQM',
'--customise' : 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksSoAonCPU,RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksForTriplets'
}

class UpgradeWorkflowPatatrack_PixelOnlyGPU(UpgradeWorkflowPatatrack):
Expand Down Expand Up @@ -487,6 +506,26 @@ def condition_(self, fragment, stepList, key, hasHarvest):
'--procModifiers': 'gpu'
}

upgradeWFs['PatatrackPixelOnlyTripletsGPU'] = UpgradeWorkflowPatatrack_PixelOnlyGPU(
steps = [
'Reco',
'HARVEST',
'RecoGlobal',
'HARVESTGlobal',
],
PU = [],
suffix = 'Patatrack_PixelOnlyTripletsGPU',
offset = 0.506,
)

upgradeWFs['PatatrackPixelOnlyTripletsGPU'].step3 = {
'-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,VALIDATION:@pixelTrackingOnlyValidation,DQM:@pixelTrackingOnlyDQM',
'--datatier': 'GEN-SIM-RECO,DQMIO',
'--eventcontent': 'RECOSIM,DQM',
'--procModifiers': 'gpu',
'--customise': 'RecoPixelVertexing/Configuration/customizePixelTracksSoAonCPU.customizePixelTracksForTriplets'
}

class UpgradeWorkflowPatatrack_ECALOnlyCPU(UpgradeWorkflowPatatrack):
def setup_(self, step, stepName, stepDict, k, properties):
if 'Reco' in step:
Expand Down