Make the the size of the binner (HistoContainer) settable at run time. The total number of Pixel Clusters is not a limit anymore on GPU #33371

VinInn · 2021-04-08T09:07:19Z

It makes LocalReco independent from the size of the event and tuple-making independent of the number of hits.
Some of the containers used by the CA will remain of fixed size (cannot be extended during the pattern recognition).

purely technical. no regression expected (unless the total number of clusters was larger than previous limit).

Further needs by HI and Phase2 will be adressed in future PRs

This makes the pixel local reconstruction independent from the size of the event, and tuple-making independent of the number of hits. Some of the containers used by the CA remain of fixed size, as they cannot be extended during the pattern recognition).

VinInn · 2021-04-08T09:08:37Z

@cmsbuild please test

cmsbuild · 2021-04-08T09:13:19Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33371/21961

This PR adds an extra 68KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File RecoLocalTracker/SiPixelClusterizer/plugins/SiPixelRawToClusterGPUKernel.cu modified in PR(s): GPU pixel local reconstruction: make the pixel cluster thresholds configurable #33259
- File RecoPixelVertexing/PixelTriplets/plugins/BrokenLineFitOnGPU.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File RecoPixelVertexing/PixelTriplets/plugins/CAConstants.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.cc modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.cu modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernelsAlloc.h modified in PR(s): [RFC] Test clang-tidy --checks misc-definitions-in-headers #33366, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernelsImpl.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorOnGPU.cc modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File RecoPixelVertexing/PixelTriplets/plugins/RiemannFitOnGPU.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

cmsbuild · 2021-04-08T09:13:47Z

A new Pull Request was created by @VinInn (Vincenzo Innocente) for master.

It involves the following packages:

CUDADataFormats/SiPixelCluster
CUDADataFormats/TrackingRecHit
HeterogeneousCore/CUDAUtilities
RecoLocalTracker/SiPixelClusterizer
RecoLocalTracker/SiPixelRecHits
RecoPixelVertexing/PixelTriplets

@perrotta, @makortel, @jpata, @fwyzard, @slava77 can you please review it and eventually sign? Thanks.
@mtosi, @makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @OzAmram, @ferencek, @dkotlins, @gpetruc, @ebrondol, @threus, @dgulhan, @tvami this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

fwyzard · 2021-04-08T09:38:30Z

please test

fwyzard · 2021-04-08T09:38:33Z

enable gpu

cmsbuild · 2021-04-08T12:55:18Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7401c4/14093/summary.html
COMMIT: 37bb68e
CMSSW: CMSSW_11_3_X_2021-04-07-2300/slc7_amd64_gcc900
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33371/14093/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 9575
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 9575
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2865506
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2865477
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

fwyzard · 2021-04-12T10:39:31Z

I'd like to retest this after #31854 is merged, to check also the triplets.

VinInn · 2021-04-14T08:08:32Z

ok. So we are ready to merge, aren't we?

cmsbuild · 2021-04-14T08:15:28Z

@VinInn, CMSSW_11_3_X branch is closed for direct updates. cms-bot is going to move this PR to master branch.
In future, please use cmssw master branch to submit your changes.

srimanob · 2021-04-14T08:15:38Z

As mentioned by @slava77, should this PR be rebased first? I see several of them are already merged with #31854

Thanks.

cmsbuild · 2021-04-14T08:15:56Z

@VinInn, CMSSW_11_3_X branch is closed for direct updates. cms-bot is going to move this PR to master branch.
In future, please use cmssw master branch to submit your changes.

VinInn · 2021-04-14T08:17:40Z

should be ok now

fwyzard · 2021-04-14T08:25:11Z

+heterogeneous

slava77 · 2021-04-14T11:54:05Z

+reconstruction

for #33371 9420952

in line with the PR title
jenkins tests pass and comparisons with the baseline in GPU and CPU tests show no differences, as expected

@VinInn please copy relevant details of cms-patatrack#590 in this PR description to improve direct self-documentation

cmsbuild · 2021-04-14T11:54:32Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

VinInn · 2021-04-15T11:53:52Z

can this be merged before the weekend?

qliphy · 2021-04-16T01:34:11Z

+1

VinInn added 5 commits April 8, 2021 10:10

Merged NewBinner_113x from repository fwyzard with cms-merge-topic

d872b83

remove unused variable

bff4459

remove all references to maxHits

369f94b

code format

37bb68e

cmsbuild added this to the CMSSW_11_3_X milestone Apr 8, 2021

cmsbuild added code-checks-pending heterogeneous-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Apr 8, 2021

cmsbuild added tests-started and removed tests-pending labels Apr 8, 2021

VinInn changed the title ~~Make the the size of the binner (HistoContainer) settable at run time. The total number of Hits is not a limit anymore.~~ Make the the size of the binner (HistoContainer) settable at run time. The total number of Pixel Clusters is not a limit anymore on GPU Apr 8, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 8, 2021

fwyzard mentioned this pull request Apr 8, 2021

Make the the size of the binner (HistoContainer) settable at run time cms-patatrack/cmssw#590

Closed

cmsbuild mentioned this pull request Apr 8, 2021

Patatrack Shared Hit Cleaner Fix & Update #33375

Merged

cmsbuild added tests-approved and removed tests-started labels Apr 8, 2021

cmsbuild mentioned this pull request Apr 9, 2021

MultiCluster-CaloParticle associator #32941

Closed

This comment has been minimized.

Sign in to view

VinInn changed the base branch from master to CMSSW_11_3_X April 14, 2021 08:15

cmsbuild changed the base branch from CMSSW_11_3_X to master April 14, 2021 08:15

cmsbuild removed pdmv-pending upgrade-pending labels Apr 14, 2021

VinInn changed the base branch from master to CMSSW_11_3_X April 14, 2021 08:15

cmsbuild changed the base branch from CMSSW_11_3_X to master April 14, 2021 08:15

cmsbuild added heterogeneous-approved and removed heterogeneous-pending labels Apr 14, 2021

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Apr 14, 2021

cmsbuild modified the milestones: CMSSW_11_3_X, CMSSW_12_0_X Apr 15, 2021

cmsbuild added orp-approved and removed orp-pending labels Apr 16, 2021

cmsbuild merged commit 53993f8 into cms-sw:master Apr 16, 2021

cmsbuild mentioned this pull request Apr 16, 2021

moving DataFormats/RecoCandidate/interface/TrackAssociation.h into SimDataFormats #33186

Merged

makortel mentioned this pull request Jul 28, 2021

GPU Tests - pixel crashes on GPU Hilton #34659

Closed

mmusich mentioned this pull request Jul 29, 2021

fix CAHitNtupletGeneratorOnGPU in case of no hits #34684

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make the the size of the binner (HistoContainer) settable at run time. The total number of Pixel Clusters is not a limit anymore on GPU #33371

Make the the size of the binner (HistoContainer) settable at run time. The total number of Pixel Clusters is not a limit anymore on GPU #33371

VinInn commented Apr 8, 2021 •

edited

Loading

VinInn commented Apr 8, 2021

cmsbuild commented Apr 8, 2021

cmsbuild commented Apr 8, 2021

fwyzard commented Apr 8, 2021

fwyzard commented Apr 8, 2021

cmsbuild commented Apr 8, 2021

This comment has been minimized.

fwyzard commented Apr 12, 2021 •

edited

Loading

VinInn commented Apr 14, 2021 •

edited

Loading

cmsbuild commented Apr 14, 2021

srimanob commented Apr 14, 2021

cmsbuild commented Apr 14, 2021

VinInn commented Apr 14, 2021

fwyzard commented Apr 14, 2021

slava77 commented Apr 14, 2021

cmsbuild commented Apr 14, 2021

VinInn commented Apr 15, 2021

qliphy commented Apr 16, 2021

Make the the size of the binner (HistoContainer) settable at run time. The total number of Pixel Clusters is not a limit anymore on GPU #33371

Make the the size of the binner (HistoContainer) settable at run time. The total number of Pixel Clusters is not a limit anymore on GPU #33371

Conversation

VinInn commented Apr 8, 2021 • edited Loading

VinInn commented Apr 8, 2021

cmsbuild commented Apr 8, 2021

cmsbuild commented Apr 8, 2021

fwyzard commented Apr 8, 2021

fwyzard commented Apr 8, 2021

cmsbuild commented Apr 8, 2021

GPU Comparison Summary

Comparison Summary

This comment has been minimized.

fwyzard commented Apr 12, 2021 • edited Loading

VinInn commented Apr 14, 2021 • edited Loading

cmsbuild commented Apr 14, 2021

srimanob commented Apr 14, 2021

cmsbuild commented Apr 14, 2021

VinInn commented Apr 14, 2021

fwyzard commented Apr 14, 2021

slava77 commented Apr 14, 2021

cmsbuild commented Apr 14, 2021

VinInn commented Apr 15, 2021

qliphy commented Apr 16, 2021

VinInn commented Apr 8, 2021 •

edited

Loading

fwyzard commented Apr 12, 2021 •

edited

Loading

VinInn commented Apr 14, 2021 •

edited

Loading