Migration to DD4Hep for RPC db Loader #32102

slomeo · 2020-11-11T13:05:13Z

PR description:

This PR (only for RPC) followed what I did for CSC in #32001 .

PR validation:

validation by "cmsRun CondTools/Geometry/test/rpcgeometrywriter.py" (with DD4hep flag set False or True):

Begin processing the 1st record. Run 1, Event 1, LumiSection 1 on stream 0 at 11-Nov-2020 13:26:09.163 CET

MessageLogger Summary

Severity # Occurrences Total Occurrences

dropped waiting message count 0

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

validation by printouts for both DD & DD4Hep build methods within the RPCGeometryParsFromDD Class in order to compare some DetId parameters (for example the 637566989 detid):

//DD
1, RPCGeometryParsFromDD, detid: 637566989 name: MB1RPC_IGasLeft number of Strips: 90
2, RPCGeometryParsFromDD, dpar.size() == 3, width: 103 length: 58.45 thickness: 0.2
4, RPCGeometryParsFromDD, tran.x(): 413.675 tran.y(): 37.6 tran.z(): -470.45
5, RPCGeometryParsFromDD, x.X(), x.Y(), x.Z(), y.X(), y.Y(), y.Z(), z.X(), z.Y(), z.Z():
5, RPCGeometryParsFromDD 3.06162e-16, -1, 3.12322e-16, -4.78895e-16, -8.5632e-16, 1, -1, 6.40079e-16, 1.15045e-17

//DD4Hep
1, RPCGeometryParsFromDD, detid: 637566989 name: MB1RPC_IGasLeft number of Strips: 90
2, RPCGeometryParsFromDD, dd4hep::Box, width: 103 length: 58.45 thickness: 0.2
4, RPCGeometryParsFromDD, tran.x(): 413.675 tran.y(): 37.6 tran.z(): -470.45
5, RPCGeometryParsFromDD, x.X(), x.Y(), x.Z(), y.X(), y.Y(), y.Z(), z.X(), z.Y(), z.Z():
5, RPCGeometryParsFromDD 2.13888e-16, -1, 2.00422e-16, -9.9934e-17, -4.99491e-16, 1, -1, 3.02876e-16, -1.22527e-16

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

validation by "runTheMatrix.py -l 25202.1" (because I edited the RPCGeometryParsFromDD class and not only the DB Class):

25202.1_TTbar_13+TTbar_13+DIGIUP15APVSimu_PU25+RECOUP15_PU25+HARVESTUP15_PU25 Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED - time date Wed Nov 11 13:59:46 2020-date Wed Nov 11 13:32:32 2020; exit: 0 0 0 0
1 1 1 1 tests passed, 0 0 0 0 failed

if this PR is a backport please specify the original PR and why you need to backport that PR:

nothing special

cmsbuild · 2020-11-11T13:12:09Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32102/19755

This PR adds an extra 20KB to repository

cmsbuild · 2020-11-11T13:12:34Z

A new Pull Request was created by @slomeo (Sergio Lo Meo) for master.

It involves the following packages:

CondTools/Geometry
Geometry/RPCGeometryBuilder

@civanch, @Dr15Jones, @makortel, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @ggovi can you please review it and eventually sign? Thanks.
@mmusich, @fabiocos this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

slomeo · 2020-11-11T17:04:49Z

@cvuosalo : please take a look at https://cmssdt.cern.ch/lxr/search?%21v=CMSSW_11_2_X_2020-11-10-2300&_filestring=&_string=RPCRecoIdealDBLoader

Do I have to edit all these .py files adding

process.RPCGeometryWriter = cms.EDAnalyzer("RPCRecoIdealDBLoader",
fromDD4Hep = cms.untracked.bool(False))

instead of

process.RPCGeometryWriter = cms.EDAnalyzer("RPCRecoIdealDBLoader") ?

civanch · 2020-11-11T17:26:51Z

please test

cmsbuild · 2020-11-11T17:27:13Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-11-10-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10655/console Started: 2020/11/11 18:28

cmsbuild · 2020-11-11T18:47:56Z

+1
Tested at: 776c017
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4a5256/10655/summary.html
CMSSW: CMSSW_11_2_X_2020-11-10-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-11-11T18:47:58Z

Comparison job queued.

cmsbuild · 2020-11-11T20:06:44Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4a5256/10655/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2529296
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2529265
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 148 log files, 22 edm output root files, 35 DQM output files

cvuosalo · 2020-11-11T20:12:26Z

Geometry/RPCGeometryBuilder/src/RPCGeometryParsFromDD.cc

+#include <DetectorDescription/DDCMS/interface/DDFilteredView.h>
+#include <DetectorDescription/DDCMS/interface/DDCompactView.h>
+#include "DetectorDescription/DDCMS/interface/DDSpecParRegistry.h"
+#include "DataFormats/Math/interface/CMSUnits.h"


Why is CMSUnits.h included? I don't see anything from it used in this file.

@cvuosalo : I deleted this #include

cvuosalo · 2020-11-11T20:29:16Z

CondTools/Geometry/test/rpcgeometrywriter.py

@@ -4,6 +4,8 @@
 process.load('CondCore.CondDB.CondDB_cfi')
 process.load('Configuration.StandardSequences.GeometryExtended_cff')
 process.load('Geometry.MuonNumbering.muonNumberingInitialization_cfi')
+process.load("Geometry.MuonNumbering.muonGeometryConstants_cff")
+process.load('Configuration.StandardSequences.DD4hep_GeometrySim_cff')


I don't think it is right to include this DD4hep geometry. This file needs to work with both old DD and DD4hep.

@cvuosalo : I added the same line for PR #32001 (for CSC). Without it an error appears:

----- Begin Fatal Exception 12-Nov-2020 09:04:49 CET-----------------------
An exception of category 'NoProxyException' occurred while
[0] Processing global begin Run run: 1
[1] Calling method for module RPCRecoIdealDBLoader/'RPCGeometryWriter'
Exception Message:
No data of type "DDCompactView" with label "" in record "IdealGeometryRecord"
Please add an ESSource or ESProducer to your job which can deliver this data.
----- End Fatal Exception -------------------------------------------------
%MSG-i DDCompactViewImpl: 12-Nov-2020 09:04:49 CET EndJob
DDD transient representation has been destructed.
%MSG

cvuosalo · 2020-11-11T20:31:53Z

Geometry/RPCGeometryBuilder/src/RPCGeometryParsFromDD.cc

    }
-
-    const std::vector<double> vtra = {tran.x(), tran.y(), tran.z()};
+    const std::vector<double> vtra = {geant_units::operators::convertMmToCm(tran.x()),


This change is worrisome. The old DD code should not need to be changed. Why would the units convention change now for old DD?

@cvuosalo : the same code is present in the RPCGeometryBuilder Class. Without it in the Log file I found:

RPCGeometryParsFromDD, tran.x(): 4136.75 tran.y(): 376 tran.z(): -4704.5 // for DD
RPCGeometryParsFromDD, tran.x(): 413.675 tran.y(): 37.6 tran.z(): -470.45 // for DD4Hep

so I think that geant4_units are necessary

makortel · 2020-12-03T14:11:41Z

I find that myLog.log and myfile.db are "fine" but I see this message:

Fatal system signal has occurred during exit
Aborted

We have an open issue on that error #32045 (basically something goes badly wrong during the exit of cmsRun). Do you see any other error messages? Could you try to run with gdb and provide the stack trace of the crash point? Or give a recipe to reproduce?

cvuosalo · 2020-12-03T14:43:07Z

@cmsbuild please test

slomeo · 2020-12-03T15:36:13Z

We have an open issue on that error #32045 (basically something goes badly wrong during the exit of cmsRun). Do you >see any other error messages? Could you try to run with gdb and provide the stack trace of the crash point? Or give a >recipe to reproduce?

@makortel , this the recipe:

In a new IB, after a "cmsenv", please perform "git cms-rebase-topic slomeo:MigrationToDD4hepForRPCdbLoader"
scram b -j 10
cmsRun CondTools/Geometry/test/rpcgeometrywriter.py (and you can see the error for DDD Geometry)
edit "CondTools/Geometry/test/rpcgeometrywriter.py" setting "True" instead "False"
rm myfile.db
cmsRun CondTools/Geometry/test/rpcgeometrywriter.py (and you can see the error for DD4HEP Geometry)

cmsbuild · 2020-12-03T16:03:25Z

+1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4a5256/11313/summary.html
CMSSW: CMSSW_11_3_X_2020-12-02-2300
SCRAM_ARCH: slc7_amd64_gcc900

cmsbuild · 2020-12-03T17:26:18Z

Comparison results are now available
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4a5256/11313/summary.html
CMSSW: CMSSW_11_3_X_2020-12-02-2300
SCRAM_ARCH: slc7_amd64_gcc900

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2529593
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2529564
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 34 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 148 log files, 37 edm output root files, 35 DQM output files

cvuosalo · 2020-12-03T19:47:57Z

+1

makortel · 2020-12-03T20:06:23Z

@slomeo Thanks, I was able reproduce and got a stack trace in gdb (with -g -O0)

Thread 1 "cmsRun" received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) where
#0  0x0000000000000000 in ?? ()
#1  0x00007fffed12b7f1 in coral::MessageStream::doOutput() ()
   from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2020-12-03-1100/external/slc7_amd64_gcc900/lib/liblcg_CoralBase.so
#2  0x00007fffd6c5a4a7 in coral::ConnectionService::ConnectionPool::~ConnectionPool() ()
   from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2020-12-03-1100/external/slc7_amd64_gcc900/lib/liblcg_ConnectionService.so
#3  0x00007fffd6c5a6e9 in coral::ConnectionService::ConnectionPool::~ConnectionPool() ()
   from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2020-12-03-1100/external/slc7_amd64_gcc900/lib/liblcg_ConnectionService.so
#4  0x00007fffd6c61c51 in coral::ConnectionService::ConnectionService::~ConnectionService() ()
   from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2020-12-03-1100/external/slc7_amd64_gcc900/lib/liblcg_ConnectionService.so
#5  0x00007fffd6c61ce9 in coral::ConnectionService::ConnectionService::~ConnectionService() ()
   from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2020-12-03-1100/external/slc7_amd64_gcc900/lib/liblcg_ConnectionService.so
#6  0x00007fffed159643 in coral::Context::~Context() ()
   from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2020-12-03-1100/external/slc7_amd64_gcc900/lib/liblcg_CoralKernel.so
#7  0x00007ffff503fce9 in __run_exit_handlers () from /lib64/libc.so.6
#8  0x00007ffff503fd37 in exit () from /lib64/libc.so.6
#9  0x00007ffff502855c in __libc_start_main () from /lib64/libc.so.6
#10 0x000000000040f7f8 in _start ()

(certainly out of scope of this PR)

slomeo · 2020-12-04T07:19:42Z

@cvuosalo : while I'm waiting for this PR will be approved and merged I'll update the Ideal Reco DB migration for CSC in order to check if "mm" is the default unit.

slomeo · 2020-12-07T07:42:39Z

Hi all, do you agree to approve this PR? or do I have to wait till #32045 will be merged?

silviodonato · 2020-12-07T15:49:50Z

@ggovi could you have a look?

cvuosalo · 2020-12-07T22:31:58Z

@slomeo I don't think there is any need to wait for #32045. It is unrelated.

ggovi · 2020-12-09T19:11:29Z

+1

cmsbuild · 2020-12-09T19:11:50Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-12-09T19:36:03Z

+1

cmsbuild added this to the CMSSW_11_2_X milestone Nov 11, 2020

cmsbuild added code-checks-pending comparison-pending db-pending geometry-pending orp-pending pending-signatures tests-pending labels Nov 11, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 11, 2020

cmsbuild added tests-started and removed tests-pending labels Nov 11, 2020

cmsbuild added tests-approved and removed tests-started labels Nov 11, 2020

cmsbuild added comparison-available and removed comparison-pending labels Nov 11, 2020

cvuosalo reviewed Nov 11, 2020

View reviewed changes

cmsbuild removed code-checks-approved comparison-available tests-approved labels Nov 12, 2020

cmsbuild added tests-started and removed tests-pending labels Dec 3, 2020

cmsbuild added tests-approved and removed tests-started labels Dec 3, 2020

cmsbuild added geometry-approved and removed geometry-pending labels Dec 3, 2020

makortel mentioned this pull request Dec 3, 2020

Fatal system signal has occurred during exit #32045

Closed

cmsbuild added db-approved fully-signed and removed db-pending pending-signatures labels Dec 9, 2020

cmsbuild added orp-approved and removed orp-pending labels Dec 9, 2020

cmsbuild merged commit e44626b into cms-sw:master Dec 9, 2020

slomeo mentioned this pull request Feb 1, 2021

Migration to DD4hep for DT db Loader #32781

Merged

slomeo mentioned this pull request Feb 25, 2021

Update Migration to DD4Hep for CSC and RPC db Loader #32992

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migration to DD4Hep for RPC db Loader #32102

Migration to DD4Hep for RPC db Loader #32102

slomeo commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

slomeo commented Nov 11, 2020

civanch commented Nov 11, 2020

cmsbuild commented Nov 11, 2020 •

edited

Loading

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

cvuosalo Nov 11, 2020

slomeo Nov 12, 2020

cvuosalo Nov 11, 2020

slomeo Nov 12, 2020

cvuosalo Nov 11, 2020

slomeo Nov 12, 2020 •

edited

Loading

makortel commented Dec 3, 2020

cvuosalo commented Dec 3, 2020

slomeo commented Dec 3, 2020

cmsbuild commented Dec 3, 2020

cmsbuild commented Dec 3, 2020

cvuosalo commented Dec 3, 2020

makortel commented Dec 3, 2020

slomeo commented Dec 4, 2020

slomeo commented Dec 7, 2020

silviodonato commented Dec 7, 2020

cvuosalo commented Dec 7, 2020

ggovi commented Dec 9, 2020

cmsbuild commented Dec 9, 2020

silviodonato commented Dec 9, 2020

Migration to DD4Hep for RPC db Loader #32102

Migration to DD4Hep for RPC db Loader #32102

Conversation

slomeo commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

slomeo commented Nov 11, 2020

civanch commented Nov 11, 2020

cmsbuild commented Nov 11, 2020 • edited Loading

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

cvuosalo Nov 11, 2020

Choose a reason for hiding this comment

slomeo Nov 12, 2020

Choose a reason for hiding this comment

cvuosalo Nov 11, 2020

Choose a reason for hiding this comment

slomeo Nov 12, 2020

Choose a reason for hiding this comment

cvuosalo Nov 11, 2020

Choose a reason for hiding this comment

slomeo Nov 12, 2020 • edited Loading

Choose a reason for hiding this comment

makortel commented Dec 3, 2020

cvuosalo commented Dec 3, 2020

slomeo commented Dec 3, 2020

cmsbuild commented Dec 3, 2020

cmsbuild commented Dec 3, 2020

cvuosalo commented Dec 3, 2020

makortel commented Dec 3, 2020

slomeo commented Dec 4, 2020

slomeo commented Dec 7, 2020

silviodonato commented Dec 7, 2020

cvuosalo commented Dec 7, 2020

ggovi commented Dec 9, 2020

cmsbuild commented Dec 9, 2020

silviodonato commented Dec 9, 2020

cmsbuild commented Nov 11, 2020 •

edited

Loading

slomeo Nov 12, 2020 •

edited

Loading