Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logging infrastructure to all algorithms #824

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

stephenswat
Copy link
Member

This commit adds ACTS-style loggers to all existing traccc algorithms, and requires that all future algorithms have the same. It also carefully threads logger objects through the computation chain. Currently, little is added in the way of useful logging outputs, as that can happen in a future commit: this is just infrastructure.

@stephenswat stephenswat added the feature New feature or request label Jan 24, 2025
@stephenswat
Copy link
Member Author

Apologies for the high code volume, the vast majority is boilerplate.

@stephenswat stephenswat changed the title Add logging to all traccc algorithms Add logging infrastructure to all algorithms Jan 24, 2025
core/CMakeLists.txt Outdated Show resolved Hide resolved
Copy link
Contributor

@beomki-yeo beomki-yeo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having logger is a good thing but let's find more silent way than this PR. At least I would not include it as an argument of the constructors as it will break all projects that use traccc. Also users are not supposed to struggle with loggers

@stephenswat
Copy link
Member Author

Having logger is a good thing but let's find more silent way than this PR. At least I would not include it as an argument of the constructors as it will break all projects that use traccc. Also users are not supposed to struggle with loggers

Would it be an acceptable solution for you if the logger had a default parameter so the external behaviour of the constructors would remain identical to the current behaviour?

@beomki-yeo
Copy link
Contributor

That would be acceptable

@stephenswat
Copy link
Member Author

There we go, all the algorithms now have default loggers so the external behaviour is unchanged, and the code now uses the ACTS core only conditionally, i.e. if it is already available; so the code continues to work without ACTS core.

Copy link
Contributor

@beomki-yeo beomki-yeo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of suggesting make the loggers in program_options and examples have default arguments as well but anyway they are out of traccc::core.

Copy link
Member

@paulgessinger paulgessinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty messy now.

I suggest we just add ActsCore as a dependency here and start making our code more unified and less complex, rather than increasing complexity.

I think littering our code with preprocessor instructions and duplicating the infrastructure without a good reason is a pretty bad idea and we should refrain from doing this to avoid tech debt.

Copy link
Member

@krasznaa krasznaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without having gone through the entire thing completely...

I agree with @beomki-yeo that I don't want the current traccc::core algorithm to depend on ActsCore. At the same time, clearly future ActsSYCL, ActsCUDA, etc. libraries will publicly depend on ActsCore. So it's a good idea to use a carbon copy of Acts's logger code.

But I would just always use the carbon copy. As long as we have to have a copy in our code, what do we win by sometimes not using it?

I would remove the conditional compilation flag, and just use the "imported" logger as-is.

Now... did you think at all about logging (error reporting...) in the GPU code? Since to me that is the actually interesting part here. For the host-side logging it was clear that we would follow what Acts has been doing since forever. (Which itself is a variation on what Gaudi does. Not sure if you've seen Heretic... Its main premise applies here as well. 😆)

CMakeLists.txt Outdated Show resolved Hide resolved
core/include/traccc/utils/logging.hpp Outdated Show resolved Hide resolved
examples/run/alpaka/seq_example_alpaka.cpp Outdated Show resolved Hide resolved
@stephenswat
Copy link
Member Author

stephenswat commented Jan 27, 2025

But I would just always use the carbon copy. As long as we have to have a copy in our code, what do we win by sometimes not using it?

The problem is that the ACTS and Athena code will be passing in the actual ACTS logger, not our carbon copy. And you and I know that those types are functionally the same, but the compiler obviously doesn't - so it will reject the code with a compilation error.

@stephenswat
Copy link
Member Author

Now... did you think at all about logging (error reporting...) in the GPU code? Since to me that is the actually interesting part here.

AFAIK nobody has come up with a better solution than what I proposed in #640, so as far as I am concerned that is going to be the way to go unless someone comes up with something really fancy!

@krasznaa
Copy link
Member

Now... did you think at all about logging (error reporting...) in the GPU code? Since to me that is the actually interesting part here.

AFAIK nobody has come up with a better solution than what I proposed in #640, so as far as I am concerned that is going to be the way to go unless someone comes up with something really fancy!

Not quite what I meant. 🤔 Though it's good that you reminded me that we'll need some generic functionality for "monitoring" as well.

I was rather thinking of something along the lines of the vecmem logging code.

https://github.com/acts-project/vecmem/blob/main/core/include/vecmem/utils/debug.hpp

But I don't think that quite scales to these algorithms. That's sort of good enough for debugging low-level functionality in special builds of the code, but in this project I would rather go with a combination of

  • warning / error messages, that are always printed when something bad is detected in the GPU code;
  • "monitoring information" being collected through a runtime flag.

We don't have to tackle those here. But we should start more seriously thinking about them. 🤔

@krasznaa
Copy link
Member

But I would just always use the carbon copy. As long as we have to have a copy in our code, what do we win by sometimes not using it?

The problem is that the ACTS and Athena code will be passing in the actual ACTS logger, not our carbon copy. And you and I know that those types are functionally the same, but the compiler obviously doesn't - so it will reject the code with a compilation error.

Wait, why?

Once the code is actually merged into acts-project/acts, then of course we will just have a single class. The current (or maybe slightly improved) Acts::Logger.

But as long as Athena and Acts use this project as an external, why couldn't they construct its own loggers? As long as this carbon copy can forward its messages to an outside logger like Acts::Logger does with MsgStream, I really don't see a problem here.

@paulgessinger
Copy link
Member

paulgessinger commented Jan 27, 2025

@krasznaa why the duplication and tech debt now though? Just adopting the core logger type at this poin we'd be converging things already, rather than increasing divergence.

Unless your argument is NIH, this has basically no downsides at this point.

We will be adding code now with the only goal of removing it for... what exactly? We'll be adding wrapper code to ACTS, we'll be adding wrapper code to Athena, we'll be adding wrapper code for wrapper code.

Or, we could just use the one thing that does exactly what we need and we have at hand already and is already integrated.

I'm sorry to make such a fuzz about this, but there's no good reason not to use a single logging system at this point. We should stop thinking about this as a separate project at some point, and I think that point is now.

@stephenswat
Copy link
Member Author

Okay, I've addressed all the topics of conversation. Clearly we have reached some impasse when it comes to the reliance on the ACTS Core library, so I think the solution of selecting either the ACTS logger or our own carbon copy of it is the best solution until the integration continues. I'd like to move towards merging this now.

This commit adds ACTS-style loggers to all existing traccc algorithms,
and requires that all future algorithms have the same. It also carefully
threads logger objects through the computation chain. Currently, little
is added in the way of useful logging outputs, as that can happen in a
future commit: this is just infrastructure.
Comment on lines +163 to +165
std::unique_ptr<const Logger> m_logger;

const Logger& logger() const override { return *m_logger; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's talk in the afternoon in person. Since I still very much dislike the amount of boilerplate code added in this PR. 😦 Since this is something that we'll keep as a design definitely for a while, I do want to come to something that we can all live with...

Copy link
Member

@paulgessinger paulgessinger Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the ACTS design, if we want to use ACTS_XXX macros (and I strongly think we do), this is what it'll be. So, in a sense, the design is already decided, therefore not much to discuss.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There has to be a way of factorizing this with a mix-in class. I have a very hard time accepting that every single client has to have their own member functions defined separately. We stopped doing that in Athena a decade ago...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That I don't have an opinion on, beyond trying to avoid multiple-inheritance if possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants