AbstractAggregator: Aggregation operations are no longer transparent downstream

[ppanopticon]

@rahelarnold98 and I just ran into an issue in the XReco context. Due to the recent changes to the AbstractAggregator, the "aggregated" Retrievable no longer holds only aggregated content. Instead, it holds the original content plus the aggregated content in a single Retrievable .

The question now is: By what mechanism can we restrict downstream operators to only work with the aggregated content? It's one thing if an operator is built from scratch. In our case, however, we mostly rely on existing operators that are being configured.

Quite frankly, this change broke our complete video extraction pipeline.

[lucaro]

That is the downside of the append-only approach. We do have a mechanism to check the author of a content element via the ContentAuthorAttribute, so at least they are distinguishable.

[ppanopticon]

While this may very well be, I currently don't see a way to leverage this in a configurable fashion (i.e., without changing all the operators). Or have I overlooked something?

[faberf]

Hey @ppanopticon sorry for just now getting to this. Yes, with the approach we are going with all of the downstream consumers must filter the content using the contentauthorattribute using a configurable value in the extraction configuration. You can check the FES Extractor class for an example.

[ppanopticon]

Okay I see - thanks for the hint.

However, I would expect the author(s) of such a breaking change to actually adjust existing operators such that they can work as they did before opening a PR. In its current state, the change breaks the pipeline.

[faberf]

Yes, I agree, I actually hadn't considered that this change is breaking before. I will push a fix to dev today, and will edit the wiki.

[ppanopticon]

Fixed by PR #103