Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Counted keyword synthetic source: inherit source keep mode from index settings #120678

Conversation

jordan-powers
Copy link
Contributor

@jordan-powers jordan-powers commented Jan 22, 2025

This PR adds a mechanism to track the value of the synthetic_source_keep setting if it is set in the index settings. This value is then used to properly implement synthetic source support in the counted_keyword field type.

This PR also adds tests to verify that if synthetic_source_keep is set in the index settings, it is respected by the CountedKeywordFieldMapper.

Unlike #120426, this implementation does not inherit synthetic_source_keep from ancestor object nodes.

Follow-up to #120078
Relates to #109796

@jordan-powers jordan-powers added >enhancement auto-backport Automatically create backport pull requests when merged :StorageEngine/Mapping The storage related side of mappings v9.0.0 v8.18.0 labels Jan 22, 2025
@jordan-powers jordan-powers marked this pull request as ready for review January 23, 2025 17:21
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

assertThat(actual, equalTo(expected));
}

public void testSyntheticSourceInheritsKeepAll() throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This functionality doesn't exist right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it's actually just testing that the object mapper is capturing the source. If you want, I can remove the test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such test should exist elsewhere i would remove it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do!


---

"synthetic_source_keep value is respected":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we test nested arrays anywhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's tested in MapperTestCase::testSyntheticSourceKeepArrays, but I'll add a few cases to the yamlRestTest too.

@jordan-powers
Copy link
Contributor Author

Backport PR for 8.x opened, #120871

elasticsearchmachine pushed a commit that referenced this pull request Jan 27, 2025
… (#120871)

This patch adds a property to CountedKeywordMapper to track the
synthetic_source_keep index setting. This property is then used to properly
implement synthetic source support in the counted_keyword field type, with
fallback to the ignore_source mechanism when synthetic_source_keep is set
in either the field mapping or the index settings.
@jordan-powers jordan-powers deleted the counted-keyword-inherit-index-keep-source branch January 27, 2025 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged >enhancement :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants