Fix common query parameters #3297

swallez · 2024-12-12T18:31:38Z

Fix common query parameter issues:

cat APIs: master_timeout is removed from CommonCatQueryParameters and added where relevant
the cat.help response body is changed to string as there's no JSON representation (note that the s and help parameters present in the json spec do not exist in the code)
fix all master_timeout and timeout issues

These changes should also apply to 8.x and can be backported. They remove 109 validation errors 🎉

The remaining query parameter issues seems to be specific to the 9.x branch and require closer investigation.

pquentin

Thanks! Going down to 58 errors is incredible, but that's also a lot of review time. I've reviewed the cat APIs for now: 23/80 files. It would make sense to create a standalone pull request with just those files.

Two things that I've not mentioned in my inline comments:

aliases, indices and recovery APIs support IndicesOptions (expand_wildcards, allow_no_indices, ignore_throttled and ignore_available, mostly) but they're not in the rest-api-spec.
Many APIs, such as the alias API, can get the alias name from the path and the query string. GET _cat/aliases?alias=.kibana is a thing. I don't know if it should be encoded.

specification/cat/segments/CatSegmentsRequest.ts

specification/cat/tasks/CatTasksRequest.ts

pquentin

I have 26 files left. I do suggest merging #3340 separately.

specification/ilm/delete_lifecycle/DeleteLifecycleRequest.ts

specification/ilm/explain_lifecycle/ExplainLifecycleRequest.ts

specification/ilm/get_lifecycle/GetLifecycleRequest.ts

specification/ilm/put_lifecycle/PutLifecycleRequest.ts

specification/ilm/start/StartIlmRequest.ts

specification/ilm/stop/StopIlmRequest.ts

specification/ingest/delete_geoip_database/DeleteGeoipDatabaseRequest.ts

specification/ingest/put_geoip_database/PutGeoipDatabaseRequest.ts

specification/license/post_start_trial/StartTrialLicenseRequest.ts

pquentin

Done! As expected, many rest-api-spec files were incorrect.

specification/ml/delete_trained_model/MlDeleteTrainedModelRequest.ts

specification/shutdown/delete_node/ShutdownDeleteNodeRequest.ts

specification/shutdown/put_node/ShutdownPutNodeRequest.ts

specification/slm/put_lifecycle/PutSnapshotLifecycleRequest.ts

specification/xpack/info/XPackInfoRequest.ts

github-actions · 2025-01-08T06:14:40Z

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
`ccr.delete_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.follow_info`	⚪	Missing test	Missing test
`ccr.follow_stats`	⚪	Missing test	Missing test
`ccr.follow`	⚪	Missing test	Missing test
`ccr.forget_follower`	⚪	Missing test	Missing test
`ccr.get_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.pause_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.pause_follow`	⚪	Missing test	Missing test
`ccr.put_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.resume_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.resume_follow`	⚪	Missing test	Missing test
`ccr.stats`	⚪	Missing test	Missing test
`ccr.unfollow`	⚪	Missing test	Missing test
`cluster.allocation_explain`	🟢	4/4	4/4
`cluster.delete_voting_config_exclusions`	🟢	1/1	1/1
`cluster.post_voting_config_exclusions`	🟢	5/5	5/5
`cluster.put_component_template`	🔴	21/29	29/29
`enrich.delete_policy`	🟢	9/9	9/9
`enrich.execute_policy`	🟢	9/9	9/9
`enrich.get_policy`	🟢	6/6	6/6
`enrich.put_policy`	🟢	10/10	10/10
`enrich.stats`	🟢	6/6	6/6
`features.get_features`	🟢	2/2	2/2
`features.reset_features`	⚪	Missing test	Missing test
`ilm.delete_lifecycle`	🟢	2/2	2/2
`ilm.explain_lifecycle`	🟢	5/5	5/5
`ilm.get_lifecycle`	🟢	8/8	8/8
`ilm.put_lifecycle`	🟢	6/6	6/6
`ilm.start`	⚪	Missing test	Missing test
`ilm.stop`	🟢	4/4	4/4
`indices.field_usage_stats`	🟢	5/5	5/5
`ingest.delete_geoip_database`	🟢	1/1	1/1
`ingest.get_geoip_database`	🟢	6/6	6/6
`ingest.put_geoip_database`	🟢	3/3	3/3
`license.delete`	🟢	3/3	3/3
`license.post_start_basic`	🟢	5/5	5/5
`license.post_start_trial`	🟢	2/2	2/2
`license.post`	🟢	1/1	1/1
`ml.delete_trained_model`	🟢	5/5	5/5
`ml.get_memory_stats`	🟢	6/6	6/6
`nodes.hot_threads`	🟢	5/5	5/5
`nodes.info`	🔴	114/114	1/114
`nodes.stats`	🔴	51/51	13/51
`search_shards`	🟢	7/7	7/7
`searchable_snapshots.clear_cache`	🟢	1/1	1/1
`shutdown.delete_node`	⚪	Missing test	Missing test
`shutdown.get_node`	⚪	Missing test	Missing test
`slm.put_lifecycle`	🟢	4/4	4/4
`snapshot.clone`	🟢	5/5	5/5
`tasks.list`	🟢	9/9	9/9
`transform.reset_transform`	🟢	4/4	4/4
`watcher.start`	🟢	1/1	1/1
`watcher.stop`	🟢	1/1	1/1
`xpack.info`	🟢	8/8	4/4

You can validate these APIs yourself by using the make validate target.

github-actions · 2025-01-08T07:17:31Z

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
`ccr.delete_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.follow_info`	⚪	Missing test	Missing test
`ccr.follow_stats`	⚪	Missing test	Missing test
`ccr.follow`	⚪	Missing test	Missing test
`ccr.forget_follower`	⚪	Missing test	Missing test
`ccr.get_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.pause_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.pause_follow`	⚪	Missing test	Missing test
`ccr.put_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.resume_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.resume_follow`	⚪	Missing test	Missing test
`ccr.stats`	⚪	Missing test	Missing test
`ccr.unfollow`	⚪	Missing test	Missing test
`cluster.allocation_explain`	🟢	4/4	4/4
`cluster.delete_voting_config_exclusions`	🟢	1/1	1/1
`cluster.post_voting_config_exclusions`	🟢	5/5	5/5
`cluster.put_component_template`	🔴	21/29	29/29
`enrich.delete_policy`	🟢	9/9	9/9
`enrich.execute_policy`	🟢	9/9	9/9
`enrich.get_policy`	🟢	6/6	6/6
`enrich.put_policy`	🟢	10/10	10/10
`enrich.stats`	🟢	6/6	6/6
`features.get_features`	🟢	2/2	2/2
`features.reset_features`	⚪	Missing test	Missing test
`ilm.delete_lifecycle`	🟢	2/2	2/2
`ilm.explain_lifecycle`	🟢	5/5	5/5
`ilm.get_lifecycle`	🟢	8/8	8/8
`ilm.put_lifecycle`	🟢	6/6	6/6
`ilm.start`	⚪	Missing test	Missing test
`ilm.stop`	🟢	4/4	4/4
`indices.field_usage_stats`	🟢	5/5	5/5
`ingest.get_geoip_database`	🟢	6/6	6/6
`license.delete`	🟢	3/3	3/3
`license.post_start_basic`	🟢	5/5	5/5
`license.post_start_trial`	🟢	2/2	2/2
`license.post`	🟢	1/1	1/1
`ml.delete_trained_model`	🟢	5/5	5/5
`ml.get_memory_stats`	🟢	6/6	6/6
`nodes.hot_threads`	🟢	5/5	5/5
`nodes.info`	🔴	114/114	1/114
`nodes.stats`	🔴	51/51	13/51
`search_shards`	🟢	7/7	7/7
`searchable_snapshots.clear_cache`	🟢	1/1	1/1
`shutdown.get_node`	⚪	Missing test	Missing test
`slm.put_lifecycle`	🟢	4/4	4/4
`snapshot.clone`	🟢	5/5	5/5
`tasks.list`	🟢	9/9	9/9
`transform.reset_transform`	🟢	4/4	4/4
`watcher.start`	🟢	1/1	1/1
`watcher.stop`	🟢	1/1	1/1

You can validate these APIs yourself by using the make validate target.

github-actions · 2025-01-08T07:27:37Z

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
`ccr.delete_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.follow_info`	⚪	Missing test	Missing test
`ccr.follow_stats`	⚪	Missing test	Missing test
`ccr.follow`	⚪	Missing test	Missing test
`ccr.forget_follower`	⚪	Missing test	Missing test
`ccr.get_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.pause_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.pause_follow`	⚪	Missing test	Missing test
`ccr.put_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.resume_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.resume_follow`	⚪	Missing test	Missing test
`ccr.stats`	⚪	Missing test	Missing test
`ccr.unfollow`	⚪	Missing test	Missing test
`cluster.allocation_explain`	🟢	4/4	4/4
`cluster.delete_voting_config_exclusions`	🟢	1/1	1/1
`cluster.post_voting_config_exclusions`	🟢	5/5	5/5
`cluster.put_component_template`	🔴	21/29	29/29
`enrich.delete_policy`	🟢	9/9	9/9
`enrich.execute_policy`	🟢	9/9	9/9
`enrich.get_policy`	🟢	6/6	6/6
`enrich.put_policy`	🟢	10/10	10/10
`enrich.stats`	🟢	6/6	6/6
`features.get_features`	🟢	2/2	2/2
`features.reset_features`	⚪	Missing test	Missing test
`ilm.explain_lifecycle`	🟢	5/5	5/5
`ilm.start`	⚪	Missing test	Missing test
`ilm.stop`	🟢	4/4	4/4
`indices.field_usage_stats`	🟢	5/5	5/5
`ingest.get_geoip_database`	🟢	6/6	6/6
`license.delete`	🟢	3/3	3/3
`license.post_start_basic`	🟢	5/5	5/5
`license.post_start_trial`	🟢	2/2	2/2
`license.post`	🟢	1/1	1/1
`ml.delete_trained_model`	🟢	5/5	5/5
`ml.get_memory_stats`	🟢	6/6	6/6
`nodes.hot_threads`	🟢	5/5	5/5
`nodes.info`	🔴	114/114	1/114
`nodes.stats`	🔴	51/51	13/51
`search_shards`	🟢	7/7	7/7
`searchable_snapshots.clear_cache`	🟢	1/1	1/1
`shutdown.get_node`	⚪	Missing test	Missing test
`snapshot.clone`	🟢	5/5	5/5
`tasks.list`	🟢	9/9	9/9
`transform.reset_transform`	🟢	4/4	4/4
`watcher.start`	🟢	1/1	1/1
`watcher.stop`	🟢	1/1	1/1

You can validate these APIs yourself by using the make validate target.

l-trotta · 2025-01-08T11:42:25Z

specification/cluster/put_component_template/ClusterPutComponentTemplateRequest.ts

+     * Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
+     * @server_default 30s
+     */
+    timeout?: Duration


does this class match this server rest action? if so then there's no timeout (and we're missing some other parameters)

Good catch, thank you! Fixed in c84c876 (#3297)

l-trotta · 2025-01-08T12:08:02Z

checked everything, ready to approve once my doubt on ClusterPutComponentTemplateRequest will been cleared

github-actions · 2025-01-08T13:46:45Z

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
`ccr.delete_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.follow_info`	⚪	Missing test	Missing test
`ccr.follow_stats`	⚪	Missing test	Missing test
`ccr.follow`	⚪	Missing test	Missing test
`ccr.forget_follower`	⚪	Missing test	Missing test
`ccr.get_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.pause_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.pause_follow`	⚪	Missing test	Missing test
`ccr.put_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.resume_auto_follow_pattern`	⚪	Missing test	Missing test
`ccr.resume_follow`	⚪	Missing test	Missing test
`ccr.stats`	⚪	Missing test	Missing test
`ccr.unfollow`	⚪	Missing test	Missing test
`cluster.allocation_explain`	🟢	4/4	4/4
`cluster.delete_voting_config_exclusions`	🟢	1/1	1/1
`cluster.post_voting_config_exclusions`	🟢	5/5	5/5
`enrich.delete_policy`	🟢	9/9	9/9
`enrich.execute_policy`	🟢	9/9	9/9
`enrich.get_policy`	🟢	6/6	6/6
`enrich.put_policy`	🟢	10/10	10/10
`enrich.stats`	🟢	6/6	6/6
`features.get_features`	🟢	2/2	2/2
`features.reset_features`	⚪	Missing test	Missing test
`ilm.explain_lifecycle`	🟢	5/5	5/5
`ilm.start`	⚪	Missing test	Missing test
`ilm.stop`	🟢	4/4	4/4
`indices.field_usage_stats`	🟢	5/5	5/5
`ingest.get_geoip_database`	🟢	6/6	6/6
`license.delete`	🟢	3/3	3/3
`license.post_start_basic`	🟢	5/5	5/5
`license.post_start_trial`	🟢	2/2	2/2
`license.post`	🟢	1/1	1/1
`ml.delete_trained_model`	🟢	5/5	5/5
`ml.get_memory_stats`	🟢	6/6	6/6
`nodes.hot_threads`	🟢	5/5	5/5
`nodes.info`	🔴	114/114	1/114
`nodes.stats`	🔴	51/51	13/51
`search_shards`	🟢	7/7	7/7
`searchable_snapshots.clear_cache`	🟢	1/1	1/1
`shutdown.get_node`	⚪	Missing test	Missing test
`snapshot.clone`	🟢	5/5	5/5
`tasks.list`	🟢	9/9	9/9
`transform.reset_transform`	🟢	4/4	4/4
`watcher.start`	🟢	1/1	1/1
`watcher.stop`	🟢	1/1	1/1

You can validate these APIs yourself by using the make validate target.

l-trotta

LGTM!

* Fix many _cat API issues * Fix "master_timeout" query parameter issues * Fix "timeout" query parameter issues * Fix "pretty" & "human" query parameter issues * Use correct type for the 'time' query parameter * Address review feedback * Reduce diff with main branch * Remove timeout parameter from put_component_template --------- Co-authored-by: Quentin Pradet <[email protected]> (cherry picked from commit 7aa6d9c)

github-actions · 2025-01-08T14:31:18Z

The backport to 8.17 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.17 8.17
# Navigate to the new working tree
cd .worktrees/backport-8.17
# Create a new branch
git switch --create backport-3297-to-8.17
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 7aa6d9cd53e143573e1de75ea225e4a04cdd8dc9
# Push it to GitHub
git push --set-upstream origin backport-3297-to-8.17
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.17

Then, create a pull request where the base branch is 8.17 and the compare/head branch is backport-3297-to-8.17.

* Fix many _cat API issues * Fix "master_timeout" query parameter issues * Fix "timeout" query parameter issues * Fix "pretty" & "human" query parameter issues * Use correct type for the 'time' query parameter * Address review feedback * Reduce diff with main branch * Remove timeout parameter from put_component_template --------- Co-authored-by: Quentin Pradet <[email protected]> (cherry picked from commit 7aa6d9c)

* Fix common query parameters (#3297) * Fix many _cat API issues * Fix "master_timeout" query parameter issues * Fix "timeout" query parameter issues * Fix "pretty" & "human" query parameter issues * Use correct type for the 'time' query parameter * Address review feedback * Reduce diff with main branch * Remove timeout parameter from put_component_template --------- Co-authored-by: Quentin Pradet <[email protected]> (cherry picked from commit 7aa6d9c) * correct rebase * this time correct rebase --------- Co-authored-by: Sylvain Wallez <[email protected]> Co-authored-by: Laura Trotta <[email protected]>

swallez added specification backport 8.17 labels Dec 12, 2024

swallez requested review from a team as code owners December 12, 2024 18:31

swallez added 5 commits December 12, 2024 19:35

Fix many _cat API issues

9235caf

Fix "master_timeout" query parameter issues

d43877e

Fix "timeout" query parameter issues

c7d9e0a

Fix "pretty" & "human" query parameter issues

dc06dcc

Use correct type for the 'time' query parameter

bbe5d70

swallez force-pushed the fix-query-parameters branch from 1dabf33 to bbe5d70 Compare December 12, 2024 18:44

This comment was marked as outdated.

Sign in to view

Merge remote-tracking branch 'origin/main' into fix-query-parameters

5ccb4e4

pquentin added the backport 8.x label Dec 13, 2024

This comment was marked as outdated.

Sign in to view

pquentin reviewed Dec 16, 2024

View reviewed changes

specification/cat/segments/CatSegmentsRequest.ts Outdated Show resolved Hide resolved

specification/cat/tasks/CatTasksRequest.ts Outdated Show resolved Hide resolved

pquentin mentioned this pull request Dec 17, 2024

Fix cat common query parameters #3340

Merged

pquentin reviewed Dec 17, 2024

View reviewed changes

Merge remote-tracking branch 'origin/main' into fix-query-parameters

99c81bb

This comment was marked as outdated.

Sign in to view

This was referenced Dec 18, 2024

Add missing parameter to xpack.info rest-api-spec elastic/elasticsearch#118954

Merged

Add missing timeouts to rest-api-spec SLM APIs elastic/elasticsearch#118958

Merged

pquentin mentioned this pull request Jan 2, 2025

Update rest-api-spec query parameters #3405

Merged

Merge remote-tracking branch 'origin/main' into fix-query-parameters

28e1971

Address review feedback

eeac082

Reduce diff with main branch

3be7359

l-trotta reviewed Jan 8, 2025

View reviewed changes

Remove timeout parameter from put_component_template

c84c876

l-trotta approved these changes Jan 8, 2025

View reviewed changes

pquentin merged commit 7aa6d9c into main Jan 8, 2025
8 checks passed

pquentin deleted the fix-query-parameters branch January 8, 2025 14:30

github-actions bot mentioned this pull request Jan 8, 2025

[Backport 8.x] Fix common query parameters #3464

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix common query parameters #3297

Fix common query parameters #3297

swallez commented Dec 12, 2024

This comment was marked as outdated.

This comment was marked as outdated.

pquentin left a comment

pquentin left a comment

pquentin left a comment

This comment was marked as outdated.

github-actions bot commented Jan 8, 2025

github-actions bot commented Jan 8, 2025

github-actions bot commented Jan 8, 2025

l-trotta Jan 8, 2025

pquentin Jan 8, 2025

l-trotta commented Jan 8, 2025

github-actions bot commented Jan 8, 2025

l-trotta left a comment

github-actions bot commented Jan 8, 2025

Fix common query parameters #3297

Fix common query parameters #3297

Conversation

swallez commented Dec 12, 2024

This comment was marked as outdated.

This comment was marked as outdated.

pquentin left a comment

Choose a reason for hiding this comment

pquentin left a comment

Choose a reason for hiding this comment

pquentin left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

github-actions bot commented Jan 8, 2025

github-actions bot commented Jan 8, 2025

github-actions bot commented Jan 8, 2025

l-trotta Jan 8, 2025

Choose a reason for hiding this comment

pquentin Jan 8, 2025

Choose a reason for hiding this comment

l-trotta commented Jan 8, 2025

github-actions bot commented Jan 8, 2025

l-trotta left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 8, 2025