Add structural OpenAPI schema to Tekton CRDs

[siamaksade]

Kubernetes 1.15 introduced structural OpenAPI schema which helps with validation, supporting kubectl explain and also building tools around it:

Structural schema should be added for all CRDs introduced by Tekton.

[siamaksade]

This PR is related: #1179

[imjasonh]

Is this something that can be easily generated using existing tooling? I'm a bit worried we'll end up with structs and YAML validation out of sync, and I'm also a bit uncomfortable with maintaining code to generate one from the other. Is there something OpenAPI provides for this?

[vdemeester]

/kind feature
/kind question

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vdemeester]

/remove-lifecycle rotten
/remove-lifecycle stale
/reopen

[tekton-robot]

@vdemeester: Reopened this issue.

In response to this:

/remove-lifecycle rotten
/remove-lifecycle stale
/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bobcatfish]

GitHub recently introduced this too https://github.blog/2020-07-27-introducing-githubs-openapi-description/

[abayer]

Well, kubernetes-sigs/controller-tools#627 (comment) - our layout is awfully nonstandard, and the not-unreasonable response from the maintainer is asking if we can restructure things rather than make a change to schemapatch. I think we could solve this by moving everything in pkg/apis/resource/v1alpha1 and pkg/apis/run/v1alpha1 into pkg/apis/pipeline/v1alpha1 - no changes to CRDs needed, just rearranging the packages. That said, I'm not sure what other possible ramifications could come out of doing that.

[abayer]

Started playing around with what would be involved in moving pkg/apis/resource/v1alpha1 and pkg/apis/run/v1alpha1 and...well, so far, I don't think it's viable without mucking up our types pretty thoroughly.

[vdemeester]

They are in separate packages because of « loop dependency ». We may want to go ahead and reorganize that by duplicating some code and make v1alpha1 and v1beta1 completely independent of each other (code wise) which could be beneficial for the future anyway 😇

[dmikalova]

OpenAPI schemas will also help with managing Tekton through Terraform - without them Terraform doesn't know how to ignore the release annotation in a taskrun which causes Terraform to constantly want to destroy and create the manifest.

[ktarplee]

I want to add that the OpenAPI schema also is really handy for users. I tend to use kubectl explain for CRD to understand what fields are available and what they mean. For Tekon CRs the result is rather uninformative.

$ kubectl explain tasks
KIND:     Task
VERSION:  tekton.dev/v1beta1

DESCRIPTION:
     <empty>

Compare the above to one that has a OpenAPI schema. You can drill down into the spec as far as you like.

$ kubectl explain podmonitors
KIND:     PodMonitor
VERSION:  monitoring.coreos.com/v1

DESCRIPTION:
     PodMonitor defines monitoring for a set of pods.

FIELDS:
   apiVersion   <string>
     APIVersion defines the versioned schema of this representation of an
     object. Servers should convert recognized schemas to the latest internal
     value, and may reject unrecognized values. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

   kind <string>
     Kind is a string value representing the REST resource this object
     represents. Servers may infer this from the endpoint the client submits
     requests to. Cannot be updated. In CamelCase. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

   metadata     <Object>
     Standard object's metadata. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata

   spec <Object> -required-
     Specification of desired Pod selection for target discovery by Prometheus.

[doctorpangloss]

if we're not going to get common recipes for Tekton, we really need the autocomplete in our editors

[abayer]

/unassign

[jbg]

OpenAPI schemas will also help with managing Tekton through Terraform - without them Terraform doesn't know how to ignore the release annotation in a taskrun which causes Terraform to constantly want to destroy and create the manifest.

Just want to second this. Managing Tekton resources with kubernetes_manifest in Terraform is a world of pain due to the lack of a schema. kubectl_manifest is a workaround, but it has its own problems.

[evankanderson]

Adding my voice here, today, that I was looking for this documentation and discovered that not only was the structural schema missing, but there's also no standard Open-API generated schema documentation, only by-hand documentation which doesn't document (for example) status fields in a consistent way.

[vdemeester]

/assign

[chriscannon]

This would be a really nice addition for developer UX.

[RobertDiebels]

Adding another use-case. I posted a question on the discussion board 2 years ago related to this.
The Kubernetes plugin by Jetbrains requries the OpenAPI spec to provide action intentions/auto-completion,

Right now the plugin is showing errors on the "spec" property of the Custom Resources (CR) of Tekton making it effectively useless for all Tekton CRs.

Is anyone working on this? If not I'd be willing to make an attempt.

[vdemeester]

Is anyone working on this? If not I'd be willing to make an attempt.

Not as we speak, so your attempt would be very welcome 🤗 👼🏼

[RobertDiebels]

@vdemeester Alright, I'll get on it at my earliest possible convenience. I have skimmed through most of the material in CONTRIBUTING.md.

While reading the documentation I noticed that:

• A link in pipeline/CONTRIBUTING.md is dead.
  • What should it refer to?
• A link refers to an archived wiki.
  • Should this be changed to the new wiki?
• A link refers to moved source code.
  • Should this be changed to the new source code location?
• A link in plumbing/README.md is dead.
  • Should this be changed to this file?

I have a few questions regarding a possible PR/contribution:

• Which CRDs are part of which tektoncd project?
• Which CRDs are expected to change, only those part of tektoncd/pipeline?
• Where can I find the current CRDs?
• Is adding the openAPIV3schema to Tekton CRDs considered a breaking change?
  • I would estimate it is not, as CRDs should already be validated according to the same rules that ought to be in the schema.
• Should the openAPIV3schema be added to all versions of the CRDs?

[burigolucas]

Hi @vdemeester, hi @RobertDiebels, I am also quite interested to have the schema of the CRDs as I think kubectl explain is a very important for the UX. I have tried to address this issue in the following PR: #8490

I would be very glad for any comments.

[RobertDiebels]

Hi @vdemeester, hi @RobertDiebels, I am also quite interested to have the schema of the CRDs as I think kubectl explain is a very important for the UX. I have tried to address this issue in the following PR: #8490

I would be very glad for any comments.

Looks good afaics, do you have any companion PR's for Triggers, EventListeners, etc.? That's why I asked which CRDs are part of which tektoncd project, because I feel this warrants a consistent approach across all of the tektoncd projects.

[burigolucas]

Looks good afaics, do you have any companion PR's for Triggers, EventListeners, etc.?

No companion PR's yet. I will wait for the maintainers to review that first PR to check for any required changes since there are still issues to be solved on the schema generation.

[RobertDiebels]

Looks good afaics, do you have any companion PR's for Triggers, EventListeners, etc.?

No companion PR's yet. I will wait for the maintainers to review that first PR to check for any required changes since there are still issues to be solved on the schema generation.

Alright, got you. I'll watch the ticket for progress then 👍