Skip to content

Commit

Permalink
Merge branch 'main' into deleting_tbds
Browse files Browse the repository at this point in the history
  • Loading branch information
lovesprung committed Nov 21, 2024
2 parents 2c0abd9 + 2264136 commit ba0ef51
Show file tree
Hide file tree
Showing 56 changed files with 1,854 additions and 1,531 deletions.
22 changes: 13 additions & 9 deletions .github/workflows/proofreading-vale.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
name: reviewdog
on:
pull_request_target:
types: [assigned, opened, edited, ready_for_review]

pull_request:
workflow_dispatch:

env:
REVIEWDOG_GITHUB_API_TOKEN: ${{secrets.VALE_GITHUB_TOKEN}}

jobs:
vale:
name: runner / vale
Expand All @@ -13,19 +14,22 @@ jobs:
- name: Checkout
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7


- name: Harden Runner
uses: step-security/harden-runner@91182cccc01eb5e619899d80e4e971d6181294a7 # v2.10.1
with:
egress-policy: audit


- name: Install prerequisites
run: |
sudo apt-get --allow-releaseinfo-change update -y && sudo apt-get install -y docutils
env
- name: Run Vale
uses: errata-ai/vale-action@91ac403e8d26f5aa1b3feaa86ca63065936a85b6 # reviewdog
env:
REVIEWDOG_GITHUB_API_TOKEN: ${{ secrets.VALE_GITHUB_TOKEN }}
continue-on-error: true
with:
files: '["content", "README.md"]'
# github-pr-check, github-pr-review, github-check
# more info on these: https://github.com/reviewdog/reviewdog#reporters
reporter: github-pr-check
token: ${{secrets.VALE_GITHUB_TOKEN}}
fail_on_error: false
filter_mode: nofilter
token: ${{secrets.VALE_GITHUB_TOKEN}}
3 changes: 3 additions & 0 deletions .linkspector.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ ignorePatterns:
- pattern: "^http://localhost.*$"
- pattern: "^http://HOSTNAME:PORT.*$"
- pattern: "172\\.18\\.255\\.200"
- pattern: "https://\\*kpt\\*\\.dev/"
- pattern: "https://my-gitlab\\.com/joe\\.bloggs/blueprints\\.git"
- pattern: "http://172\\.18\\.0\\.200:3000/nephio/"
replacementPatterns:
- pattern: ".md#.*$"
replacement: ".md"
4 changes: 3 additions & 1 deletion .vale.ini
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ MinAlertLevel = suggestion

# Configure a dictionary specific to Nephio
Vocab = Nephio
SkippedScopes = iframe

[*.md]
BasedOnStyles = Vale, proselint, alex
Expand All @@ -24,4 +25,5 @@ TokenIgnores = ({{[%<] .* [%>]}}.*?{{[%<] ?/.* [%>]}}), \

# Exclude `{{< myshortcode `This is some <b>HTML</b>, ... >}}`
BlockIgnores = (?sm)^({{[%<] \w+ [^{]*?\s[%>]}})\n$, \
(?s) *({{< highlight [^>]* ?>}}.*?{{< ?/ ?highlight >}})
(?s) *({{< highlight [^>]* ?>}}.*?{{< ?/ ?highlight >}})

59 changes: 56 additions & 3 deletions .vale/config/vocabularies/Nephio/accept.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# Accepted words (other acronyms must be explained)
Ansible
Anthos
API
APIs
apiserver
Expand All @@ -7,45 +9,96 @@ ASNs
[Aa]utoscaling
cabundle
[Cc]onfigmap
Codegen
[Cc]loudified
Containerlab
CLI
CR
CRs
CRs
CRD
CRDs
Crossplane
Dockerfile
Docsy
envtest
[Ee]tcd
gcloud
Gitea
GitHub
GitLab
GitOps
Graphviz
GRPC
GVK
GVKs
[Hh]ostname
[Hh]omebrew
html
[Kk]pt
[Kk]ptfile
kubeadm
Kubebuilder
kubectl
kubelet
Kubernetes
Kustomize
Libvirt
MacBook
MetalLB
[Nn]amespace
Nephio
Netlify
NF
NFDeploy
NFDeployment
NFs
[Mm]akefile
Multus
[Mm]ultivendor
[Mm]ulticloud
Nginx
OAuth
OCI
[Oo]nboarding
[Oo]nboarded
Okta
[Pp]arameterization
parameterRef
passwordless
[Pp]kgserver
Podman
[Pp]orch
[Pp]orchctl
Postgres
[Rr]pkg
rootsync
SDK
starlark
stdout
stderr
sudo
[Ss]udo
TLS
[Tt]riage
[Tt]ko
[Tt]oolchain
UI
UIs
URI
URIs
[Uu]ntar
vCPU
VirtualBox
VLAN
VLANs
VM
VMs
VSCode
VSphere
VS Code
vSphere
WebUI
xApps
YAML

# Nephio contributors mentioned in the docs
Tal
Liron
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ The documentation is served from [docs.nephio.org](https://docs.nephio.org/) and

## Status of the documentation

In R2 release, a Hugo / Docsy based documentation site was introduced for the Nephio documentation. Hugo / Docsy uses the Markdown files hosted in the Github repo to generate the documentation website. We are still working on the restructuring the content and finalizing the look and feel of the website.
In R2 release, a Hugo / Docsy based documentation site was introduced for the Nephio documentation. Hugo / Docsy uses
the Markdown files hosted in the GitHub repo to generate the documentation website.

## How to contribute to the documentation

Expand Down
1 change: 1 addition & 0 deletions content/en/docs/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,4 @@ demonstration purposes, the same principles and code can be used for managing
other infrastructure and network functions. The *uniformity in systems*
principle means that as long as something is manageable via the Kubernetes
Resource Model, it is manageable via Nephio.

9 changes: 6 additions & 3 deletions content/en/docs/apis/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,15 @@ relationships, and is based on an
[original document](https://docs.google.com/document/d/1-5nlpY4FbuhWtdKTvIqPOv4bWmA6zx6TdHoEfk9Jc_Q/edit)
developed by Tal Liron.

The aim is to keep these diagrams as simple as possible for now and only convey the important aspects of the modelled entities. As such they are intended to give a high-level overview of the entities and relationships that can be accessed and modified via the Nephio API, and provide reference to detailed documentation, generated from the code, where available.

The aim is to keep these diagrams as simple as possible for now and only convey the important aspects of the modelled
entities. As such they are intended to give a high-level overview of the entities and relationships that can be accessed
and modified via the Nephio API, and provide reference to detailed documentation, generated from the code, where
available.

## Topology and Network APIs

This is a high-level overview of the Nephio models and their relationships, with links to the relevant API documentation where available, and to the source code where not.
This is a high-level overview of the Nephio models and their relationships, with links to the relevant API documentation
where available, and to the source code where not.

```mermaid
flowchart TD
Expand Down
2 changes: 1 addition & 1 deletion content/en/docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The system context view gives a high level perspective of the Nephio software sy

![System Landscape](/static/images/architecture/level2-nephio-container.png)

Nephio is an amalgamation of software systems, so a system landscape provides a high-level view of how those software systems interoperate.
Nephio is an amalgamation of software systems, so a system landscape provides a high-level view of how those software systems operate together.

## Component Views

Expand Down
8 changes: 4 additions & 4 deletions content/en/docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ weight: 4
---

We use many terms in our Nephio discussions, coming from different domains
including telco, Kubernetes, configuration management, and our own
including telecom, Kubernetes, configuration management, and our own
Nephio-specific terms. This glossary is intended to help clarify our usage of
these terms.

Expand Down Expand Up @@ -141,7 +141,7 @@ Day 2 and beyond, all of which are challenges with current techniques.
Hydration may be *out-of-place*, where the source material (e.g., the Helm
chart), is separate from the output of the hydration process (the manifests).
This is probably the most familiar type of hydration, used by Helm and
kustomize, for example. Think of it as a pipeline with an input artifact, input
Kustomize, for example. Think of it as a pipeline with an input artifact, input
values, and output artifacts.

Hydration may also be *in-place*, where modifications are directly written to
Expand All @@ -159,7 +159,7 @@ outcomes achievable. This leads to "over-parameterization" - where effectively
every option possible in the outputs becomes an option in the input. At that
point, you have mostly *moved* complexity rather than *reduced* complexity.
In-place hydration can help with the over-parameterization, as values that are
rarely changed by users can simply be edited in-place.
rarely changed by users can be edited in-place.

While related, *DRY* and *WET* are not exactly the same concepts as *in-place* and
*out-of-place* hydration. The former two refer to principles, whereas the latter
Expand Down Expand Up @@ -218,7 +218,7 @@ with KRM on the input and output, rather than simple streams.
Generally, best practices suggest KRM functions be hermetic (that is, they do
not access the outside world).

In terms of the specification linked above, kustomize, kpt, and Porch are all
In terms of the specification linked above, Kustomize, kpt, and Porch are all
*orchestrators*.

*See also*: [Controller](#controller), [kpt](#kpt), [Porch](#porch)
Expand Down
2 changes: 1 addition & 1 deletion content/en/docs/guides/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ weight: 2
---

{{% pageinfo %}}
This page is draft and the separation of the content to different categories is not clearly done.
This page is draft and the separation of the content to different categories is not done.
{{% /pageinfo %}}


Expand Down
2 changes: 1 addition & 1 deletion content/en/docs/guides/contributor-guides/documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ weight: 5
The Nephio documentation is built with the .md / Hugo / Docsy / Netlify framework.

* **.md:** The documentation itself is written in Markdown (,md) with some Hugo and Docsy related extensions. The .md
files are stored and managed in a Git repo in [nephio-project/docs](https://github.com/nephio-project/docs).
files are stored and managed in a Git repository in [nephio-project/docs](https://github.com/nephio-project/docs).
* **Hugo:** [Hugo](https://gohugo.io/) is used to render the documentation fo static html pages.
* **Docsy:** [Docsy](https://www.docsy.dev/) is a theme for Hugo what we use to provide the basic structure and look
and feel of the documentation.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ weight: 1
The [Helm to Operator Codegen SDK](https://github.com/nephio-project/nephio-sdk/tree/main/helm-to-operator-codegen-sdk) offers a streamlined solution for translating existing Helm charts into Kubernetes operators with minimal effort and cost.

## The Flow Diagram
In a nutshell, the Helm-Charts are converted to YAML files using the values provided in "values.yaml". Then, each Kubernetes Resource Model (KRM) in the YAML is translated into Go code, employing one of two methods.
In a nutshell, the Helm-Charts are converted to YAML files using the values provided in *values.yaml*. Then, each Kubernetes Resource Model (KRM) in the YAML is translated into Go code, employing one of two methods.
1) If the resource is Runtime-Supported, it undergoes a conversion process where the KRM resource is first transformed into a Runtime Object, then into JSON, and finally into Go code.
2) Otherwise, if the resource is not Runtime-Supported, it is converted into an Unstructured Object and then into Go code.

Expand All @@ -20,7 +20,7 @@ Helm to YAML conversion is achieved by running the following command
`helm template <chart> --namespace <namespace> --output-dir “temp/templated/”` internally. As of now, it retrieves the values from default "values.yaml"

### Flow-2: YAML Split
The SDK iterates over each YAML file in the "converted-yamls" directory. If a YAML file contains multiple Kubernetes Resource Models (KRM), separated by "---", the SDK splits the YAML file accordingly to isolate each individual KRM resource. This ensures that each KRM resource is processed independently.
The SDK iterates over each YAML file in the *converted-yamls* directory. If a .yaml file contains multiple Kubernetes Resource Models (KRM), separated by "---", the SDK splits the .yaml file accordingly to isolate each individual KRM resource. This ensures that each KRM resource is processed independently.

### Runtime-Object and Unstruct-Object
The SDK currently employs the "runtime-object method" to handle Kubernetes resources whose structure is recognized by Kubernetes by default. Examples of such resources include Deployment, Service, and ConfigMap. Conversely, resources that are not inherently known to Kubernetes and require explicit installation or definition, such as Third-Party Custom Resource Definitions (CRDs) like NetworkAttachmentDefinition or PrometheusRule, are processed using the "unstructured-object" method. Such examples are given below:
Expand Down Expand Up @@ -69,15 +69,14 @@ networkAttachmentDefinition1 := &unstructured.Unstructured{
```

### Flow-3.1: KRM to Runtime-Object
The conversion process relies on the "k8s.io/apimachinery/pkg/runtime" package. Currently, only the API version "v1" is supported. The supported kinds for the Runtime Object method include:
`Deployment, Service, Secret, Role, RoleBinding, ClusterRoleBinding, PersistentVolumeClaim, StatefulSet, ServiceAccount, ClusterRole, PriorityClass, ConfigMap`
The conversion process relies on the "k8s.io/apimachinery/pkg/runtime" package. Currently, only the API version "v1" is supported. The supported kinds for the Runtime Object method include: Deployment, Service, Secret, Role, RoleBinding, ClusterRoleBinding, PersistentVolumeClaim, StatefulSet, ServiceAccount, ClusterRole, PriorityClass, ConfigMap

### Flow-3.2: Runtime-Object to JSON
Firstly, the SDK performs a typecast of the runtime object to its actual data type. For instance, if the Kubernetes Kind is "Service," the SDK typecasts the runtime object to the specific data type corev1.Service. Then, it conducts a Depth-First Search (DFS) traversal over the corev1.Service object using reflection. During this traversal, the SDK generates a JSON structure that encapsulates information about the struct hierarchy, including corresponding data types and values. This transformation results in a JSON representation of the corev1.Service object's structure and content.

#### DFS Algorithm Cases

The DFS function iterates over the runtime object, traversing its structure in a Depth-First Search manner. During this traversal, it constructs the JSON structure while inspecting each attribute for its data type and value. Attributes that have default values in the runtime object but are not explicitly set in the YAML file are omitted from the conversion process. This ensures that only explicitly defined attributes with their corresponding values are included in the resulting JSON structure. The function follows this flow to accurately capture the structure, data types, and values of the Kubernetes resource while excluding default attributes that are not explicitly configured in the YAML file.
The DFS function iterates over the runtime object, traversing its structure in a Depth-First Search manner. During this traversal, it constructs the JSON structure while inspecting each attribute for its data type and value. Attributes that have default values in the runtime object but are not explicitly set in the .yaml file are omitted from the conversion process. This ensures that only explicitly defined attributes with their corresponding values are included in the resulting JSON structure. The function follows this flow to accurately capture the structure, data types, and values of the Kubernetes resource while excluding default attributes that are not explicitly configured in the .yaml file.


A) Base-Cases:
Expand Down Expand Up @@ -156,7 +155,7 @@ spec:
```

### Flow-3.3: JSON to String (Go-Code)
The SDK reads the JSON file containing the information about the Kubernetes resource and then translates this information into a string of Go code. This process involves parsing the JSON structure and generating corresponding Go code strings based on the structure, data types, and values extracted from the JSON representation. Ultimately, this results in a string that represents the Kubernetes resource in a format compatible with Go code.
The SDK reads the .json file containing the information about the Kubernetes resource and then translates this information into a string of Go code. This process involves parsing the JSON structure and generating corresponding Go code strings based on the structure, data types, and values extracted from the JSON representation. Ultimately, this results in a string that represents the Kubernetes resource in a format compatible with Go code.

#### TraverseJSON Cases (Json-to-String)
The traverse JSON function is responsible for converting JSON data into Go code. Here's how it handles base cases:
Expand Down Expand Up @@ -275,12 +274,12 @@ Structs need to be initialized using curly brackets {}, whereas enums need Paren

Solution: We solve the above problems by building an “enumModuleMapping” which is a set that stores all data types that are enums. i.e. If a data type belongs to the set, then It is an Enum.

There is an automation-script that takes the types.go files of packages and build the config-json. For details, Please refer [here](https://github.com/nephio-project/nephio-sdk/tree/main/helm-to-operator-codegen-sdk/config)
There is an automation-script that takes the *types.go* files of packages and build the config-json. For details, Please refer [here](https://github.com/nephio-project/nephio-sdk/tree/main/helm-to-operator-codegen-sdk/config)


### Flow-4: KRM to Unstruct-Obj to String(Go-code)
All Kubernetes resource kinds that are not supported by the runtime-object method are handled using the unstructured method. In this approach, the Kubernetes Resource MOdel (KRM) is converted to an unstructured object using the package "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured".
Then, We traverse the unstructured-Obj in a DFS fashion and build the gocode-string.
All Kubernetes resource kinds that are not supported by the runtime-object method are handled using the unstructured method. In this approach, the Kubernetes Resource MOdel (KRM) is converted to an unstructured object using the package *k8s.io/apimachinery/pkg/apis/meta/v1/unstructured*"*.
Then, we traverse the unstructured-Obj in a DFS fashion and build the gocode-string.


#### DFS Algorithm Cases (Unstruct-Version)
Expand Down
Loading

0 comments on commit ba0ef51

Please sign in to comment.