Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Setting vsphere cloud_provider prevents nodes from getting INTERNAL-IP #2890

Open
waldner opened this issue Mar 29, 2022 · 32 comments
Open

Setting vsphere cloud_provider prevents nodes from getting INTERNAL-IP #2890

waldner opened this issue Mar 29, 2022 · 32 comments

Comments

@waldner
Copy link

waldner commented Mar 29, 2022 •
edited
Loading

RKE version:

1.3.8 (but happens with earlier versions too)

Docker version: (docker version,docker info preferred)

[rke@rke-0 ~]$ docker version
Client: Docker Engine - Community
 Version:           20.10.14
 API version:       1.41
 Go version:        go1.16.15
 Git commit:        a224086
 Built:             Thu Mar 24 01:47:44 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.14
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.15
  Git commit:       87a90dc
  Built:            Thu Mar 24 01:46:10 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.5.11
  GitCommit:        3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc:
  Version:          1.0.3
  GitCommit:        v1.0.3-0-gf46b6ba
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
[rke@rke-0 ~]$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.1-docker)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 22
  Running: 12
  Paused: 0
  Stopped: 10
 Images: 18
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc version: v1.0.3-0-gf46b6ba
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.18.0-348.20.1.el8_5.x86_64
 Operating System: Rocky Linux 8.5 (Green Obsidian)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.585GiB
 Name: rke-0
 ID: SZFJ:DV3F:EU4K:UNGP:4GRQ:3H3Z:LFH2:TGHO:TJA6:MMFB:AVX7:I26K
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Operating system and kernel: (cat /etc/os-release, uname -r preferred)

[rke@rke-0 ~]$ cat /etc/os-release 
NAME="Rocky Linux"
VERSION="8.5 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.5"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.5 (Green Obsidian)"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
ROCKY_SUPPORT_PRODUCT="Rocky Linux"
ROCKY_SUPPORT_PRODUCT_VERSION="8"
[rke@rke-0 ~]$ uname -r
4.18.0-348.20.1.el8_5.x86_64

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)

VMWare VSphere VMs

cluster.yml file:

nodes:
- address: 10.90.0.118
  port: "22"
  internal_address: 10.90.0.118
  role:
  - controlplane
  - etcd
  - worker
  hostname_override: rke-0
  user: rke
  docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: "/tmp/ssh_key"
  ssh_cert: ""
  ssh_cert_path: ""
  labels: {}
  taints: []
network:
  plugin: flannel
authentication:
  strategy: x509
addons: ""
addons_include: []
ssh_key_path: ""
ssh_cert_path: ""
ssh_agent_auth: false
authorization:
  mode: rbac
  options: {}
ignore_docker_version: false
enable_cri_dockerd: null
kubernetes_version: v1.22.4-rancher1-1
private_registries: []
ingress:
  provider: none
cluster_name: "RKE1"
cloud_provider:
  name: vsphere
  vsphereCloudProvider:
    global:
      insecure-flag: true
    virtual_center:
      vcenter.example.com:
        user: [email protected]
        password: secretpassword
        port: "443"
        datacenters: DC1
    workspace:
      server: vcenter.example.com
      datacenter: DC1
      folder: vm/RKE
      default-datastore: vsanDatastore

Steps to Reproduce:

$ rke up
...
$ KUBECONFIG=kube_config_cluster.yml kubectl get nodes -o wide
NAME    STATUS   ROLES                      AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                           KERNEL-VERSION                 CONTAINER-RUNTIME
rke-0   Ready    controlplane,etcd,worker   40s   v1.22.4   <none>        <none>        Rocky Linux 8.5 (Green Obsidian)   4.18.0-348.20.1.el8_5.x86_64   docker://20.10.14

As you see, INTERNAL-IP is <none>. Run rke remove, then comment out the cloud_provider section in cluster.yml, so it becomes:

cloud_provider:
  name: ""

Then:

$ rke up
...
$ KUBECONFIG=kube_config_cluster.yml kubectl get nodes -o wide
NAME    STATUS   ROLES                      AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                           KERNEL-VERSION                 CONTAINER-RUNTIME
rke-0   Ready    controlplane,etcd,worker   31s   v1.22.4   10.90.0.118   <none>        Rocky Linux 8.5 (Green Obsidian)   4.18.0-348.20.1.el8_5.x86_64   docker://20.10.14

Now the INTERNAL-IP is correctly set.
@github-actions
Copy link
Contributor

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented May 29, 2022

The issue is still relevant.

@github-actions
Copy link
Contributor

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Jul 29, 2022

The issue is still relevant.

@github-actions
Copy link
Contributor

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Sep 28, 2022

The issue is still relevant.

@github-actions
Copy link
Contributor

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Nov 29, 2022

The issue is still relevant.

@github-actions
Copy link
Contributor

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Jan 30, 2023

The issue is still relevant.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 1, 2023

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Apr 1, 2023

The issue is still relevant.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 1, 2023

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Jun 1, 2023

The issue is still relevant.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2023

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Oct 3, 2023

Ths issue is still relevant.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 3, 2023

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Dec 3, 2023

The issue is still relevant.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 3, 2024

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Feb 3, 2024

The issue is still relevant.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 5, 2024

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Apr 5, 2024

The issue is still relevant.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 5, 2024

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Jun 5, 2024

The issue is still relevant.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 5, 2024

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Aug 5, 2024

The issue is still relevant.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 5, 2024

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Oct 5, 2024

The issue is still relevant.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 6, 2024

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@waldner
Copy link
Author

waldner commented Dec 6, 2024

The issue is still relevant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@waldner