Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

集群apiserver证书支持直接配置IP和域名 #3002

Merged
merged 2 commits into from
Mar 5, 2024
Merged

集群apiserver证书支持直接配置IP和域名 #3002

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a6e9ad8
集群apiserver证书支持直接配置IP和域名
denglouping Mar 1, 2024
d2a2868
集群apiserver证书支持直接配置IP和域名
denglouping Mar 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions bcs-ops/functions/k8s.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,60 @@ k8s::safe_add_helmrepo() {
return 0
}

#######################################
# add vip to K8S apiserver certs
# Arguments:
# $1: op_type
# $2: excute (if op_type is $1, then $2 is path)
# $3: vips
# $4: path
# Return:
# excute success - return 0
# excute - return 1
#######################################
k8s::config_master_vip() {
op_type=$1
excute=$2
vips=${3:-""}
path=${4:-$2}

if [[ -z "${path}" ]];then
path=$(kubeadm_config_file="/tmp/kubeadm-$(date +%Y-%m-%d).yaml")
kubectl -n kube-system get configmap kubeadm-config -o jsonpath='{.data.ClusterConfiguration}' >"${path}"
fi

case "$op_type" in
add)
for vip in ${vips//,/ };do
if [[ -n "${vip}" ]];then
yq e -i '(select(.apiServer != null)|.apiServer.certSANs) += ["'${vip}'"]| select(.apiServer != null)|.apiServer.certSANs|= unique' ${path}
fi
done
;;
delete)
for vip in ${vips//,/ };do
if [[ -n "${vip}" ]];then
yq e 'del(.apiServer.certSANs[] | select(. == "'${vip}'"))' -i ${path}
fi
done
;;
list)
yq 'select(.apiServer.certSANs != null)|.apiServer.certSANs' ${path}
;;
*)
export ERR_CODE=1
utils::log "ERROR" "unkown command: $1"
;;
esac

if [[ "${excute}" == "true" ]] && [[ "${op_type}" != "list" ]];then
kubeadm init phase certs apiserver --config "${path}" \
|| utils::log "ERROR" "failed to ${op_type} ${vips} ${path}"
fi

utils::log "OK" "${op_type} ${vips} ${path}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

op_type 为 list 的时候，vips 没有值。或者赋值的时候使用 ${3:-""} 默认空值。

}

#######################################
# add vip to K8S apiserver certs
# Arguments:
Expand Down
4 changes: 3 additions & 1 deletion bcs-ops/k8s/render_kubeadm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,8 @@ IPv6EOF
)
---
EOF

k8s::config_master_vip add false ${EXTERNAL_VIP},${EXTERNAL_HOST} ${config_file}
}

render_kubelet() {
Expand Down Expand Up @@ -318,7 +320,7 @@ safe_source() {

main() {
local source_files ipv6_status cri_type
source_files=("${ROOT_DIR}/functions/utils.sh" "${ROOT_DIR}/env/bcs.env")
source_files=("${ROOT_DIR}/functions/utils.sh" "${ROOT_DIR}/env/bcs.env" "${ROOT_DIR}/functions/k8s.sh")
for file in "${source_files[@]}"; do
safe_source "$file"
done
Expand Down
37 changes: 19 additions & 18 deletions bcs-ops/readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ set +x

1. 通过`set -a 命令`配置环境变量，环境变量配置见[`环境变量`](#环境变量)
2. `./bcs-ops -r bcsenv` 在第一台主机（后称中控机）上渲染配置文件 `env/bcs.env`
3. 在中控机上启动集群控制平面：`./bcs-ops --instal master`，集群启动成功后会显示加入集群的指令
3. 在中控机上启动集群控制平面：`./bcs-ops --install master`，集群启动成功后会显示加入集群的指令
4. 集群加入指令有效期为 1 小时，中控机执行 `./bcs-ops --render joincmd` 可再次渲染生成加入集群的指令，渲染结果如下所示

```plaintext
Expand Down Expand Up @@ -196,23 +196,24 @@ set +x

#### apiserver ha 环境变量

| 环境变量 | 默认值 | 说明 |
| ------------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENABLE_APISERVER_HA` | `false` | apiserver ha 模式，默认关闭。 |
| `APISERVER_HA_MODE` | `bcs-apiserver-proxy` | 模式选择，支持 [bcs-apiserver-proxy](https://github.com/TencentBlueKing/bk-bcs/blob/625be3183d99ee3500123016a6dea99d78165565/docs/features/bcs-apiserver-proxy/bcs-apiserver-proxy.md#L1)`​[kube-vip](https://kube-vip.io/)` |
| `VIP` | | VIP 地址，可配置与集群内不冲突的 ip 地址 |
| `VS_PORT` | `6443` | bap 代理端口 |
| `APISERVER_PROXY_VERSION` | `v1.29.0-alpha.130-tencent` | bap 镜像版本 |
| `PROXY_TOOL_PATH` | `/usr/bin` | bap 工具安装目录 |
| `PERSIST_DIR` | `/root/.bcs` | bap 持久化目录 |
| `LVS_SCHEDULER` | `rr` | bap 负载均衡策略 |
| `MANAGER_INTERVAL` | `10` | bap 监听时间 |
| `DEBUG_MODE` | `true` | bap DEBUG 模式 默认开启 |
| `LOG_LEVEL` | `3` | bap 日志等级 |
| `KUBE_VIP_VERSION` | `v0.5.12` | kube-vip 镜像版本 |
| `BIND_INTERFACE` | `""` | kube-vip 绑定网卡名 |
| `VIP_CIDR` | `32` | VIP CIDR 掩码长度 |

| 环境变量 | 默认值 | 说明 |
| ------------------------- | --------------------------- |---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `ENABLE_APISERVER_HA` | `false` | apiserver ha 模式，默认关闭。 |
| `APISERVER_HA_MODE` | `bcs-apiserver-proxy` | 模式选择，支持 [bcs-apiserver-proxy](https://github.com/TencentBlueKing/bk-bcs/blob/625be3183d99ee3500123016a6dea99d78165565/docs/features/bcs-apiserver-proxy/bcs-apiserver-proxy.md#L1), [kube-vip](https://kube-vip.io/) |
| `VIP` | | VIP 地址，可配置与集群内不冲突的 ip 地址 |
| `VS_PORT` | `6443` | bap 代理端口 |
| `APISERVER_PROXY_VERSION` | `v1.29.0-alpha.130-tencent` | bap 镜像版本 |
| `PROXY_TOOL_PATH` | `/usr/bin` | bap 工具安装目录 |
| `PERSIST_DIR` | `/root/.bcs` | bap 持久化目录 |
| `LVS_SCHEDULER` | `rr` | bap 负载均衡策略 |
| `MANAGER_INTERVAL` | `10` | bap 监听时间 |
| `DEBUG_MODE` | `true` | bap DEBUG 模式 默认开启 |
| `LOG_LEVEL` | `3` | bap 日志等级 |
| `KUBE_VIP_VERSION` | `v0.5.12` | kube-vip 镜像版本 |
| `BIND_INTERFACE` | `""` | kube-vip 绑定网卡名 |
| `VIP_CIDR` | `32` | VIP CIDR 掩码长度 |
| `EXTERNAL_VIP` | `""` | 外部LB ip |
| `EXTERNAL_HOST` | `""` | 外部域名解析域名 |
### 示例

#### a. 选择 `1.24.15` 的 k8s 版本 ，并使用 `containerd` 作为容器运行时
Expand Down
6 changes: 6 additions & 0 deletions bcs-ops/system/config_envfile.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,9 @@ init_env() {
KUBE_VIP_VERSION=${KUBE_VIP_VERSION:-"v0.5.12"}
BIND_INTERFACE=${BIND_INTERFACE:-}
VIP_CIDR=${VIP_CIDR:-"32"}
## external-vip
EXTERNAL_VIP=${EXTERNAL_VIP:-}
EXTERNAL_HOST=${EXTERNAL_HOST:-}
## multus
ENABLE_MULTUS_HA=${ENABLE_MULTUS_HA:-"true"}
}
Expand Down Expand Up @@ -336,6 +339,9 @@ DEBUG_MODE="${DEBUG_MODE}"
KUBE_VIP_VERSION="${KUBE_VIP_VERSION}"
BIND_INTERFACE="${BIND_INTERFACE}"
VIP_CIDR="${VIP_CIDR}"
## external-vip
EXTERNAL_VIP="${EXTERNAL_VIP}"
EXTERNAL_HOST="${EXTERNAL_HOST}"
## multus
ENABLE_MULTUS_HA="${ENABLE_MULTUS_HA}"
# bcs config end
Expand Down
Loading