使用的主机信息如下:
角色 | HOSTNAME | IP | CPU | 内存 | 系统盘 | CPU 架构 | 操作系统 |
---|---|---|---|---|---|---|---|
控制平面 | k8s-master | 192.168.0.101 | 2 | 4G | 64G | x86_64 | openSUSE Leap Micro 6.1 |
工作平面 | k8s-worker-1 | 192.168.0.102 | 2 | 4G | 64G | x86_64 | openSUSE Leap Micro 6.1 |
工作平面 | k8s-worker-2 | 192.168.0.103 | 4 | 8G | 64G | x86_64 | openSUSE Leap Micro 6.1 |
transactional-update dup
更新后重启生效:
reboot
wget
下载二进制文件时需要,若使用其他方式获取文件,则无需安装)transactional-update pkg install -y wget
安装后重启生效:
reboot
hostnamectl set-hostname k8s-master
hostnamectl set-hostname k8s-worker-1
hostnamectl set-hostname k8s-worker-2
vim /etc/hosts
192.168.0.101 k8s-master
192.168.0.102 k8s-worker-1
192.168.0.103 k8s-worker-2
swapoff -a
vim /etc/fstab
注释掉包含 swap 的行
cat /proc/swaps
返回为空时表示已禁用
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
getenforce
返回为 Permissive
表示关闭
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
modprobe overlay
modprobe br_netfilter
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system
mkdir -p /opt/containerd/{bin,conf,run,data}
其中:
bin
: 二进制文件目录conf
: 配置文件目录run
: 运行时的 sock 及相关文件目录data
: 数据目录cd /opt/containerd/bin
wget https://github.com/opencontainers/runc/releases/download/v1.2.6/runc.amd64
mv runc.amd64 runc
chmod +x runc
示例主机 CPU 架构为
x86_64
,因此使用amd64
格式文件,若 CPU 为其他架构,则需使用与之对应的文件。
cd /opt/containerd/bin
wget https://github.com/containernetworking/plugins/releases/download/v1.6.2/cni-plugins-linux-amd64-v1.6.2.tgz
tar zxvf cni-plugins-linux-amd64-v1.6.2.tgz -C /opt/containerd/bin --strip-components 1
rm -rf LICENSE README.md
示例主机 CPU 架构为
x86_64
,因此使用amd64
格式文件,若 CPU 为其他架构,则需使用与之对应的文件。
mkdir -pv /opt/containerd/net.d
vim /opt/containerd/net.d/10-containerd-net.conflist
{
"cniVersion": "1.0.0",
"name": "containerd-net",
"plugins": [
{
"type": "bridge",
"bridge": "cni0",
"isGateway": true,
"ipMasq": true,
"promiscMode": true,
"ipam": {
"type": "host-local",
"ranges": [
[{
"subnet": "10.10.0.0/16",
"gateway": "10.10.0.1"
}]
],
"routes": [
{ "dst": "0.0.0.0/0" }
]
}
},
{
"type": "portmap",
"capabilities": {"portMappings": true}
},
{
"type": "firewall"
}
]
}
其中:
plugins -> ipam -> ranges -> subnet
为容器使用的网段,可按需设置cd /opt/containerd/bin
wget https://github.com/containerd/containerd/releases/download/v2.0.4/containerd-2.0.4-linux-amd64.tar.gz
tar zxvf containerd-2.0.4-linux-amd64.tar.gz -C /opt/containerd/bin --strip-components 1
示例主机 CPU 架构为
x86_64
,因此使用amd64
格式文件,若 CPU 为其他架构,则需使用与之对应的文件。
vim /opt/containerd/conf/config.toml
version = 2
root = "/opt/containerd/data"
state = "/opt/containerd/run"
oom_score = 0
imports = ["/opt/containerd/conf/runtime_*.toml"]
[grpc]
address = "/opt/containerd/run/containerd.sock"
uid = 0
gid = 0
[debug]
address = "/opt/containerd/run/debug.sock"
uid = 0
gid = 0
level = "info"
[metrics]
address = ""
grpc_histogram = false
[cgroup]
path = ""
[plugins]
[plugins."io.containerd.monitor.v1.cgroups"]
no_prometheus = false
[plugins."io.containerd.service.v1.diff-service"]
default = ["walking"]
[plugins."io.containerd.gc.v1.scheduler"]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = 0
startup_delay = "100ms"
[plugins."io.containerd.runtime.v2.task"]
platforms = ["linux/amd64"]
sched_core = true
[plugins."io.containerd.service.v1.tasks-service"]
blockio_config_file = ""
rdt_config_file = ""
[plugins."io.containerd.grpc.v1.cri"]
disable = false
sandbox_image = "registry.k8s.io/pause:3.10"
stats_collect_period = 10
enable_selinux = false
enable_unprivileged_ports = false
enable_unprivileged_icmp = false
stream_server_address = "127.0.0.1"
stream_server_port = "0"
max_container_log_line_size = 16384
[plugins."io.containerd.grpc.v1.cri".containerd]
snapshotter = "overlayfs"
default_runtime_name = "runc"
discard_unpacked_layers = false
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true
BinaryName = ""
CriuImagePath = ""
CriuWorkPath = ""
[plugins."io.containerd.grpc.v1.cri".cni]
bin_dir = "/opt/containerd/bin"
conf_dir = "/opt/containerd/net.d"
max_conf_num = 1
其中:
root
: 数据目录,可按需设置state
: 运行时 sock 及相关文件的目录,可按需设置[grpc] -> address
: containerd.sock 文件路径[debug] -> address
: debug.sock 文件路径[grpc] 和 [debug] -> uid
: sock 文件所属用户 id[grpc] 和 [debug] -> gid
: sock 文件所属用户组 id[plugins] -> [plugins."io.containerd.grpc.v1.cri".cni] -> bin_dir
为 containernetworking-plugins 可执行文件目录[plugins] -> [plugins."io.containerd.grpc.v1.cri".cni] -> conf_dir
为 containernetworking-plugins 配置文件目录[plugins] -> [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] -> SystemdCgroup
部分的值需要为 true
vim /etc/systemd/system/containerd.service
# Copyright The containerd Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target dbus.service
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/opt/containerd/bin/containerd --config=/opt/containerd/conf/config.toml
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
其中:
/opt/containerd/bin
为二进制文件目录/opt/containerd/conf/config.toml
为配置文件systemctl daemon-reload
vim ~/.bashrc
# containerd Env Begin
export CONTAINERD_HOME=/opt/containerd/bin
export PATH=$PATH:$CONTAINERD_HOME
# containerd Env End
source ~/.bashrc
其中:
/opt/containerd/bin
为二进制文件目录vim /etc/profile
# containerd Env Begin
export CONTAINERD_HOME=/opt/containerd/bin
export PATH=$PATH:$CONTAINERD_HOME
# containerd Env End
source /etc/profile
其中:
/opt/containerd/bin
为二进制文件目录systemctl enable --now containerd
systemctl status containerd
注意返回的 Active
是否为 running
,同时查看日志中是否存在报错(error
)信息,启动正常时的示例如下:
● containerd.service - containerd Application Container Engine
Loaded: loaded (/etc/systemd/system/containerd.service; enabled; preset: disabled)
Active: active (running) since Wed 2025-04-09 16:33:02 CST; 2s ago
TriggeredBy: ● containerd.socket
Docs: https://containerd.io/docs/
Main PID: 1430 (containerd)
Tasks: 8
CPU: 40ms
CGroup: /system.slice/containerd.service
└─1430 /opt/containerd/bin/containerd --config /opt/containerd/conf/config.toml unix://var/run/containerd.sock
Apr 09 16:33:02 k8s-master containerd[1430]: time="2025-04-09T16:33:02.326504840+08:00" level=info msg="using experimental NRI integration - disable nri plugin to prevent this"
Apr 09 16:33:02 k8s-master containerd[1430]: time="2025-04-09T16:33:02.335955557+08:00" level=info msg="Start subscribing containerd event"
Apr 09 16:33:02 k8s-master containerd[1430]: time="2025-04-09T16:33:02.336057245+08:00" level=info msg=serving... address=/run/containerd/containerd.sock.ttrpc
Apr 09 16:33:02 k8s-master containerd[1430]: time="2025-04-09T16:33:02.336087662+08:00" level=info msg="Start recovering state"
Apr 09 16:33:02 k8s-master containerd[1430]: time="2025-04-09T16:33:02.336180815+08:00" level=info msg="Start event monitor"
Apr 09 16:33:02 k8s-master containerd[1430]: time="2025-04-09T16:33:02.336209588+08:00" level=info msg="Start cni network conf syncer for default"
Apr 09 16:33:02 k8s-master containerd[1430]: time="2025-04-09T16:33:02.336218124+08:00" level=info msg="Start streaming server"
Apr 09 16:33:02 k8s-master containerd[1430]: time="2025-04-09T16:33:02.336130211+08:00" level=info msg=serving... address=/run/containerd/containerd.sock
Apr 09 16:33:02 k8s-master systemd[1]: Started containerd Application Container Engine.
Apr 09 16:33:02 k8s-master containerd[1430]: time="2025-04-09T16:33:02.337768555+08:00" level=info msg="containerd successfully booted in 0.019553s"
containerd --version
返回值根据安装的版本有所不同,示例如下:
containerd github.com/containerd/containerd/v2 v2.0.4 1a43cb6a1035441f9aca8f5666a9b3ef9e70ab20
或:
ctr version
返回示例如下:
Client:
Version: v2.0.4
Revision: 1a43cb6a1035441f9aca8f5666a9b3ef9e70ab20
Go version: go1.23.7
Server:
Version: v2.0.4
Revision: 1a43cb6a1035441f9aca8f5666a9b3ef9e70ab20
UUID: a7774173-cab4-4c19-84a7-e7aec8730441
mkdir -pv /opt/kubernetes/{bin,conf,data,etcd,logs}
mkdir -pv /opt/kubernetes/conf/{manifests,pki}
其中:
bin
为二进制文件目录conf
为配置文件目录data
为数据目录etcd
为 etcd 数据存储目录logs
为日志目录conf/manifests
为静态 Pod 清单目录conf/pki
为证书和密钥目录cd /opt/kubernetes/bin
wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.32.0/crictl-v1.32.0-linux-amd64.tar.gz
tar zxvf crictl-v1.32.0-linux-amd64.tar.gz -C /opt/kubernetes/bin
示例主机 CPU 架构为
x86_64
,因此使用amd64
格式文件,若 CPU 为其他架构,则需使用与之对应的文件。
cd /opt/kubernetes/bin
wget https://dl.k8s.io/v1.32.3/bin/linux/amd64/kubelet
wget https://dl.k8s.io/v1.32.3/bin/linux/amd64/kubeadm
wget https://dl.k8s.io/v1.32.3/bin/linux/amd64/kubectl
示例主机 CPU 架构为
x86_64
,因此使用amd64
格式文件,若 CPU 为其他架构,则需使用与之对应的文件。
下载后配置可执行权限:
chmod +x /opt/kubernetes/bin/*
vim /etc/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
Documentation=https://kubernetes.io/docs/
After=containerd.service
Requires=containerd.service
[Service]
ExecStart=/opt/kubernetes/bin/kubelet --container-runtime-endpoint /opt/containerd/run/containerd.sock
Restart=always
StartLimitInterval=0
RestartSec=10
[Install]
WantedBy=multi-user.target
其中:
/opt/kubernetes/bin
为二进制文件目录systemctl daemon-reload
systemctl enable --now kubelet
vim ~/.bashrc
# Kubernetes Env Begin
export KUBERNETES_HOME=/opt/kubernetes/bin
export PATH=$PATH:$KUBERNETES_HOME
# Kubernetes Env End
source ~/.bashrc
其中:
/opt/kubernetes/bin
为二进制文件目录vim /etc/profile
# Kubernetes Env Begin
export KUBERNETES_HOME=/opt/kubernetes/bin
export PATH=$PATH:$KUBERNETES_HOME
# Kubernetes Env End
source /etc/profile
其中:
/opt/kubernetes/bin
为二进制文件目录部分地区无法访问
registry.k8s.io
镜像仓库,因此需要修改 containerd 的配置,指定可用的镜像仓库地址。(endpoint
部分即为镜像仓库地址,具体地址可自行查找)
vim /opt/containerd/conf/config.toml
[plugins."io.containerd.grpc.v1.cri"]
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.10"
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.k8s.io"]
endpoint = ["https://registry.aliyuncs.com/google_containers"]
也可以手动拉取镜像,之后在本地进行标记:
ctr image pull registry.k8s.io/kube-apiserver:v1.32.3
ctr image pull registry.k8s.io/kube-controller-manager:v1.32.3
ctr image pull registry.k8s.io/kube-scheduler:v1.32.3
ctr image pull registry.k8s.io/etcd:3.5.10-0
ctr image pull registry.k8s.io/kube-proxy:v1.32.3
ctr image pull registry.k8s.io/pause:3.10
ctr image pull registry.k8s.io/coredns/coredns:v1.11.1
ctr image pull registry.k8s.io/metrics-server/metrics-server:v0.7.0
ctr image pull registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.10.1
ctr image pull registry.k8s.io/autoscaling/cluster-autoscaler:v1.29.0
ctr image pull registry.k8s.io/dashboard:v2.7.0
cd /opt/kubernetes/conf
kubeadm config print init-defaults > kubeadm-custom.yaml
vim kubeadm-custom.yaml
apiVersion: kubeadm.k8s.io/v1beta4
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.0.101
bindPort: 6443
nodeRegistration:
criSocket: unix:///opt/containerd/run/containerd.sock
imagePullPolicy: IfNotPresent
imagePullSerial: true
name: node
taints: null
timeouts:
controlPlaneComponentHealthCheck: 4m0s
discovery: 5m0s
etcdAPICall: 2m0s
kubeletHealthCheck: 4m0s
kubernetesAPICall: 1m0s
tlsBootstrap: 5m0s
upgradeManifests: 5m0s
---
apiServer: {}
apiVersion: kubeadm.k8s.io/v1beta4
caCertificateValidityPeriod: 87600h0m0s
certificateValidityPeriod: 8760h0m0s
certificatesDir: /opt/kubernetes/conf/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
encryptionAlgorithm: RSA-2048
etcd:
local:
dataDir: /opt/kubernetes/etcd
imageRepository: registry.k8s.io
kind: ClusterConfiguration
kubernetesVersion: 1.32.0
networking:
dnsDomain: cluster.local
serviceSubnet: 10.20.0.0/12
proxy: {}
scheduler: {}
kubeadm init --config=kubeadm-custom.yaml
其中:
localAPIEndpoint -> advertiseAddress: 192.168.0.101
指定 API 服务公布的 IP 地址certificatesDir
为证书目录etcd -> local -> dataDir
为 etcd 数据报错目录imageRepository
为镜像仓库kubernetesVersion
为需要部署的 Kubernetes 版本networking -> serviceSubnet
为 Pod 网段kubeadm init --cri-socket /opt/containerd/run/containerd.sock --kubernetes-version=1.32.3 --pod-network-cidr=10.20.0.0/16 --service-cidr==10.30.0.0/16
其中:
--kubernetes-version=1.32.3
指定部署的 Kubernetes 版本--pod-network-cidr=10.20.0.0/16
指定 Pod 使用的网段--service-cidr==10.30.0.0/16
指定 Service 使用的网段其他可选参数:
--image-repository=[镜像仓库地址]
指定自定义镜像仓库地址,默认使用 registry.k8s.io
--apiserver-advertise-address=[IP]
指定 API 服务公布的 IP 地址,默认使用主机默认网卡 IP 地址--apiserver-bind-port=[端口号]
指定 API 服务绑定端口,默认为 6443
--control-plane-endpoint=[IP:端口号]
指定控制平面共享 IP:端口,用于高可用集群--service-dns-domain=[域名]
指定服务 DNS 域,默认为 cluster.local
--cert-dir=[目录]
指定证书存储目录,默认为 /etc/kubernetes/pki
--certificate-key=[密钥]
指定证书加密密钥,用于高可用集群控制平面之间传输证书--upload-certs
指定将证书至集群,用于高可用集群--dry-run
指定不实际执行,只打印日志kubeadm join 192.168.0.101:6443 --cri-socket /opt/containerd/run/containerd.sock --token=123456 --discovery-token-ca-cert-hash=sha256:123456
其中:
192.168.0.101:6443
为控制平面地址--token=123456
为 token,由控制平面自动生成--discovery-token-ca-cert-hash=sha256:123456
为证书 hash 值,由控制平面自动生成其他可选参数:
--node-name=[名称]
指定节点名称,默认为主机名--control-plane
指定此节点以控制平面身份加入--certificate-key=[密钥]
指定证书解密密钥,和 kubeadm init
的 --certificate-key
相同--apiserver-advertise-address=[IP:端口号]
指定控制平面共享 IP:端口,用于高可用集群--dry-run
指定不实际执行,只打印日志systemctl status containerd
systemctl status kubelet
kubectl get nodes
kubectl cluster-info
kubectl get pods -n kube-system
kubectl get componentstatuses
kubectl run busybox --image=busybox:1.28 -- sleep 3600
kubectl exec -it busybox -- nslookup kubernetes.default
kubectl get --raw='/healthz'
kubectl -n kube-system exec -it etcd-<控制面节点名称> -- etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
endpoint health
kubectl 默认使用 $HOME/.kube/config
配置文件,需要手动配置该文件后才能使用 kubectl 管理集群。
可直接复制 admin.conf
文件(默认路径为 /etc/kubernetes
,此处使用自定义目录 /opt/kubernetes/conf
)后配置权限:
cp /opt/kubernetes/conf/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
openSUSE Leap Micro
是一个基于事务性更新的不可变操作系统,因此需要使用 transactional-update
命令来进行相关操作transactional-update
命令操作的内容,需要重启生效containerd
作为容器运行时,如使用其他容器运行时,请参考相关文档 kubeadm reset
和 kubeadm reset -f
命令重置,并在删除 $HOME/.kube/config
后重新初始化