##前言
k8s提供多重部署，例如最复杂的二进制部署、yaml部署等，这个笔记记录的是kubeadm部署k8s集群，以防忘记
简介
kubeadm是Kubernetes官方提供的用于快速安装Kubernetes集群的工具，伴随Kubernetes每个版本的发布都会同步更新，kubeadm会对集群配置方面的一些实践做调整，通过实验kubeadm可以学习到Kubernetes官方在集群配置上一些新的最佳实践
准备
系统配置
再部署之前，需要两台linux服务器 修改两台机器的hosts文件
vim /etc/hosts
10.199.0.187 master
10.122.104.49 node1
需要将节点名修改为与其对应的名字
创建/etc/sysctl.d/k8s.conf文件
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
执行命令，生效
modprobe br_netfilter
sysctl -p /etc/sysctl.d/k8s.conf
关闭swap, Kubernetes 1.8开始要求关闭系统的Swap，如果不关闭，默认配置下kubelet将无法启动
swapoff -a
安装container runtime(docker)
Kubernetes从1.6开始使用CRI(Container Runtime Interface)容器运行时接口。默认的容器运行时仍然是Docker，使用的是kubelet中内置dockershim CRI实现。
安装docker
apt install docker docker.io
可能存在墙的问题，可以选择用阿里的源
查看是否安装成功
docker -v
使用kubeadm部署Kubernetes
安装kubelet kubeadm kubectl
apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl
查看是否安装成功
kubelet --version
kubeadm version
默认自启动kubelet服务(两个节点)
systemctl enable kubelet.service
使用kubeadm config print init-defaults可以打印集群初始化默认的使用的配置：
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 1.2.3.4
bindPort: 6443
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: templateubuntu16
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
serviceSubnet: 10.96.0.0/12
scheduler: {}
imageRepository定制在集群初始化时拉取k8s所需镜像的地址。基于默认配置定制出本次使用kubeadm初始化集群所需的配置文件kubeadm.yaml：
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 10.199.0.187
bindPort: 6443
nodeRegistration:
taints:
- effect: PreferNoSchedule
key: node-role.kubernetes.io/master
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.16.0
networking:
podSubnet: 10.244.0.0/16
再初始化集群之前，可以使用 kubeadm config images pull拉取部署k8s所需镜像（所有节点）
kubeadm config images pull
然后初始化master节点
kubeadm init --config kubeadm.yaml --ignore-preflight-errors=Swap
............
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 10.199.0.187:6443 --token 4qcl2f.gtl3h8e5kjltuo0r \
--discovery-token-ca-cert-hash sha256:7ed5404175cc0bf18dbfe53f19d4a35b1e3d40c19b10924275868ebf2a3bbe6e
[kubelet-start] 生成kubelet的配置文件”/var/lib/kubelet/config.yaml” [certs]生成相关的各种证书 [kubeconfig]生成相关的kubeconfig文件 [control-plane]使用/etc/kubernetes/manifests目录中的yaml文件创建apiserver、controller-manager、scheduler的静态pod [bootstraptoken]生成token记录下来，后边使用kubeadm join往集群中添加节点时会用到 先常规用户如何使用kubectl访问集群：
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
子节点如何加入集群
kubeadm join 10.199.0.187:6443 --token 4qcl2f.gtl3h8e5kjltuo0r \
--discovery-token-ca-cert-hash sha256:7ed5404175cc0bf18dbfe53f19d4a35b1e3d40c19b10924275868ebf2a3bbe6e
查看一下集群状态，确认个组件都处于healthy状态：
kubectl get cs
NAME AGE
scheduler <unknown>
controller-manager <unknown>
etcd-0 <unknown>
暂时不知道为什么unknown，但是没有影响，后续调研
现在通过kubectl get nodes 发现一个master.node, 但是状态是NotReady，是因为没有网络插件，所以需要安装网络插件
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
以上方法有可能安装不成功，也可以直接下载镜像
docker pull quay.io/coreos/flannel:v0.11.0-ppc64le
查看网络是否加入成功,并确保所有Pod处于running状态
kubectl get pods -n kube-system
coredns-5644d7b6d9-jhx5r 1/1 Running 0 45h
coredns-5644d7b6d9-p7rkj 1/1 Running 0 45h
etcd-templateubuntu16 1/1 Running 0 45h
kube-apiserver-templateubuntu16 1/1 Running 0 45h
kube-controller-manager-templateubuntu16 1/1 Running 1 45h
kube-flannel-ds-amd64-7l5sn 1/1 Running 0 44h
kube-flannel-ds-amd64-ps4n4 1/1 Running 0 44h
kube-proxy-b9vfw 1/1 Running 0 44h
kube-proxy-hh264 1/1 Running 0 45h
kube-scheduler-templateubuntu16 1/1 Running 0 45h
像集群中添加node节点
使用上面部署master时候产生的命令
kubeadm join 10.199.0.187:6443 --token 4qcl2f.gtl3h8e5kjltuo0r \
--discovery-token-ca-cert-hash sha256:7ed5404175cc0bf18dbfe53f19d4a35b1e3d40c19b10924275868ebf2a3bbe6e
查看子节点是否加入成功，去master节点上查看一下
kubectl get nodes
NAME STATUS ROLES AGE VERSION
node1 Ready <none> 44h v1.16.3
templateubuntu16 Ready master 45h v1.16.3
从集群中删除node节点
再master节点上
kubectl drain node1 --delete-local-data --force --ignore-daemonsets
kubectl delete node node1
在node1节点上
kubeadm reset
ifconfig cni0 down
ip link delete cni0
ifconfig flannel.1 down
ip link delete flannel.1
rm -rf /var/lib/cni/
部署一个简单的nginx
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port 80 --type=NodePort
查看服务
kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 46h
nginx NodePort 10.110.230.144 <none> 80:30042/TCP 9s
现在通过子节点，访问30042端口，就可以访问nginx
