Kubernetes-Controller控制器
控制器Controller
控制器的作用
1.pod类型的资源，删除pod后，不会重建
2.替用户监视并保证相应的节点上始终有用户所期望的副本数量的pod在运行
3.如果所运行的pod副本数超过了用户期望的，那么控制器就会删掉，直到和用户期望的一致
4.如果所运行的pod副本数低于用户期望的，那么控制器就会创建，直到和用户期望的一致控制器类型
RC: 标准控制器
RS（ReplicaSet）: 按用户期望的副本创建Pod，并始终保持相应数量的副本
Deployment:
1）控制RS来保证Pod始终保持相应的数量副本
2）滚动更新，回滚，回滚默认保留10个版本
3）声明式配置，支持动态修改
4）管理无状态应用最理想的控制器
5）de节点可能会运行0个或多个pod
DaemonSet: 一个节点只运行一个，必须是始终运行的状态（例：kube-proxy，zabbix-agent）
StatefullSet: 有状态的应用RS控制器
# rs控制器创建10个nginx的pod示例
[root@master-1 tmp]# vim 111.yml
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: nginx-rs
labels:
# pod标签
app: nginx-labels
spec:
# 副本数量
replicas: 10
# 标签选择器
selector:
matchLabels:
# pod标签
app: nginx-labels
# 包含pod设置
template:
metadata:
labels:
# pod标签
app: nginx-labels
spec:
containers:
- name: nginx-rs
image: nginx:alpine
imagePullPolicy: IfNotPresent
# 纯净版示例
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: nginx-rs
labels:
app: nginx-labels
spec:
replicas: 6
selector:
matchLabels:
app: nginx-labels
template:
metadata:
labels:
app: nginx-labels
spec:
containers:
- name: nginx-rs
image: nginx:alpine
imagePullPolicy: IfNotPresent
# 运行
[root@master-1 tmp]# kubectl apply -f 111.yml
replicaset.apps/nginx-rs created
# 查看nginx
[root@master-1 tmp]# kubectl get pod
NAME READY STATUS RESTARTS AGE
liveness 1/1 Running 0 16h
livenss-pod 0/1 Running 0 16h
nginx-565785f75c-vrgg7 1/1 Running 0 2d
nginx-init 1/1 Running 0 23h
nginx-poststart 1/1 Running 0 17h
nginx-rs-2zpx4 1/1 Running 0 14s
nginx-rs-4lfd5 1/1 Running 0 14s
nginx-rs-4zfjc 1/1 Running 0 14s
nginx-rs-65xjx 1/1 Running 0 14s
nginx-rs-9hfp2 1/1 Running 0 14s
nginx-rs-p2bd7 1/1 Running 0 14s
nginx-rs-rwj2f 1/1 Running 0 14s
nginx-rs-vcnws 1/1 Running 0 14s
nginx-rs-wrbkj 1/1 Running 0 14s
nginx-rs-wvkxd 1/1 Running 0 14s
nginx10-test 1/1 Running 0 2d
resource-demo 1/1 Running 0 61m
# 查看控制器
[root@master-1 tmp]# kubectl get replicasets
NAME DESIRED CURRENT READY AGE
nginx-565785f75c 1 1 1 2d
nginx-rs 10 10 10 3m33s
# 扩缩容（法一：资源清单修改）
[root@master-1 tmp]# vim 111.yml
## 修改为6个
replicas: 6
[root@master-1 tmp]# kubectl apply -f 111.yml
replicaset.apps/nginx-rs configured
[root@master-1 tmp]# kubectl get replicasets
NAME DESIRED CURRENT READY AGE
nginx-565785f75c 1 1 1 2d
nginx-rs 6 6 6 5m56s
# 扩缩容（法二：kube命令修改）
[root@master-1 tmp]# kubectl edit rs nginx-rs
## 扩容为12个
replicas: 12
[root@master-1 tmp]# kubectl get replicasets
NAME DESIRED CURRENT READY AGE
nginx-565785f75c 1 1 1 2d
nginx-rs 12 12 12 8m31s
# 简单方法的扩缩容（rs）
[root@master-1 tmp]# kubectl scale rs nginx-rs --replicas=1
replicaset.apps/nginx-rs scaled
[root@master-1 tmp]# kubectl get replicasets
NAME DESIRED CURRENT READY AGE
nginx-565785f75c 1 1 1 2d
nginx-rs 1 1 1 24mDeployment控制器（运用的比较多，实践工作中看需求）
例：nginx mysql
# Deployment控制器创建10个nginx的pod示例
[root@master-1 tmp]# vim 111.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-db
labels:
app: nginx-labels
spec:
# modify replicas according to your case
replicas: 6
selector:
matchLabels:
app: nginx-labels
template:
metadata:
labels:
app: nginx-labels
spec:
containers:
- name: nginx-db
image: nginx:alpine
imagePullPolicy: IfNotPresent
# 运行
[root@master-1 tmp]# kubectl apply -f 111.yml
deployment.apps/nginx-rs created
# 查看rs控制器
[root@master-1 tmp]# kubectl get rs
NAME DESIRED CURRENT READY AGE
nginx-dp-7649948664 6 6 6 52s
# 查看deployments控制器
[root@master-1 tmp]# kubectl get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-dp 6/6 6 6 66s
# 查看pod
[root@master-1 tmp]# kubectl get pod
NAME READY STATUS RESTARTS AGE
liveness 1/1 Running 0 16h
livenss-pod 0/1 Running 0 17h
nginx-dp-7649948664-dq8kd 1/1 Running 0 7s
nginx-dp-7649948664-fjk56 1/1 Running 0 7s
nginx-dp-7649948664-gxtlj 1/1 Running 0 7s
nginx-dp-7649948664-hcc7v 1/1 Running 0 7s
nginx-dp-7649948664-hlst7 1/1 Running 0 7s
nginx-dp-7649948664-tbdqk 1/1 Running 0 7s
nginx-init 1/1 Running 0 23h
nginx-poststart 1/1 Running 0 18h
nginx10-test 1/1 Running 0 2d
resource-demo 1/1 Running 0 91m
## 结论
通过控制RS来保证Pod始终保持相应的数量副本
# 滚动更新示例
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-dp
labels:
app: nginx-labels
spec:
# modify replicas according to your case
replicas: 6
selector:
matchLabels:
app: nginx-labels
template:
metadata:
labels:
app: nginx-labels
spec:
containers:
- name: nginx-dp
image: nginx:1.18
imagePullPolicy: IfNotPresent
# 查看之前的deployments控制器
[root@master-1 tmp]# kubectl get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-dp 6/6 6 6 10m
# 运行
[root@master-1 tmp]# kubectl apply -f 111.yml
deployment.apps/nginx-dp configured
# 动态查看deployments控制器
[root@master-1 tmp]# kubectl get deployments.apps -w
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-dp 5/6 6 5 10m
nginx-dp 6/6 6 6 10m
nginx-dp 6/6 6 6 10m
nginx-dp 5/6 6 5 10m
nginx-dp 6/6 6 6 11m
nginx-dp 6/6 6 6 11m
nginx-dp 5/6 6 5 11m
nginx-dp 6/6 6 6 11m
## 总结
滚动更新，替换版本时是启动好一台再把旧之前版本剔除掉版本更新
# 版本更新
kubectl set image -f 111.yml nginx-dp=nginx:1.18.0
# 查看滚动更新状态
kubectl rollout status deployment nginx-dp
Waiting for deployment "nginx-dp" rollout to finish: 3 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 3 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 3 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 3 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 4 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 4 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 4 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 4 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 4 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 5 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 5 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 5 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 5 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 5 out of 6 new replicas have been updated...
Waiting for deployment "nginx-dp" rollout to finish: 3 old replicas are pending termination...
Waiting for deployment "nginx-dp" rollout to finish: 2 old replicas are pending termination...
Waiting for deployment "nginx-dp" rollout to finish: 2 old replicas are pending termination...
Waiting for deployment "nginx-dp" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "nginx-dp" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "nginx-dp" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "nginx-dp" rollout to finish: 5 of 6 updated replicas are available...
deployment "nginx-dp" successfully rolled out
# 版本回滚到上一个版本
[root@master-1 tmp]# kubectl rollout undo deployment nginx-dp
deployment.apps/nginx-dp rolled back
# 回滚指定版本
## 如果想查看指定版本详细信息（版本记录）
需要在创建命令后面加上 --record
例：kubectl set image -f 111.yml nginx-dp=nginx:1.18.0 --record
## 查看历史版本（默认只保留10个版本）
[root@master-1 tmp]# kubectl rollout history deployment nginx-dp
deployment.apps/nginx-dp
REVISION CHANGE-CAUSE
1 <none>
3 <none>
4 <none>
## 查看指定历史版本的详细信息
[root@master-1 tmp]# kubectl rollout history deployment nginx-dp --revision=1
deployment.apps/nginx-dp with revision #1
Pod Template:
Labels: app=nginx-labels
pod-template-hash=7649948664
Containers:
nginx-dp:
Image: nginx:alpine
Port: <none>
Host Port: <none>
Environment: <none>
Mounts: <none>
Volumes: <none>
## 回滚到指定版本
[root@master-1 tmp]# kubectl rollout undo deployment nginx-dp --to-revision=1
deployment.apps/nginx-dp rolled back
### 查看历史版本（版本1消失了，因为已经回滚了）
[root@master-1 tmp]# kubectl rollout history deployment nginx-dp
deployment.apps/nginx-dp
REVISION CHANGE-CAUSE
3 <none>
4 <none>
5 <none>扩缩容
kubectl scale Deployment nginx-dp --replicas=1 DaemonSet控制器
# DaemonSet控制器创建nginx的pod示例
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: nginx-ds
labels:
app: nginx-labels
spec:
selector:
matchLabels:
app: nginx-labels
template:
metadata:
labels:
app: nginx-labels
spec:
containers:
- name: nginx-ds
image: nginx:alpine
imagePullPolicy: IfNotPresent
# 运行
[root@master-1 tmp]# kubectl apply -f 111.yml
daemonset.apps/nginx-ds created
# 可以观察到两个node节点各起了一个nginx
[root@master-1 tmp]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ds-7zdz5 1/1 Running 0 28s 10.2.1.45 node-1 <none> <none>
nginx-ds-fl64s 1/1 Running 0 28s 10.2.2.61 node-2 <none> <none>
# 查看DaemonSet控制器
[root@master-1 tmp]# kubectl get ds
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
nginx-ds 2 2 2 2 2 <none> 105sHPA
官网地址：TP
工作原理
HAP通过收集来的监控指标分析所有Pod的负载情况，并且根据我们设定好的标准来自动扩容收缩ReplicationController、 Deployment、ReplicaSet 或 StatefulSet 中的 Pod 数量
## HPA通过以下命令，获取到各个节点上POD负载情况
[root@master-1 opt]# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
master-1 63m 6% 2214Mi 57%
node-1 61m 6% 903Mi 48%
node-2 77m 7% 1001Mi 53%
获取到POD负载情况，指标，可以根据我们自定义数量，进行扩缩容
## HPA -- MetricsServer -- Deployment
kubectl scale deployment nginx-dp --replicas=5
kubectl scale deployment nginx-dp --replicas=2部署Metrics-server
# 1.下载资源清单
[root@master-1 opt]# wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.4.0/components.yaml
# 2.导入镜像
[root@master-1 opt]# for i in node-1 node-2;do scp metrics-server.tar $i:/root;done
[root@node-1 ~]# docker load < metrics-server.tar
[root@node-2 ~]# docker load < metrics-server.tar
# 3.修改资源清单
[root@master-1 opt]# vim components.yaml
containers:
- args:
- --cert-dir=/tmp
- --secure-port=4443
- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
- --kubelet-use-node-status-port
- --kubelet-insecure-tls // 跳过握手
image: metrics-server:v0.4.0 // 使用本地镜像
# 4.应用资源清单
[root@master-1 opt]# kubectl apply -f components.yaml
# 5.查看节点负载
[root@master-1 opt]# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
master-1 63m 6% 2214Mi 57%
node-1 61m 6% 903Mi 48%
node-2 77m 7% 1001Mi 53% 生成测试镜像
创建测试首页
cat > index.php << 'EOF'
<?php
$x = 0.0001;
for ($i = 0; $i <= 1000000; $i++) {
$x += sqrt($x);
}
echo "OK!";
?>
EOF创建dockerfile
cat > dockerfile << 'EOF'
FROM php:5-apache
ADD index.php /var/www/html/index.php
RUN chmod a+rx index.php
EOF生成镜像
docker build -t php:v1 .保存镜像
[root@master-1 php]# docker save php:v1 > /tmp/php.tgz
[root@master-1 tmp]# for i in node-1 node-2;do scp /tmp/php.tgz $i:/root/;done
[root@node-1 ~]# docker load < php.tgz
[root@node-2 ~]# docker load < php.tgz 创建Deployment资源
[root@master-1 php]# vim php-dp.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: php-apache
spec:
replicas: 1
selector:
matchLabels:
run: php-apache
template:
metadata:
labels:
run: php-apache
spec:
containers:
- image: php:v1
imagePullPolicy: IfNotPresent
name: php-apache
resources:
requests:
cpu: 200m
[root@master-1 php]# kubectl apply -f php-dp.yml
deployment.apps/php-apache created创建HPA资源
[root@master-1 php]# vim php-hpa.yml
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: php-apache
spec:
minReplicas: 1
maxReplicas: 10
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: php-apache
targetCPUUtilizationPercentage: 50压测
# 查看HPA扩容情况
kubectl get hpa -w
kubectl get pod -w
kubectl top node
kubectl top pod
# 压测
while true; do wget -q -O- http://10.2.1.47; done
# 动态观察hpa（压测时与压测结束限制过程）
[root@master-1 php]# kubectl get hpa -w
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 0%/50% 1 10 1 6m38s
php-apache Deployment/php-apache 464%/50% 1 10 1 7m21s
php-apache Deployment/php-apache 464%/50% 1 10 4 7m36s
php-apache Deployment/php-apache 464%/50% 1 10 8 7m51s
php-apache Deployment/php-apache 464%/50% 1 10 10 8m6s
php-apache Deployment/php-apache 133%/50% 1 10 10 8m21s
php-apache Deployment/php-apache 0%/50% 1 10 10 9m21s
php-apache Deployment/php-apache 0%/50% 1 10 10 14m
php-apache Deployment/php-apache 0%/50% 1 10 1 14m简单命令
# 创建dp
kubectl run php-apache --image=php:v1 --requests=cpu=200m --expose --port=80
# 创建hpa
kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10wordpress自动扩容示例
## mysql
apiVersion: "v1"
kind: "Namespace"
metadata:
name: wordpress
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
namespace: wordpress
labels:
app: mysql
spec:
replicas: 1
selector:
matchLabels:
app: mysql
strategy: {}
template:
metadata:
labels:
app: mysql
spec:
containers:
- image: mysql:5.7
name: mysql
imagePullPolicy: IfNotPresent
ports:
- containerPort: 3306
name: dbport
env: #环境编排，进入数据库中创建Wordpress数据库和WordPress用户
- name: MYSQL_ROOT_PASSWORD
value: PassWord
- name: MYSQL_DATABASE
value: wordpress
- name: MYSQL_USER
value: wordpress
- name: MYSQL_PASSWORD
value: wordpress
volumeMounts:
- name: db
mountPath: /var/lib/mysql #mysql数据存放的容器目录
resources: {}
volumes:
- name: db
hostPath:
path: /var/lib/mysql # 容器目录中的数据映射到本地目录中
---
apiVersion: v1
kind: Service
metadata:
name: mysql
namespace: wordpress
spec:
selector:
app: mysql
ports:
- name: mysqlport
protocol: TCP
port: 3306
targetPort: dbport
## wordpress
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress
namespace: wordpress
labels:
app: wordpress
spec:
replicas: 1
selector:
matchLabels:
app: wordpress
strategy: {}
template:
metadata:
labels:
app: wordpress
spec:
containers:
- name: wordpress
image: wordpress
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: wdport
env:
- name: WORDPRESS_DB_HOST
value: 10.2.2.24:3306 #连接数据库，这里的ip地址查看mysql启动时的ip
- name: WORDPRESS_DB_USER
value: wordpress
- name: WORDPRESS_DB_PASSWORD
value: wordpress
resources:
requests:
cpu: 500m
---
apiVersion: v1
kind: Service
metadata:
name: wordpress
namespace: wordpress
spec:
type: NodePort
selector:
app: wordpress
ports:
- name: wordpressport
protocol: TCP
port: 80
targetPort: wdport
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: wordpress-auto
namespace: wordpress
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: wordpress
minReplicas: 1
maxReplicas: 3
targetCPUUtilizationPercentage: 50
压测
# 查看暴露端口
[root@master-1 tmp]# kubectl get svc -n wordpress
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mysql ClusterIP 10.1.147.212 <none> 3306/TCP 2d
wordpress NodePort 10.1.235.4 <none> 80:30886/TCP 47h
# 命令下载
[root@master-1 ~]# yum install -y install siege
# 测试
[root@master-1 ~]# siege -c10 -t0 http://10.0.0.111:30886
## 解析
-c10:虚拟10个用户执行
-t0:是不手动停止就一直执行压测监测结果
# pod展示
[root@master-1 tmp]# kubectl get pod -n wordpress -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mysql-67457648cd-flctn 1/1 Running 0 2d 10.2.2.24 node-2 <none> <none>
wordpress-544f8567bf-4z7td 1/1 Running 0 23s 10.2.2.71 node-2 <none> <none>
wordpress-544f8567bf-qt9f7 1/1 Running 0 23s 10.2.1.50 node-1 <none> <none>
[root@master-1 tmp]# kubectl get pod -n wordpress -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mysql-67457648cd-flctn 1/1 Running 0 2d 10.2.2.24 node-2 <none> <none>
wordpress-544f8567bf-4z7td 1/1 Running 0 2m17s 10.2.2.71 node-2 <none> <none>
wordpress-544f8567bf-qt9f7 1/1 Running 0 2m17s 10.2.1.50 node-1 <none> <none>
wordpress-544f8567bf-wjlb9 0/1 Pending 0 43s <none> <none> <none> <none>
# hpa展示wordpress数量由2个增加到了3个
[root@master-1 php]# kubectl get hpa -n wordpress -w
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
wordpress-auto Deployment/wordpress 147%/50% 1 3 2 94m
wordpress-auto Deployment/wordpress 147%/50% 1 3 2 94m
wordpress-auto Deployment/wordpress 130%/50% 1 3 2 95m
wordpress-auto Deployment/wordpress 130%/50% 1 3 3 96m
wordpress-auto Deployment/wordpress 134%/50% 1 3 3 96m
# 查看详细信息发现CPU没空间了，所以第三起不来
Warning FailedScheduling 2m17s default-scheduler 0/3 nodes are available: 3 Insufficient cpu.
Warning FailedScheduling 2m17s default-scheduler 0/3 nodes are available: 3 Insufficient cpu.