By Valleylord

更新日期:

文章目录
  1. 1. 镜像介绍
  2. 2. Kubernetes 集群搭建
  3. 3. Kubernetes 集群搭建验证
  4. 4. Pod 缩放和扩容
  5. 5. 应用的滚动升级

本文使用的《Kubernetes 权威指南》中的例子。

镜像介绍

本文搭建的系统是一个留言板系统，使用的 docker 镜像是《Kubernetes 权威指南》中提供的，托管在 docker.io 上。如下，

1
2
3
4
5
6
[root@Centos-L410]~arnes/kubernetes# docker search kubeguide
INDEX       NAME                                         DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
docker.io   docker.io/kubeguide/redis-master             redis-master with "Hello World!"                3                    
docker.io   docker.io/kubeguide/centos7-ansible          ansible with sshpass tool installed and ss...   2                    
docker.io   docker.io/kubeguide/guestbook-php-frontend   Guestbook PHP website                           2                    
docker.io   docker.io/kubeguide/guestbook-redis-slave    Guestbook redis slave                           2

除了 centos7-ansible 镜像，其他3个镜像会在本例中用到。该留言板系统使用 php 搭建，使用 redis 作为存储，redis 使用一主多备。

Kubernetes 集群搭建

先定义 redis-master 的 Pod 和 Service，redis-master 的 RC redis-master-rc.yaml 如下，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: v1
kind: ReplicationController
metadata:
  name: redis-master
  labels:
    name: redis-master
spec:
  replicas: 1
  selector:
    name: redis-master
    version: v1
  template:
    metadata:
      labels:
        name: redis-master
        version: v1
    spec:
      containers:
      - name: master
        image: kubeguide/redis-master:1.0
        ports:
        - containerPort: 6379

redis-master 的 Service redis-master-service.yaml 如下，

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: v1
kind: Service
metadata:
  name: redis-master
  labels:
    name: redis-master
spec:
  ports:
  - port: 6379
    targetPort: 6379
  selector:
    name: redis-master

再定义 redis-slave 的 Pod 和 Service，redis-slave 的 RC redis-slave-rc.yaml 如下，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
apiVersion: v1
kind: ReplicationController
metadata:
  name: redis-slave
  labels:
    name: redis-slave
spec:
  replicas: 2
  selector:
    name: redis-slave
  template:
    metadata:
      labels:
        name: redis-slave
    spec:
      containers:
      - name: slave
        image: kubeguide/guestbook-redis-slave
        env:
        - name: GET_HOSTS_FROM
          value: env
        ports:
        - containerPort: 6379

redis-slave 的 Service redis-slave-service.yaml 如下，

1
2
3
4
5
6
7
8
9
10
11
apiVersion: v1
kind: Service
metadata:
  name: redis-slave
  labels:
    name: redis-slave
spec:
  ports:
  - port: 6379
  selector:
    name: redis-slave

最后定义 php 的 Pod 和 Service，php 的 RC frontend-rc.yaml 如下，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
piVersion: v1
kind: ReplicationController
metadata:
  name: frontend
  labels:
    name: frontend
spec:
  replicas: 3
  selector:
    name: frontend
  template:
    metadata:
      labels:
        name: frontend
    spec:
      containers:
      - name: frontend
        image: kubeguide/guestbook-php-frontend
        env:
        - name: GET_HOSTS_FROM
          value: env
        ports:
        - containerPort: 80

php 的 Service frontend-service.yaml 如下，

1
2
3
4
5
6
7
8
9
10
11
12
13
apiVersion: v1
kind: Service
metadata:
  name: frontend
  labels:
    name: frontend
spec:
  type: NodePort
  ports:
  - port: 80
    nodePort: 30001
  selector:
    name: frontend

创建这些 Service 和 Pod 只需要依次运行以下命令即可，

1
2
3
4
5
6
kubectl create -f redis-master-rc.yaml
kubectl create -f redis-master-service.yaml
kubectl create -f redis-slave-rc.yaml
kubectl create -f redis-slave-service.yaml
kubectl create -f frontend-rc.yaml
kubectl create -f frontend-service.yaml

注意，redis-slave 和 redis-master 镜像的不同主要体现在启动命令的不同，redis-slave 使用 redis-server --slaveof ${REDIS_MASTER_SERVICE_HOST} 6379来启动。

Kubernetes 集群搭建验证

如果搭建正常，可以看到类似以下的输出，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl get pods                  
NAME                 READY     STATUS    RESTARTS   AGE
frontend-2qzcj       1/1       Running   0          1m
frontend-f0cb6       1/1       Running   0          1m
frontend-ubedy       1/1       Running   0          1m
redis-master-28uug   1/1       Running   0          13m
redis-slave-3etxv    1/1       Running   0          2m
redis-slave-spf75    1/1       Running   0          2m
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl get services                      
NAME           CLUSTER_IP      EXTERNAL_IP   PORT(S)    SELECTOR            AGE
frontend       10.254.49.190   nodes         80/TCP     name=frontend       35s
kubernetes     10.254.0.1      <none>        443/TCP    <none>              7d
redis-master   10.254.35.174   <none>        6379/TCP   name=redis-master   12m
redis-slave    10.254.32.178   <none>        6379/TCP   name=redis-slave    2m
[root@Centos-L410]~arnes/project/kubernetes-roll# docker ps
CONTAINER ID        IMAGE                                                        COMMAND                  CREATED              STATUS              PORTS               NAMES
635ea3d0181f        kubeguide/guestbook-php-frontend                             "apache2-foreground"     About a minute ago   Up About a minute                       k8s_frontend.34ee268e_frontend-2qzcj_default_9ce06947-e384-11e5-a10f-c80aa9c034dc_ec302c6b
20a3b593e158        kubeguide/guestbook-php-frontend                             "apache2-foreground"     About a minute ago   Up About a minute                       k8s_frontend.34ee268e_frontend-ubedy_default_9ce06115-e384-11e5-a10f-c80aa9c034dc_caf3ba3b
789a1d419315        kubeguide/guestbook-php-frontend                             "apache2-foreground"     About a minute ago   Up About a minute                       k8s_frontend.34ee268e_frontend-f0cb6_default_9cdffcd7-e384-11e5-a10f-c80aa9c034dc_53183bf4
627531a72664        registry.access.redhat.com/rhel7/pod-infrastructure:latest   "/pod"                   About a minute ago   Up About a minute                       k8s_POD.c36b0a77_frontend-ubedy_default_9ce06115-e384-11e5-a10f-c80aa9c034dc_11c886e6
217e25b56c9e        registry.access.redhat.com/rhel7/pod-infrastructure:latest   "/pod"                   About a minute ago   Up About a minute                       k8s_POD.c36b0a77_frontend-2qzcj_default_9ce06947-e384-11e5-a10f-c80aa9c034dc_89b18d65
70f93bcc4da7        registry.access.redhat.com/rhel7/pod-infrastructure:latest   "/pod"                   About a minute ago   Up About a minute                       k8s_POD.c36b0a77_frontend-f0cb6_default_9cdffcd7-e384-11e5-a10f-c80aa9c034dc_f3710ab7
672cdc93fc3f        kubeguide/guestbook-redis-slave                              "/entrypoint.sh /bin/"   2 minutes ago        Up 2 minutes                            k8s_slave.6a232544_redis-slave-3etxv_default_6a74487e-e384-11e5-a10f-c80aa9c034dc_69ad2ac7
ccc447a1c80b        kubeguide/guestbook-redis-slave                              "/entrypoint.sh /bin/"   2 minutes ago        Up 2 minutes                            k8s_slave.6a232544_redis-slave-spf75_default_6a7456a7-e384-11e5-a10f-c80aa9c034dc_a937374a
02e4cc97f365        registry.access.redhat.com/rhel7/pod-infrastructure:latest   "/pod"                   2 minutes ago        Up 2 minutes                            k8s_POD.4f810ae8_redis-slave-3etxv_default_6a74487e-e384-11e5-a10f-c80aa9c034dc_d8416c26
72463f2a2d15        registry.access.redhat.com/rhel7/pod-infrastructure:latest   "/pod"                   2 minutes ago        Up 2 minutes                            k8s_POD.4f810ae8_redis-slave-spf75_default_6a7456a7-e384-11e5-a10f-c80aa9c034dc_d9d627a5
e3f31eaa0164        kubeguide/redis-master:1.0                                   "redis-server /etc/re"   13 minutes ago       Up 13 minutes                           k8s_master.b7750720_redis-master-28uug_default_eb715755-e382-11e5-a10f-c80aa9c034dc_b4d36e11
eaabc381aa3a        registry.access.redhat.com/rhel7/pod-infrastructure:latest   "/pod"                   13 minutes ago       Up 13 minutes                           k8s_POD.4f810ae8_redis-master-28uug_default_eb715755-e382-11e5-a10f-c80aa9c034dc_16f151f1

可以使用 curl 命令来查看是否已经搭建成功，如下，

1
2
3
4
5
6
7
8
9
10
11
$ curl 192.168.2.202:30001 
<html ng-app="redis">
  <head>
    <title>Guestbook</title>
    <link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css">
    <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.12/angular.min.js"></script>
    <script src="controllers.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/angular-ui-bootstrap/0.13.0/ui-bootstrap-tpls.js"></script>
  </head>
  <body ng-controller="RedisCtrl">
......

也可以用浏览器登陆 http://192.168.2.202:30001/，可以看到如下网页，

Pod 缩放和扩容

可以使用 kubectl scale 命令来控制副本数量，以实现 Pod 的缩放和扩容。例如，要将 redis-slave 的副本数由最初的2更新为1，以节约系统资源，可以如下操作，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl scale rc redis-slave --replicas=1
replicationcontroller "redis-slave" scaled
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl get pods                         
NAME                 READY     STATUS        RESTARTS   AGE
frontend-2qzcj       1/1       Running       0          17m
frontend-f0cb6       1/1       Running       0          17m
frontend-ubedy       1/1       Running       0          17m
redis-master-28uug   1/1       Running       0          29m
redis-slave-3etxv    1/1       Running       0          18m
redis-slave-spf75    1/1       Terminating   0          18m
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl get pods
NAME                 READY     STATUS    RESTARTS   AGE
frontend-2qzcj       1/1       Running   0          17m
frontend-f0cb6       1/1       Running   0          17m
frontend-ubedy       1/1       Running   0          17m
redis-master-28uug   1/1       Running   0          29m
redis-slave-3etxv    1/1       Running   0          19m

可以看到，过了一段时间之后，redis-slave 的副本数已经降为1。如果要将 redis-slave 扩容以适应更大的业务需求，例如，将副本数更新为3，可以如下操作，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl scale rc redis-slave --replicas=3
replicationcontroller "redis-slave" scaled
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl get pods                         
NAME                 READY     STATUS    RESTARTS   AGE
frontend-2qzcj       1/1       Running   0          20m
frontend-f0cb6       1/1       Running   0          20m
frontend-ubedy       1/1       Running   0          20m
redis-master-28uug   1/1       Running   0          32m
redis-slave-3etxv    1/1       Running   0          22m
redis-slave-6ib0f    0/1       Pending   0          5s
redis-slave-bimjq    0/1       Pending   0          5s
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl get pods
NAME                 READY     STATUS    RESTARTS   AGE
frontend-2qzcj       1/1       Running   0          21m
frontend-f0cb6       1/1       Running   0          21m
frontend-ubedy       1/1       Running   0          21m
redis-master-28uug   1/1       Running   0          33m
redis-slave-3etxv    1/1       Running   0          22m
redis-slave-6ib0f    1/1       Running   0          24s
redis-slave-bimjq    1/1       Running   0          24s

应用的滚动升级

如果系统中的一个服务需要升级，就需要停止该服务的所有 Pod，并用新的镜像创建容器并启动。如果该集群比较大，这个工作量就非常巨大，而且会可能引起服务中断。Kubernetes 提供了滚动升级来解决上述问题。滚动升级可以逐步替换其中的 Pod，并在升级的过程中用新的 Pod 逐步对外提供服务。

假设上述的 redis-master 发生了更新，新的镜像来自 redis-master:latest，新的 RC 文件 redis-master-rc-v2.yaml 如下，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
apiVersion: v1
kind: ReplicationController
metadata:
  name: redis-master-v2
  labels:
    name: redis-master
    version: v2
spec:
  replicas: 1
  selector:
    name: redis-master
    version: v2
  template:
    metadata:
      labels:
        name: redis-master
        version: v2
    spec:
      containers:
      - name: master
        image: kubeguide/redis-master:latest
        ports:
        - containerPort: 6379

然后，即可开始滚动升级，使用如下命令，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl rolling-update redis-master -f redis-master-rc-v2.yaml
Created redis-master-v2
Scaling up redis-master-v2 from 0 to 1, scaling down redis-master from 1 to 0 (keep 1 pods available, don't exceed 2 pods)
Scaling redis-master-v2 up to 1
Scaling redis-master down to 0
Update succeeded. Deleting redis-master
replicationcontroller "redis-master" rolling updated to "redis-master-v2"
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl get pods                                              
NAME                    READY     STATUS    RESTARTS   AGE
frontend-2qzcj          1/1       Running   0          1h
frontend-f0cb6          1/1       Running   0          1h
frontend-ubedy          1/1       Running   0          1h
redis-master-v2-o3pfb   1/1       Running   0          1m
redis-slave-2gupi       1/1       Running   0          1m
redis-slave-cfj8j       1/1       Running   0          1m

运行完成之后，可以发现，Pod已经完成替换。实际的替换过程是，先建立一个新的 Pod，然后再删除原先的 Pod，并用新的 Pod 开始提供服务。在运行的过程中发现一个error，如下，

1
2
3
[root@Centos-L410]~arnes/project/kubernetes-roll# kubectl rolling-update redis-master -f redis-master-rc-v2.yaml 
error: redis-master-rc-v2.yaml must specify a matching key with non-equal value in Selector for redis-master
See 'kubectl rolling-update -h' for help and examples.

这个问题从报错信息上看，是因为 redis-master-v2 中的 selector 中的元素与 redis-master 中一致造成的，而实际上，我最初的配置是不一样的，redis-master-v2 中配置的是 name: redis-master version: v2，两个；redis-master 中配置的是 name: redis-master，只有1个。 因此，不应该认为是一样的。细节上来看，有文章1提到了相关代码，如下，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
if oldName == newRc.Name {
	return cmdutil.UsageError(cmd, "%s cannot have the same name as the existing ReplicationController %s",
		filename, oldName)
}
updater := kubectl.NewRollingUpdater(newRc.Namespace, client)
// To successfully pull off a rolling update the new and old rc have to differ
// by at least one selector. Every new pod should have the selector and every
// old pod should not have the selector.
var hasLabel bool
for key, oldValue := range oldRc.Spec.Selector {
	if newValue, ok := newRc.Spec.Selector[key]; ok && newValue != oldValue {
		hasLabel = true
		break
	}
}
if !hasLabel {
	return cmdutil.UsageError(cmd, "%s must specify a matching key with non-equal value in Selector for %s",
		filename, oldName)
}

上述代码在 /pkg/kubectl/cmd/cmd.go 的281~301行，对于新的rc和旧的rc，有2项限制，一个是新旧名字需要不同，另一个是rc的selector中需要至少有一项的值不一样。但是，代码的逻辑只判断了旧的 rc 中的条目，并没有检查新的 rc 多出来的条目，感觉这应该是一个bug。

也可以使用更简单的命令来升级，只需要指定新的镜像即可，如下，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[root@Centos-L410]/home/arnes/project/kubernetes-roll# kubectl rolling-update redis-master --image=kubeguide/redis-master:latest
Created redis-master-48ec5825c021205af16519fb67fa2b1d
Scaling up redis-master-48ec5825c021205af16519fb67fa2b1d from 0 to 1, scaling down redis-master from 1 to 0 (keep 1 pods available, don't exceed 2 pods)
Scaling redis-master-48ec5825c021205af16519fb67fa2b1d up to 1
Scaling redis-master down to 0
Update succeeded. Deleting old controller: redis-master
Renaming redis-master-48ec5825c021205af16519fb67fa2b1d to redis-master
replicationcontroller "redis-master" rolling updated
[root@Centos-L410]/home/arnes/project/kubernetes-roll# kubectl get pods                                                         
NAME                                                  READY     STATUS    RESTARTS   AGE
frontend-2qzcj                                        1/1       Running   0          1h
frontend-f0cb6                                        1/1       Running   0          1h
frontend-ubedy                                        1/1       Running   0          1h
redis-master-48ec5825c021205af16519fb67fa2b1d-61kj1   1/1       Running   0          3m
redis-slave-2gupi                                     1/1       Running   0          37m
redis-slave-cfj8j                                     1/1       Running   0          37m
[root@Centos-L410]/home/arnes/project/kubernetes-roll# kubectl get replicationControllers
CONTROLLER     CONTAINER(S)   IMAGE(S)                           SELECTOR                                                                   REPLICAS   AGE
frontend       frontend       kubeguide/guestbook-php-frontend   name=frontend                                                              3          1h
redis-master   master         kubeguide/redis-master:latest      deployment=457221237209715cf35015690e05dfeb,name=redis-master,version=v1   1          3m
redis-slave    slave          kubeguide/guestbook-redis-slave    name=redis-slave                                                           2          38m

上述过程仅仅替换了镜像版本，并增加了 selector 中的一个随机条目 deployment，label、selector 等配置都没有改变。

  1. http://dockone.io/article/328↩

文章目录
  1. 1. 镜像介绍
  2. 2. Kubernetes 集群搭建
  3. 3. Kubernetes 集群搭建验证
  4. 4. Pod 缩放和扩容
  5. 5. 应用的滚动升级

欢迎来到Valleylord的博客！

本博的文章尽量原创。