前文我们了解了k8s节点污点和pod的对节点污点容忍度相关话题,回顾请参考:https://www.cnblogs.com/qiuhom-1874/p/14255486.html;今天我们来聊一下扩展k8s相关话题;

k8s上创建资源对象的过程

我们知道在k8s上,资源是有类型的,不同类型的资源,其定义方式和使用的字段各不相同;用户创建一个资源,实际上就是把k8s抽象的资源做实例化,即把k8s抽象出来的资源,通过资源清单赋值,创建出来的对象就是我们实例化对应类型资源的结果;用户创建一个资源,首先会把请求发送给apiserver,通过apiserver的认证、授权、准入控制以后,对应创建资源的定义就存放在etcd中,控制器通过watch机制监视apiserver上的资源变动,通过对应资源变动事件触发对应类型资源的控制器将对应资源创建出来,并通过控制器内部的和解循环监控着对应资源状态是否和用户定义的期望状态一样,如果发现不一样,内部和解循环就会被触发,对应控制器会向apiserver发起创建资源的请求,将对应资源重建,让对应资源的状态始终满足用户期望的状态;从上述的过程来看,用户创建一个资源分两个步骤,第一步是将对应请求发送给apiserver,通过apiserver把对应资源定义的信息存放在etcd中;第二个步骤是对应资源类型的控制器通过apiserver从etcd中读取对应资源的定义,将其创建出来;对于etcd来说,它本是就一个kv数据库,可以存储任意类型的kv数据,但在在k8s上,apiserver将不同类型的资源定义抽象成不同的资源,使得用户创建对应资源,必须是满足对应类型资源定义的规范,然后将规范的定义存放在etcd中;简单讲apiserver就是把用户存入etcd中的数据做了一层抽象,使得用户不能够随意将任意数据存储到etcd中,存入etcd中的数据必须是满足对应apiserver接口定义的规范;这就好比我们在使用mysql数据库时,必须遵守对应库中的对应表的定义;

在k8s上创建自定义资源类型

在k8s上,资源的类型有很多,比如pod,service,PersistentVolume,PersistentVolumeClaim等等,这些都是一些基础的资源类型;我们要创建某种资源,直接使用对应的资源类型,实例化一个对象即可;假如我们要在k8s上创建一个集群,我们是不是可以直接使用某种类型的资源,实例化一个集群对象呢?理论上是可以的,但是前提是对应k8s上有对应类型的资源;有对应类型的资源,用户就可以把对应创建资源的定义存放在etcd中;除了有对应类型的资源,我们还需要有对应的控制器将对应资源创建出来;这样一来对于不同集群或应用来说,其组织方式和逻辑都不同,使用的资源类型和控制器也有所不同;用户要想实例化更高级的资源来,就必须自己手动实现定义其资源类型,将对应资源类型实例化为对象;除此之外必要时还需要自己实现对应资源的控制器;简单讲用户想要实现更高级的资源类型,就必须扩展现有k8s的资源类型和控制器;

在k8s上扩展资源类型的方式有三种,第一种是crd,crd是k8s内建的资源类型,该类型资源主要用来创建用户自定义资源类型的资源;即通过crd资源,可以将用户自定义资源类型转换为k8s上资源类型;第二种是自定义apiserver;这种方式要比第一种方式要复杂一点,需要用户手动开发程序实现对应功能的apiserver,让其用户创建自定义类型资源能够通过自定义apiserver实现;第三种方式就是修改现有k8sapiserver,让其支持对应用户自定义资源类型;

自定义控制器

自定义资源类型我们可以使用crd资源实现,也可以使用自定义apiserver或修改原有apiserver代码实现,但是只有资源类型是不能够让对应自定义类型资源实例化为一个自定义资源对象,只有自定义资源类型,用户创建对应资源类型的资源对象时,只能把对应资源类型的定义信息写入到etcd中,它不能真正的跑起来,要想真正的跑起来,我们还需要一个自定义控制器,专门负责监听对应的资源类型的资源变化,将对应资源实例化为对应k8s上的资源对象;当然不是所有的自定义类型的资源都需要自定义控制器,如果对应自定义类型资源调用了底层基础控制器来管控对应自定义资源,那么对应自定义类型资源就不需要使用自定义控制器;我们知道控制器是k8s上的一个重要组件,它的工作逻辑是注册监听在apiserver上对应类型的资源变动,如果对应资源状态不满足用户期望状态,它就会根据内部的和解循环来请求apiserver将对应类型资源的定义发送给它,然后根据资源定义来重建对应的资源,让其状态始终和用户期望的状态保持一致;自定义控制器也是同样的逻辑,使用自定义控制器的目的也是让对应自定义类型资源能够被自定义控制器监听,一旦对应资源发生变动,它能够将其在k8s上创建出来,并一直保持和用户期望的状态吻合;自定义控制器和自定义资源类型可以分开实现,也可以合并在一起实现,即自定义控制器程序能够自动创建crd资源,让其对应自定义类型资源能够被k8s识别并将其创建出来;具体是分开实现还是合并在一起实现,取决开发自定义控制器程序员;

crd资源定义帮助

[root@master01 ~]# kubectl explain crd
KIND: CustomResourceDefinition
VERSION: apiextensions.k8s.io/v1 DESCRIPTION:
CustomResourceDefinition represents a resource that should be exposed on
the API server. Its name MUST be in the format <.spec.name>.<.spec.group>. FIELDS:
apiVersion <string>
APIVersion defines the versioned schema of this representation of an
object. Servers should convert recognized schemas to the latest internal
value, and may reject unrecognized values. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources kind <string>
Kind is a string value representing the REST resource this object
represents. Servers may infer this from the endpoint the client submits
requests to. Cannot be updated. In CamelCase. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds metadata <Object> spec <Object> -required-
spec describes how the user wants the resources to appear status <Object>
status indicates the actual state of the CustomResourceDefinition [root@master01 ~]#

提示:crd资源是k8s上的标准资源之一,它的定义主要有apiVersion,kind,metadata,spec和status;其中kind类型为CustomResourceDefinition,apiVersion是apiextensions.k8s.io/v1;这两个是固定格式;spec字段是用来定义对应指定资源类型资源的相关属性;

crd.spec字段说明

[root@master01 ~]# kubectl explain crd.spec
KIND: CustomResourceDefinition
VERSION: apiextensions.k8s.io/v1 RESOURCE: spec <Object> DESCRIPTION:
spec describes how the user wants the resources to appear CustomResourceDefinitionSpec describes how a user wants their resource to
appear FIELDS:
conversion <Object>
conversion defines conversion settings for the CRD. group <string> -required-
group is the API group of the defined custom resource. The custom resources
are served under `/apis/<group>/...`. Must match the name of the
CustomResourceDefinition (in the form `<names.plural>.<group>`). names <Object> -required-
names specify the resource and kind names for the custom resource. preserveUnknownFields <boolean>
preserveUnknownFields indicates that object fields which are not specified
in the OpenAPI schema should be preserved when persisting to storage.
apiVersion, kind, metadata and known fields inside metadata are always
preserved. This field is deprecated in favor of setting
`x-preserve-unknown-fields` to true in
`spec.versions[*].schema.openAPIV3Schema`. See
https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#pruning-versus-preserving-unknown-fields
for details. scope <string> -required-
scope indicates whether the defined custom resource is cluster- or
namespace-scoped. Allowed values are `Cluster` and `Namespaced`. versions <[]Object> -required-
versions is the list of all API versions of the defined custom resource.
Version names are used to compute the order in which served versions are
listed in API discovery. If the version string is "kube-like", it will sort
above non "kube-like" version strings, which are ordered lexicographically.
"Kube-like" versions start with a "v", then are followed by a number (the
major version), then optionally the string "alpha" or "beta" and another
number (the minor version). These are sorted first by GA > beta > alpha
(where GA is a version with no suffix such as beta or alpha), and then by
comparing major version, then minor version. An example sorted list of
versions: v10, v2, v1, v11beta2, v10beta3, v3beta1, v12alpha1, v11alpha2,
foo1, foo10. [root@master01 ~]#

提示:crd.spec中group字段使用来描述对应自定义类型资源的群组名称,其值为字符串;names字段是用来描述自定义类型资源对应类型,名称等等,其值为一个对象;scope字段用来定义对应自定义资源是那个级别的资源;该字段的值只能为Cluster或Namespaced;versions字段是用来指定对应自定义资源的版本,以及对应类型资源的属性字段等信息,该字段为一个列表对象;

示例:定义一个自定义类型资源

[root@master01 ~]# cat crontab-crd.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
# 名字必需与下面的 spec 字段匹配,并且格式为 '<名称的复数形式>.<组名>'
name: crontabs.stable.example.com
spec:
# 组名称,用于 REST API: /apis/<组>/<版本>
group: stable.example.com
# 列举此 CustomResourceDefinition 所支持的版本
versions:
- name: v1
# 每个版本都可以通过 served 标志来独立启用或禁止
served: true
# 其中一个且只有一个版本必需被标记为存储版本
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
cronSpec:
type: string
image:
type: string
replicas:
type: integer
# 可以是 Namespaced 或 Cluster
scope: Namespaced
names:
# 名称的复数形式,用于 URL:/apis/<组>/<版本>/<名称的复数形式>
plural: crontabs
# 名称的单数形式,作为命令行使用时和显示时的别名
singular: crontab
# kind 通常是单数形式的驼峰编码(CamelCased)形式。你的资源清单会使用这一形式。
kind: CronTab
# shortNames 允许你在命令行使用较短的字符串来匹配资源
shortNames:
- ct
[root@master01 ~]#

应用资源清单前使用kubectl get crontab

[root@master01 ~]# kubectl get crontab
error: the server doesn't have a resource type "crontab"
[root@master01 ~]#

提示:在没有应用资源清单前使用kubectl get crontab,它会提示我们所没有类型为crontab的资源

应用资源清单

[root@master01 ~]# kubectl apply -f crontab-crd.yaml
customresourcedefinition.apiextensions.k8s.io/crontabs.stable.example.com created
[root@master01 ~]# kubectl get crontab
No resources found in default namespace.
[root@master01 ~]#

提示:应用资源清单后,再次使用kubectl get crontab就没有报错了,只是提示在default名称空间没有对应类型资源;

查看crd

[root@master01 ~]# kubectl get crd
NAME CREATED AT
bgpconfigurations.crd.projectcalico.org 2021-01-03T15:49:21Z
bgppeers.crd.projectcalico.org 2021-01-03T15:49:21Z
blockaffinities.crd.projectcalico.org 2021-01-03T15:49:21Z
clusterinformations.crd.projectcalico.org 2021-01-03T15:49:21Z
crontabs.stable.example.com 2021-01-12T12:39:00Z
felixconfigurations.crd.projectcalico.org 2021-01-03T15:49:21Z
globalnetworkpolicies.crd.projectcalico.org 2021-01-03T15:49:21Z
globalnetworksets.crd.projectcalico.org 2021-01-03T15:49:21Z
hostendpoints.crd.projectcalico.org 2021-01-03T15:49:21Z
ipamblocks.crd.projectcalico.org 2021-01-03T15:49:21Z
ipamconfigs.crd.projectcalico.org 2021-01-03T15:49:21Z
ipamhandles.crd.projectcalico.org 2021-01-03T15:49:21Z
ippools.crd.projectcalico.org 2021-01-03T15:49:21Z
kubecontrollersconfigurations.crd.projectcalico.org 2021-01-03T15:49:21Z
networkpolicies.crd.projectcalico.org 2021-01-03T15:49:21Z
networksets.crd.projectcalico.org 2021-01-03T15:49:22Z
[root@master01 ~]# kubectl get crd/crontabs.stable.example.com
NAME CREATED AT
crontabs.stable.example.com 2021-01-12T12:39:00Z
[root@master01 ~]#

查看详情

[root@master01 ~]# kubectl get crd/crontabs.stable.example.com
NAME CREATED AT
crontabs.stable.example.com 2021-01-12T12:39:00Z
[root@master01 ~]# kubectl describe crd/crontabs.stable.example.com
Name: crontabs.stable.example.com
Namespace:
Labels: <none>
Annotations: <none>
API Version: apiextensions.k8s.io/v1
Kind: CustomResourceDefinition
Metadata:
Creation Timestamp: 2021-01-12T12:39:00Z
Generation: 1
Managed Fields:
API Version: apiextensions.k8s.io/v1
Fields Type: FieldsV1
fieldsV1:
f:status:
f:acceptedNames:
f:kind:
f:listKind:
f:plural:
f:shortNames:
f:singular:
f:conditions:
Manager: kube-apiserver
Operation: Update
Time: 2021-01-12T12:39:00Z
API Version: apiextensions.k8s.io/v1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
f:conversion:
.:
f:strategy:
f:group:
f:names:
f:kind:
f:listKind:
f:plural:
f:shortNames:
f:singular:
f:scope:
f:versions:
f:status:
f:storedVersions:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2021-01-12T12:39:00Z
Resource Version: 805506
UID: b92a90f4-c953-4876-a496-030c9ba023fd
Spec:
Conversion:
Strategy: None
Group: stable.example.com
Names:
Kind: CronTab
List Kind: CronTabList
Plural: crontabs
Short Names:
ct
Singular: crontab
Scope: Namespaced
Versions:
Name: v1
Schema:
openAPIV3Schema:
Properties:
Spec:
Properties:
Cron Spec:
Type: string
Image:
Type: string
Replicas:
Type: integer
Type: object
Type: object
Served: true
Storage: true
Status:
Accepted Names:
Kind: CronTab
List Kind: CronTabList
Plural: crontabs
Short Names:
ct
Singular: crontab
Conditions:
Last Transition Time: 2021-01-12T12:39:00Z
Message: no conflicts found
Reason: NoConflicts
Status: True
Type: NamesAccepted
Last Transition Time: 2021-01-12T12:39:00Z
Message: the initial names have been accepted
Reason: InitialNamesAccepted
Status: True
Type: Established
Stored Versions:
v1
Events: <none>
[root@master01 ~]#

使用自定义资源类型crontab创建资源

[root@master01 ~]# cat my-crontab.yaml
apiVersion: "stable.example.com/v1"
kind: CronTab
metadata:
name: my-new-cron-object
spec:
cronSpec: "* * * * */5"
image: my-awesome-cron-image
[root@master01 ~]#

提示:上述资源清单表示创建一个类型为Crontab的资源,该资源的群组版本为stable.example.com/v1;

应用资源清单

[root@master01 ~]# kubectl apply -f my-crontab.yaml
crontab.stable.example.com/my-new-cron-object created
[root@master01 ~]# kubectl get ct
NAME AGE
my-new-cron-object 5s
[root@master01 ~]# kubectl describe ct/my-new-cron-object
Name: my-new-cron-object
Namespace: default
Labels: <none>
Annotations: <none>
API Version: stable.example.com/v1
Kind: CronTab
Metadata:
Creation Timestamp: 2021-01-12T12:45:29Z
Generation: 1
Managed Fields:
API Version: stable.example.com/v1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:cronSpec:
f:image:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2021-01-12T12:45:29Z
Resource Version: 806182
UID: 31a88a3d-fa99-42b8-80f6-3e4559efdc40
Spec:
Cron Spec: * * * * */5
Image: my-awesome-cron-image
Events: <none>
[root@master01 ~]#

提示:可以看到对应类型资源已经创建成功;以上示例只是单纯的crd的使用示例,没有任何实质的作用;

部署自定义控制器

示例:部署mongodb-aperator

1、克隆项目

[root@master01 ~]# git clone https://github.com/mongodb/mongodb-kubernetes-operator.git
Cloning into 'mongodb-kubernetes-operator'...
remote: Enumerating objects: 95, done.
remote: Counting objects: 100% (95/95), done.
remote: Compressing objects: 100% (74/74), done.
remote: Total 4506 (delta 30), reused 60 (delta 15), pack-reused 4411
Receiving objects: 100% (4506/4506), 18.04 MiB | 183.00 KiB/s, done.
Resolving deltas: 100% (2621/2621), done.
[root@master01 ~]#

2、创建名称空间mongodb,并进入到mongodb-kubernetes-operator目录应用crd资源,创建自定义资源类型

[root@master01 mongodb-kubernetes-operator]# kubectl create ns mongodb
namespace/mongodb created
[root@master01 mongodb-kubernetes-operator]# kubectl get ns
NAME STATUS AGE
default Active 35d
ingress-nginx Active 22d
kube-node-lease Active 35d
kube-public Active 35d
kube-system Active 35d
kubernetes-dashboard Active 11d
mongodb Active 4s
[root@master01 mongodb-kubernetes-operator]# ls
agent build deploy docs go.sum pkg release.json scripts testdata version
APACHE2 cmd dev_notes go.mod LICENSE.md README.md requirements.txt test tools.go
[root@master01 mongodb-kubernetes-operator]# kubectl apply -f deploy/crds/mongodb.com_mongodb_crd.yaml -n mongodb
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/mongodb.mongodb.com created
[root@master01 mongodb-kubernetes-operator]#

验证:查看mongodb类型资源是否已经创建成功?

[root@master01 mongodb-kubernetes-operator]# kubectl get crd
NAME CREATED AT
bgpconfigurations.crd.projectcalico.org 2021-01-03T15:49:21Z
bgppeers.crd.projectcalico.org 2021-01-03T15:49:21Z
blockaffinities.crd.projectcalico.org 2021-01-03T15:49:21Z
clusterinformations.crd.projectcalico.org 2021-01-03T15:49:21Z
crontabs.stable.example.com 2021-01-12T12:39:00Z
felixconfigurations.crd.projectcalico.org 2021-01-03T15:49:21Z
globalnetworkpolicies.crd.projectcalico.org 2021-01-03T15:49:21Z
globalnetworksets.crd.projectcalico.org 2021-01-03T15:49:21Z
hostendpoints.crd.projectcalico.org 2021-01-03T15:49:21Z
ipamblocks.crd.projectcalico.org 2021-01-03T15:49:21Z
ipamconfigs.crd.projectcalico.org 2021-01-03T15:49:21Z
ipamhandles.crd.projectcalico.org 2021-01-03T15:49:21Z
ippools.crd.projectcalico.org 2021-01-03T15:49:21Z
kubecontrollersconfigurations.crd.projectcalico.org 2021-01-03T15:49:21Z
mongodb.mongodb.com 2021-01-13T06:38:22Z
networkpolicies.crd.projectcalico.org 2021-01-03T15:49:21Z
networksets.crd.projectcalico.org 2021-01-03T15:49:22Z
[root@master01 mongodb-kubernetes-operator]# kubectl get crd/mongodb.mongodb.com
NAME CREATED AT
mongodb.mongodb.com 2021-01-13T06:38:22Z
[root@master01 mongodb-kubernetes-operator]#

3、安装operator

[root@master01 mongodb-kubernetes-operator]# kubectl apply -f deploy/operator/ -n mongodb
deployment.apps/mongodb-kubernetes-operator created
role.rbac.authorization.k8s.io/mongodb-kubernetes-operator created
rolebinding.rbac.authorization.k8s.io/mongodb-kubernetes-operator created
serviceaccount/mongodb-kubernetes-operator created
[root@master01 mongodb-kubernetes-operator]#

提示:mongodb-kubernetes-operator这个项目是将自定义控制器和自定义资源类型分开实现的;其operator只负责创建和监听对应资源类型的变化,在资源有变化时,实例化为对应资源对象,并保持对应资源对象状态吻合用户期望状态;上述四个清单中主要是创建了一个sa账户,并对对应的sa用户授权;

operator.yaml内容

[root@master01 mongodb-kubernetes-operator]# cat deploy/operator/operator.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mongodb-kubernetes-operator
spec:
replicas: 1
selector:
matchLabels:
name: mongodb-kubernetes-operator
template:
metadata:
labels:
name: mongodb-kubernetes-operator
spec:
serviceAccountName: mongodb-kubernetes-operator
containers:
- name: mongodb-kubernetes-operator
image: quay.io/mongodb/mongodb-kubernetes-operator:0.3.0
command:
- mongodb-kubernetes-operator
imagePullPolicy: Always
env:
- name: WATCH_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: OPERATOR_NAME
value: "mongodb-kubernetes-operator"
- name: AGENT_IMAGE # The MongoDB Agent the operator will deploy to manage MongoDB deployments
value: quay.io/mongodb/mongodb-agent:10.19.0.6562-1
- name: VERSION_UPGRADE_HOOK_IMAGE
value: quay.io/mongodb/mongodb-kubernetes-operator-version-upgrade-post-start-hook:1.0.2
- name: MONGODB_IMAGE
value: "library/mongo"
- name: MONGODB_REPO_URL
value: "registry.hub.docker.com"
[root@master01 mongodb-kubernetes-operator]#

提示:上述资源主要用deploy控制器运行对应自定义控制器为一个pod;

验证:查看operator是否正常运行

[root@master01 mongodb-kubernetes-operator]# kubectl get pods -n mongodb
NAME READY STATUS RESTARTS AGE
mongodb-kubernetes-operator-7d557bcc95-th8js 1/1 Running 0 26s
[root@master01 mongodb-kubernetes-operator]#

提示:能够看到operator正常运行,就表示operator已经安装成功;

验证:使用自定义资源类型创建一个mongodb 副本集集群

[root@master01 mongodb-kubernetes-operator]# cat deploy/crds/mongodb.com_v1_mongodb_cr.yaml
---
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
name: example-mongodb
spec:
members: 3
type: ReplicaSet
version: "4.2.6"
security:
authentication:
modes: ["SCRAM"]
users:
- name: my-user
db: admin
passwordSecretRef: # a reference to the secret that will be used to generate the user's password
name: my-user-password
roles:
- name: clusterAdmin
db: admin
- name: userAdminAnyDatabase
db: admin
scramCredentialsSecretName: my-scram # the user credentials will be generated from this secret
# once the credentials are generated, this secret is no longer required
---
apiVersion: v1
kind: Secret
metadata:
name: my-user-password
type: Opaque
stringData:
password: 58LObjiMpxcjP1sMDW
[root@master01 mongodb-kubernetes-operator]# kubectl apply -f deploy/crds/mongodb.com_v1_mongodb_cr.yaml
mongodb.mongodb.com/example-mongodb created
secret/my-user-password created
[root@master01 mongodb-kubernetes-operator]#

应用清单

[root@master01 mongodb-kubernetes-operator]# kubectl apply -f deploy/crds/mongodb.com_v1_mongodb_cr.yaml -n mongodb
mongodb.mongodb.com/example-mongodb created
secret/my-user-password created
[root@master01 mongodb-kubernetes-operator]# kubectl get pods -n mongodb
NAME READY STATUS RESTARTS AGE
example-mongodb-0 0/2 Pending 0 9s
mongodb-kubernetes-operator-7d557bcc95-th8js 1/1 Running 0 88s
[root@master01 mongodb-kubernetes-operator]#

提示:这里可以看到对应pod处于pending状态;

查看pod详细信息

[root@master01 mongodb-kubernetes-operator]# kubectl describe pod/example-mongodb-0 -n mongodb|grep -A 10 "Events"
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 66s (x2 over 66s) default-scheduler 0/5 nodes are available: 5 pod has unbound immediate PersistentVolumeClaims.
[root@master01 mongodb-kubernetes-operator]#

提示:这里提示没有可以用的pvc;

删除mongodb名称空间下pvc

[root@master01 mongodb-kubernetes-operator]# kubectl get pvc -n mongodb
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-volume-example-mongodb-0 Pending 92s
[root@master01 mongodb-kubernetes-operator]# kubectl delete pvc --all -n mongodb
persistentvolumeclaim "data-volume-example-mongodb-0" deleted
[root@master01 mongodb-kubernetes-operator]# kubectl get pvc -n mongodb
No resources found in mongodb namespace.
[root@master01 mongodb-kubernetes-operator]#

创建pv和pvc

[root@master01 mongodb-kubernetes-operator]# cd
[root@master01 ~]# cat pv-demo.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv-v1
labels:
app: example-mongodb-svc
spec:
capacity:
storage: 1Gi
volumeMode: Filesystem
accessModes: ["ReadWriteOnce","ReadWriteMany","ReadOnlyMany"]
persistentVolumeReclaimPolicy: Retain
mountOptions:
- hard
- nfsvers=4.1
nfs:
path: /data/v1
server: 192.168.0.99
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv-v2
labels:
app: example-mongodb-svc
spec:
capacity:
storage: 1Gi
volumeMode: Filesystem
accessModes: ["ReadWriteOnce","ReadWriteMany","ReadOnlyMany"]
persistentVolumeReclaimPolicy: Retain
mountOptions:
- hard
- nfsvers=4.1
nfs:
path: /data/v2
server: 192.168.0.99
--- apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv-v3
labels:
app: example-mongodb-svc
spec:
capacity:
storage: 1Gi
volumeMode: Filesystem
accessModes: ["ReadWriteOnce","ReadWriteMany","ReadOnlyMany"]
persistentVolumeReclaimPolicy: Retain
mountOptions:
- hard
- nfsvers=4.1
nfs:
path: /data/v3
server: 192.168.0.99
[root@master01 ~]#

应用清单创建pv

[root@master01 ~]# kubectl apply -f pv-demo.yaml
persistentvolume/nfs-pv-v1 created
persistentvolume/nfs-pv-v2 created
persistentvolume/nfs-pv-v3 created
[root@master01 ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
nfs-pv-v1 1Gi RWO,ROX,RWX Retain Available 3s
nfs-pv-v2 1Gi RWO,ROX,RWX Retain Available 3s
nfs-pv-v3 1Gi RWO,ROX,RWX Retain Available 3s
[root@master01 ~]#

创建pvc清单

[root@master01 ~]# cat pvc-demo.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data-volume-example-mongodb-0
namespace: mongodb
spec:
accessModes:
- ReadWriteMany
volumeMode: Filesystem
resources:
requests:
storage: 500Mi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data-volume-example-mongodb-1
namespace: mongodb
spec:
accessModes:
- ReadWriteMany
volumeMode: Filesystem
resources:
requests:
storage: 500Mi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data-volume-example-mongodb-2
namespace: mongodb
spec:
accessModes:
- ReadWriteMany
volumeMode: Filesystem
resources:
requests:
storage: 500Mi
[root@master01 ~]#

应用清单创建pvc

[root@master01 ~]# kubectl get pvc -n mongodb
No resources found in mongodb namespace.
[root@master01 ~]# kubectl apply -f pvc-demo.yaml
persistentvolumeclaim/data-volume-example-mongodb-0 created
persistentvolumeclaim/data-volume-example-mongodb-1 created
persistentvolumeclaim/data-volume-example-mongodb-2 created
[root@master01 ~]# kubectl get pvc -n mongodb
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-volume-example-mongodb-0 Bound nfs-pv-v1 1Gi RWO,ROX,RWX 6s
data-volume-example-mongodb-1 Bound nfs-pv-v2 1Gi RWO,ROX,RWX 6s
data-volume-example-mongodb-2 Bound nfs-pv-v3 1Gi RWO,ROX,RWX 6s
[root@master01 ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
nfs-pv-v1 1Gi RWO,ROX,RWX Retain Bound mongodb/data-volume-example-mongodb-0 102s
nfs-pv-v2 1Gi RWO,ROX,RWX Retain Bound mongodb/data-volume-example-mongodb-1 102s
nfs-pv-v3 1Gi RWO,ROX,RWX Retain Bound mongodb/data-volume-example-mongodb-2 102s
[root@master01 ~]#

提示:可以看到对应pvc和pv已经绑定好了;

验证:查看mongodb副本集集群是否跑起来了?

[root@master01 ~]# kubectl get pods -n mongodb
NAME READY STATUS RESTARTS AGE
example-mongodb-0 2/2 Running 0 6m19s
example-mongodb-1 0/2 PodInitializing 0 10s
mongodb-kubernetes-operator-7d557bcc95-th8js 1/1 Running 0 7m38s
[root@master01 ~]# kubectl get pods -n mongodb -w
NAME READY STATUS RESTARTS AGE
example-mongodb-0 2/2 Running 0 6m35s
example-mongodb-1 1/2 Running 0 26s
mongodb-kubernetes-operator-7d557bcc95-th8js 1/1 Running 0 7m54s
example-mongodb-1 2/2 Running 0 43s
example-mongodb-2 0/2 Pending 0 0s
example-mongodb-2 0/2 Pending 0 0s
example-mongodb-2 0/2 Init:0/1 0 0s
example-mongodb-2 0/2 Init:0/1 0 1s
example-mongodb-2 0/2 Terminating 0 4s
example-mongodb-2 0/2 Terminating 0 6s
example-mongodb-2 0/2 Terminating 0 20s
example-mongodb-2 0/2 Terminating 0 20s
example-mongodb-2 0/2 Pending 0 0s
example-mongodb-2 0/2 Pending 0 0s
example-mongodb-2 0/2 Init:0/1 0 0s
example-mongodb-2 0/2 Init:0/1 0 1s
example-mongodb-2 0/2 PodInitializing 0 7s
example-mongodb-2 1/2 Running 0 14s
example-mongodb-2 2/2 Running 0 36s
^C[root@master01 ~]# kubectl get pods -n mongodb
NAME READY STATUS RESTARTS AGE
example-mongodb-0 2/2 Running 0 8m
example-mongodb-1 2/2 Running 0 111s
example-mongodb-2 2/2 Running 0 48s
mongodb-kubernetes-operator-7d557bcc95-th8js 1/1 Running 0 9m19s
[root@master01 ~]#

提示:可以看到对应的pod已经正常跑起来了;

验证:使用mongo 连接mongodbpod,看看对应副本集集群是否工作正常?

[root@master01 ~]# kubectl get pods -n mongodb -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
example-mongodb-0 2/2 Running 0 9m12s 10.244.4.101 node04.k8s.org <none> <none>
example-mongodb-1 2/2 Running 0 3m3s 10.244.2.130 node02.k8s.org <none> <none>
example-mongodb-2 2/2 Running 0 2m 10.244.1.130 node01.k8s.org <none> <none>
mongodb-kubernetes-operator-7d557bcc95-th8js 1/1 Running 0 10m 10.244.3.116 node03.k8s.org <none> <none>
[root@master01 ~]# mongo 10.244.4.101
MongoDB shell version v4.4.3
connecting to: mongodb://10.244.4.101:27017/test?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("b9d16fe9-6a74-4638-96e6-70aaf3c83bfa") }
MongoDB server version: 4.2.6
WARNING: shell and server versions do not match
example-mongodb:PRIMARY> show dbs
example-mongodb:PRIMARY> db.auth('my-user','58LObjiMpxcjP1sMDW')
Error: Authentication failed.
0
example-mongodb:PRIMARY> use admin
switched to db admin
example-mongodb:PRIMARY> db.auth('my-user','58LObjiMpxcjP1sMDW')
1
example-mongodb:PRIMARY> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
example-mongodb:PRIMARY> db.isMaster()
{
"hosts" : [
"example-mongodb-0.example-mongodb-svc.mongodb.svc.cluster.local:27017",
"example-mongodb-1.example-mongodb-svc.mongodb.svc.cluster.local:27017",
"example-mongodb-2.example-mongodb-svc.mongodb.svc.cluster.local:27017"
],
"setName" : "example-mongodb",
"setVersion" : 1,
"ismaster" : true,
"secondary" : false,
"primary" : "example-mongodb-0.example-mongodb-svc.mongodb.svc.cluster.local:27017",
"me" : "example-mongodb-0.example-mongodb-svc.mongodb.svc.cluster.local:27017",
"electionId" : ObjectId("7fffffff0000000000000003"),
"lastWrite" : {
"opTime" : {
"ts" : Timestamp(1610520741, 1),
"t" : NumberLong(3)
},
"lastWriteDate" : ISODate("2021-01-13T06:52:21Z"),
"majorityOpTime" : {
"ts" : Timestamp(1610520741, 1),
"t" : NumberLong(3)
},
"majorityWriteDate" : ISODate("2021-01-13T06:52:21Z")
},
"maxBsonObjectSize" : 16777216,
"maxMessageSizeBytes" : 48000000,
"maxWriteBatchSize" : 100000,
"localTime" : ISODate("2021-01-13T06:52:27.873Z"),
"logicalSessionTimeoutMinutes" : 30,
"connectionId" : 153,
"minWireVersion" : 0,
"maxWireVersion" : 8,
"readOnly" : false,
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1610520741, 1),
"signature" : {
"hash" : BinData(0,"EcWzL7O9Ue9kmm6cQ4FumkcIP6g="),
"keyId" : NumberLong("6917119940596072451")
}
},
"operationTime" : Timestamp(1610520741, 1)
}
example-mongodb:PRIMARY>

提示:可以看到三个mongodb pod是副本集关系;其中example-mongodb-0是主节点,其他两个是从节点;

最后还是说一下,我在做上面的实验中,虽然mongodb operator已经正常工作,但是用mongo这个客户端工具连接到主节点没法写数据,提示没有权限;但对应的用户在对应库是有读写权限的;在admin库下创建用户能提示用户添加成功,但是过几秒查询用户信息,发现用户不存在,不知道什么原因,有知道的朋友麻烦告知下(博主邮箱:linux-1874@qq.com),博主将感激不尽。。

容器编排系统K8s之crd资源的更多相关文章

  1. 一文带你看透kubernetes 容器编排系统

    本文由云+社区发表 作者:turboxu Kubernetes作为容器编排生态圈中重要一员,是Google大规模容器管理系统borg的开源版本实现,吸收借鉴了google过去十年间在生产环境上所学到的 ...

  2. ASP.NET Core 借助 K8S 玩转容器编排

    Production-Grade Container Orchestration - Automated container deployment, scaling, and management. ...

  3. K8s容器编排

    K8s容器编排 Kubernetes(k8s)具有完备的集群管理能力: 包括多层次的安全防护和准入机制 多租户应用支撑能力 透明的服务注册和服务发现机制 内建智能负载均衡器 强大的故障发现和自我修复能 ...

  4. K8S - 容器编排工具Kubernetes简介

    1 - Kubernetes Kubernetes(简称K8s,用8代替8个字符"ubernete")是Google开源的一个容器编排引擎. 目前最为广泛且流行的容器编排调度系统, ...

  5. k8s的容器编排

    1.K8S是如何对容器编排? 在K8S集群中,容器并非最小的单位,K8S集群中最小的调度单位是Pod,容器则被封装在Pod之中.由此可知,一个容器或多个容器可以同属于在一个Pod之中. 2.Pod是怎 ...

  6. docker,容器,编排,和基于容器的系统设计模式

    目录 从容器说起 背景 docker实现原理 编排之争 基于容器的分布式系统设计之道 单节点协作模式 Sidecar pattern(边车模式) Ambassador pattern(外交官模式) A ...

  7. K8S 中的容器编排和应用编排

    众所周知,Kubernetes 是一个容器编排平台,它有非常丰富的原始的 API 来支持容器编排,但是对于用户来说更加关心的是一个应用的编排,包含多容器和服务的组合,管理它们之间的依赖关系,以及如何管 ...

  8. DOCKER 学习笔记9 Kubernetes (K8s) 生产级容器编排 上

    前言 在上一节的学习中.我们已经可以通过最基本的 Docker Swarm 创建集群,然后在集群里面加入我们需要运行的任务 以及任务的数量 这样我们就创建了一个服务. 当然,这样的方式在我们本地虚拟机 ...

  9. 如何提升集群资源利用率? 阿里容器调度系统Sigma 深入解析

    阿里妹导读:为了保证系统的在线交易服务顺利运转,最初几年,阿里都是在双11大促来临之前大量采购机器储备计算资源,导致了双11之后资源大量闲置点现象.是否能把计算任务与在线服务进行混合部署,在现有弹性资 ...

  10. 01 . 容器编排简介及Kubernetes核心概念

    Kubernetes简介 Kubernetes是谷歌严格保密十几年的秘密武器-Borg的一个开源版本,是Docker分布式系统解决方案.2014年由Google公司启动. Kubernetes提供了面 ...

随机推荐

  1. 读懂IL代码就这么简单(三)完结篇

    一 前言 写了两篇关于IL指令相关的文章,分别把值类型与引用类型在 堆与栈上的操作区别详细的写了一遍 这第三篇也是最后一篇,之所以到第三篇就结束了,是因为以我现在的层次,能理解到的都写完了,而且个人认 ...

  2. c++链表归并排序的迭代版本

    之前用js写了个归并排序非递归版,而这一次,c++封装链表的时候也遇到了一个归并排序的接口.邓老师实现了递归版本的归并排序,但是递归的调用函数栈的累积是很占内存空间的.于是乎,那试试在链表结构上实现以 ...

  3. JS深入了解闭包

    闭包(closure)是Javascript语言的一个难点,也是它的特色,很多高级应用都要依靠闭包实现.   一.变量的作用域 要理解闭包,首先必须理解Javascript特殊的变量作用域. 变量的作 ...

  4. 【JQGRID DOCUMENTATION】.学习笔记.3.Pager

    处理大量数据时,一次只想显示一小部分.这时就需要Navigation Bar.目前Pager不能用于TreeGrid. 定义 }); 定义高度grid,pager是grid的一部分,宽度等于gird的 ...

  5. T-SQL查询进阶—理解SQL Server中的锁

    在SQL Server中,每一个查询都会找到最短路径实现自己的目标.如果数据库只接受一个连接一次只执行一个查询.那么查询当然是要多快好省的完成工作.但对于大多数数据库来说是需要同时处理多个查询的.这些 ...

  6. 禁用win7自己主动配置ipv4地址

    现象 一台新电脑,连了网线,没有dhcp,须要手动配置Ip. 配置了一个Ip后,发现ping网关不通. ipconfig 发现有2 个IP:  自己主动配置 IPv4 地址  . . . . . . ...

  7. ArcMap合并之路 -- 该段路合并成一个完整的路

    #1: 用 Arctoolbox\Data Management Tools\Generalization\dissolve 工具 #2: Options:dissolve field 项选" ...

  8. asp.net MVC 框架中控制器里使用Newtonsoft.Json对前端传过来的字符串进行解析

    下面我用一个实例来和大家分享一下我的经验,asp.net MVC 框架中控制器里使用Newtonsoft.Json对前端传过来的字符串进行解析. using Newtonsoft.Json; usin ...

  9. ORACLE 归档日志打开关闭方法

    一 设置为归档方式 1 sql> archive log list;   #查看是不是归档方式 2 sql> alter system set log_archive_start=true ...

  10. C#一套简单的单例系统

    单例基类 public class CSingletonBase<TYPE> { public static TYPE Singleton { get { return m_singlet ...