首頁>技術>

kubernetes系列教程(十八)TKE中實現ingress服務暴露

寫在前面

上一篇文章中介紹了基於Nginx實現Ingress Controller的實現，介紹了Nginx Ingress Controller安裝、相關功能，TLS，高階特性等介紹，本章開始介紹基於騰訊雲TKE實現ingress服務暴露。

1. TKE ingress1.1 TKE ingress架構

TKE是Tencent Kubernetes Engine即騰訊雲基於kubernetes提供的公有云上容器雲服務，TKE提供了兩種暴露服務的方式：service和ingress。

內網CLB，四層負載均衡，提供VPC內訪問，通過node節點的NodePort轉發至service；外網CLB，四層負載均衡，提供公網訪問，需要node節點具有訪問公網的能力；ingress， 七層負載均衡，提供http://127.0.0.1/vhost/conf/img_echo.php?w=640&h=439&src=http和http://127.0.0.1/vhost/conf/img_echo.php?w=640&h=439&src=https接入，提供ingress控制器的功能，藉助NodePort轉發

TKE service和ingress

要使用TKE的ingress功能，需要了解一下相關的元件內容：

l7-lb-controller ingress客戶端，安裝在kube-system，用於解析ingress配置並更新CLB的規則CLB 七層負載均衡，提供ingress controller的功能，根據ingress規則建立http/https監聽器，配置轉發規則，以NodePort埠繫結後端RSService 用於ingress服務發現，通過NodePort方式接入CLB證書 用於提供https接入，配置在CLB負載均衡上，提供CA簽名證書，通過Secrets封裝給CLB使用

由於nginx ingress controller是直接以Pod的形式部署在kubernetes叢集中，藉助於service的服務發現可直接實現和pod通訊，而TKE中ingress controller未直接部署在k8s叢集中，網路的接入需藉助於service的NodePort實現接入，其資料流如下圖：

TKE ingress資料流走向

1.2 ingress虛擬主機

環境說明: 建立兩個Deployment並以NodePort方式暴露服務，www1.happylau.cn對應tke-app-1服務，同理www2.happylau.cn對應tke-app-2服務，如下演示操作過程：

1、建立兩個Deployments

[root@VM_10_2_centos ingress]# kubectl create deployment tke-app-1 --image=nginx:1.7.9[root@VM_10_2_centos ingress]# kubectl create deployment tke-app-2 --image=nginx:1.7.9

2、 將兩個Deployment以NodePort的方式暴露服務

[root@VM_10_2_centos ~]# kubectl expose deployment tke-app-1 --port=80 --type=NodePort[root@VM_10_2_centos ~]# kubectl expose deployment tke-app-2 --port=80 --type=NodePort檢視服務列表[root@VM_10_2_centos ~]# kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 172.16.255.1 <none> 443/TCP 83dtke-app-1 NodePort 172.16.255.91 <none> 80:30597/TCP 2stke-app-2 NodePort 172.16.255.236 <none> 80:31674/TCP 73s

3、定義ingress規則，定義兩個host將不同主機轉發至backend不同的service

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: tke-ingress-demo annotations: kubernetes.io/ingress.class: qcloudspec: rules: - host: www1.happylau.cn http: paths: - path: / backend: serviceName: tke-app-1 servicePort: 80 - host: www2.happylau.cn http: paths: - path: / backend: serviceName: tke-app-2 servicePort: 80

4、 應用ingress規則，並檢視ingress詳情，可以看到ingress建立了一個公網CLB例項

#應用ingress規則[root@VM_10_2_centos ingress]# kubectl apply -f tke-ingress-demo.yaml ingress.extensions/tke-ingress-demo created#檢視ingress列表[root@VM_10_2_centos ingress]# kubectl get ingresses NAME HOSTS ADDRESS PORTS AGEtke-ingress-demo www1.happylau.cn,www2.happylau.cn 140.143.84.xxx 80 67s#檢視 ingress詳情[root@VM_10_2_centos ingress]# kubectl describe ingresses tke-ingress-demo Name: tke-ingress-demoNamespace: defaultAddress: 140.143.84.xxxDefault backend: default-http-backend:80 (<none>)Rules: Host Path Backends ---- ---- -------- www1.happylau.cn / tke-app-1:80 (172.16.1.15:80) www2.happylau.cn / tke-app-2:80 (172.16.2.17:80)Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"qcloud"},"name":"tke-ingress-demo","namespace":"default"},"spec":{"rules":[{"host":"www1.happylau.cn","http":{"paths":[{"backend":{"serviceName":"tke-app-1","servicePort":80},"path":"/"}]}},{"host":"www2.happylau.cn","http":{"paths":[{"backend":{"serviceName":"tke-app-2","servicePort":80},"path":"/"}]}}]}} kubernetes.io/ingress.class: qcloud kubernetes.io/ingress.qcloud-loadbalance-id: lb-a0xwhcx3Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal EnsuringIngress 69s (x3 over 89s) loadbalancer-controller Ensuring ingress Normal CREATE 69s (x2 over 70s) loadbalancer-controller create loadbalancer succ Normal EnsuredIngress 68s (x3 over 70s) loadbalancer-controller Ensured ingress

5、測試驗證，將IP和域名寫入到hosts檔案中，訪問域名測試驗證，如下通過curl解析的方式測試驗證

ingress測試驗證

CLB規則

通過上面演示可知：

自動建立CLB例項CLB例項上配置監聽器配置轉發規則繫結Node節點繫結埠為service建立的NodePort1.3 ingress證書加密

TKE支援將在CLB中載入證書實現https加密傳輸，證書是經過第三方認證的CA簽名過的證書，需要先購買好證書，通過Secrets物件在kubernetes叢集中定義，如下演示https的實現。

1、 通過Secrets建立證書，先獲取到證書的id，如果沒有則先建立證書，證書管理，本文以證書id TKPmsWb3 為例，通過stringData能實現base64自動加密

apiVersion: v1kind: Secretmetadata: name: ingress-ssl-keystringData: qcloud_cert_id: TKPmsWb3 type: Opaque#生成Secrets物件[root@VM_10_2_centos ingress]# kubectl apply -f ingress-secret.yaml secret/ingress-ssl-key created[root@VM_10_2_centos ingress]# kubectl get secrets ingress-ssl-key NAME TYPE DATA AGEingress-ssl-key Opaque 1 7s#檢視secrets詳情，可得知VEtQbXNXYjM= 已自動通過base64加密[root@VM_10_2_centos ingress]# kubectl get secrets ingress-ssl-key -o yamlapiVersion: v1data: qcloud_cert_id: VEtQbXNXYjM= kind: Secretmetadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","kind":"Secret","metadata":{"annotations":{},"name":"ingress-ssl-key","namespace":"default"},"stringData":{"qcloud_cert_id":"TKPmsWb3"},"type":"Opaque"} creationTimestamp: "2020-01-03T11:53:33Z" name: ingress-ssl-key namespace: default resourceVersion: "7083702418" selfLink: /api/v1/namespaces/default/secrets/ingress-ssl-key uid: aaea4a86-2e1f-11ea-a618-ae9224ffad1atype: Opaque#可以通過base64檢視解密後的內容，和配置檔案中定義的id一致[root@VM_10_2_centos ingress]# echo VEtQbXNXYjM= | base64 -dTKPmsWb3

2、準備環境，建立一個nginx的Deployment

[root@VM_10_2_centos ~]# kubectl create deployment tke-ingress-ssl-demo --image=nginx:1.7.9deployment.apps/tke-ingress-ssl-demo created[root@VM_10_2_centos ~]# kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGEtke-ingress-ssl-demo 1/1 1 1 6s

3、將Deployment暴露以NodePort型別暴露service

[root@VM_10_2_centos ~]# kubectl expose deployment tke-ingress-ssl-demo --port=80 --type=NodePortservice/tke-ingress-ssl-demo exposed[root@VM_10_2_centos ~]# kubectl get service tke-ingress-ssl-demo -o yamlapiVersion: v1kind: Servicemetadata: creationTimestamp: "2020-01-03T12:00:05Z" labels: app: tke-ingress-ssl-demo name: tke-ingress-ssl-demo namespace: default resourceVersion: "7083890283" selfLink: /api/v1/namespaces/default/services/tke-ingress-ssl-demo uid: 94659f42-2e20-11ea-a618-ae9224ffad1aspec: clusterIP: 172.16.255.64 externalTrafficPolicy: Cluster ports: - nodePort: 30324 port: 80 protocol: TCP targetPort: 80 selector: app: tke-ingress-ssl-demo sessionAffinity: None type: NodePort #型別為NodePortstatus: loadBalancer: {}

4、定義ingress規則，載入證書實現https轉發

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: tke-ingress-ssl annotations: kubernetes.io/ingress.class: qcloud qcloud_cert_id: TKPmsWb3spec: rules: - host: www.happylauliu.cn http: paths: - path: / backend: serviceName: tke-ingress-ssl-demo servicePort: 80 tls: - hosts: - www.happylauliu.cn secretName: ingress-ssl-key

5、應用ingress規則，並檢視詳情，此時已正常建立CLB並配置規則

[root@VM_10_2_centos ingress]# kubectl apply -f ingress-demo.yaml ingress.extensions/tke-ingress-ssl created#檢視ingress詳情[root@VM_10_2_centos ingress]# kubectl describe ingresses tke-ingress-ssl Name: tke-ingress-sslNamespace: defaultAddress: 140.143.83.xxx #CLB的外網IPDefault backend: default-http-backend:80 (<none>)TLS: ingress-ssl-key terminates www.happylauliu.cnRules: Host Path Backends ---- ---- -------- www.happylauliu.cn / tke-ingress-ssl-demo:80 (172.16.0.25:80)Annotations: qcloud_cert_id: TKPmsWb3 kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"qcloud","qcloud_cert_id":"TKPmsWb3"},"name":"tke-ingress-ssl","namespace":"default"},"spec":{"rules":[{"host":"www.happylauliu.cn","http":{"paths":[{"backend":{"serviceName":"tke-ingress-ssl-demo","servicePort":80},"path":"/"}]}}],"tls":[{"hosts":["www.happylauliu.cn"],"secretName":"ingress-ssl-key"}]}} kubernetes.io/ingress.class: qcloud kubernetes.io/ingress.qcloud-loadbalance-id: lb-2kcrtwbn #CLB的例項idEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal EnsuringIngress 51s (x3 over 73s) loadbalancer-controller Ensuring ingress Normal CREATE 51s (x2 over 52s) loadbalancer-controller create loadbalancer succ Normal EnsuredIngress 49s (x3 over 52s) loadbalancer-controller Ensured ingress

6、測試驗證，hosts檔案中解析www.happylauliu.cn到CLB的VIP，或者DNS解析，開啟瀏覽器訪問站點，由於是經過CA認證簽名的證書，因此沒有提示告警資訊，檢視證書的詳情資訊

tke ingress ssl驗證

7、檢視CLB的配置可得知，CLB上配置了443的監聽埠，並關聯了證書，採用單向認證方式

tke ingres ssl配置規則

通過CLB的配置規則可知，CLB配置了監聽443的監聽器，80埠並未設定規則，因此此時無法訪問http，如何實現在TKE使用ingress實現http和https共存呢，可以通過定義kubernetes.io/ingress.http-rules和

kubernetes.io/ingress.https-rules實現

apiVersion: extensions/v1beta1kind: Ingressmetadata:  name: tke-ingress-ssl  annotations:    kubernetes.io/ingress.class: qcloud    kubernetes.io/ingress.rule-mix: "true"  #開啟混合規則配置，kubernetes.io/ingress.http-rules配置規則    kubernetes.io/ingress.http-rules: '[{"host":"www.happylauliu.cn","path":"/","backend":{"serviceName":"tke-ingress-ssl-demo","servicePort":"80"}}]'    qcloud_cert_id: TKPmsWb3spec:  rules:  - host: www.happylauliu.cn    http:      paths:      - path: /        backend:          serviceName: tke-ingress-ssl-demo           servicePort: 80  tls:  - hosts:    - www.happylauliu.cn    secretName: ingress-ssl-key

設定ingress.http-rules和ingress.https-rules註解之後，會在監聽器中建立http和https的轉發規則，並繫結RS，此時訪問http和https均能實現站點訪問，CLB對應的規則內容如下圖：

http和https規則混合使用

通過測試訪問http://www.happylauliu.cn/和https://www.happylauliu.cn/均能正常訪問，如果要實現訪問http自動跳轉到https，則可以在控制檯開啟自動跳轉的功能，如下圖：

開啟http自動重定向功能

開啟重定向功能後再次訪問http站點後此時會自動跳轉到https，如下圖所示location已經跳轉至https://www.happylauliu.cn/

http自動跳轉測試

寫在最後

通過上述的演示在騰訊雲公有云環境下ingress controller的實現，騰訊雲TKE通過使用CLB實現和kubernetes ingress整合，藉助於service的NodePort實現轉發，通過公有云專用的CLB能夠最大程度保障ingress接入效能。同時，ingress能夠使用騰訊雲上的證書實現https加密功能。

參考文獻

Ingress配置：https://kubernetes.io/docs/concepts/services-networking/ingress/

Ingress控制器：https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/

ingress基本配置：https://cloud.tencent.com/document/product/457/31711

ingress證書：https://cloud.tencent.com/document/product/457/40538

CLB配置http自動跳轉：https://cloud.tencent.com/document/product/214/8839

當你的才華撐不起你的野心時，你就應該靜下心來學習

97

Nginx

美好，一直在身邊

騰訊雲端計算

騰訊

雲端計算

最新評論

相關內容

  • ∧ BSA-TRITC(10mg/ml) TRITC-BSA 牛血清白蛋白改性標記羅丹明
  • ∨ kubernetes系列教程（十一）深入學習Deployment控制器