非容器應用的網格化需要前一篇講述的WorkloadEntry配置,同時需要本篇講述的sidecar配置。前者讓kubernetes容器內的POD可以找到VM中的應用,後者讓VM中的應用可以找到POD。(注:本系列的重點是流量轉移,但VM網格化後不止具備流量管理能力。)
1 搭建實驗環境示例實驗需要一個ack叢集和3個ecs節點。在ack叢集中,包含hello1和hello3版本為en的POD,3個ecs節點各啟動一個hello2 app,每個app對應一個版本(en/fr/es)的hello容器。請求從hello1 POD到VM中的hello2 app,再到hello3 POD。在此基礎上,我們希望路由到hello2 en/fr/es的流量比例為:30%:60%:10%。
示例(http_reciprocal_demo)包含如下元素:
hello1 deployment(映象http_springboot_v1)hello3 deployment(映象http_springboot_v1)hello2 docker containers(映象http_springboot_v1/http_springboot_v2/http_springboot_v3)入口閘道器:istio-ingressgateway入口流量配置:gateway/virtualservicehello1流量配置:hello1 service(/hello1 virtualservice/hello1 destinationrule)hello2流量配置:hello2 service/hello2 virtualservice/hello2 destinationrulehello3流量配置:hello3 service(/hello3 virtualservice/hello3 destinationrule)hello2 serviceentry/hello2 workloadentry啟動hello2應用
sh vm/ssh1.shdocker run \\--rm \\--network host \\--name http_v1 \\-e HTTP_HELLO_BACKEND=hello3-svc.http-reciprocal-hello.svc.cluster.local \\registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v1:1.0.1
sh vm/ssh2.shdocker run \\--rm \\--network host \\--name http_v2 \\-e HTTP_HELLO_BACKEND=hello3-svc.http-reciprocal-hello.svc.cluster.local \\registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v2:1.0.1
sh vm/ssh3.shdocker run \\--rm \\--network host \\--name http_v3 \\-e HTTP_HELLO_BACKEND=hello3-svc.http-reciprocal-hello.svc.cluster.local \\registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v3:1.0.1
啟動hello1/hello3 POD
ack.deploy.sh
k apply -f data_plane/http-reciprocal-hello-all.yamlk -n http-reciprocal-hello get pod,svcNAME READY STATUS RESTARTS AGEpod/hello1-deploy-6c857bc499-f658r 2/2 Running 0 2m7spod/hello3-deploy-f57bb9db7-lzvg8 2/2 Running 0 2m7sNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEservice/hello1-svc ClusterIP 172.19.8.178 <none> 8004/TCP 2m7sservice/hello3-svc ClusterIP 172.19.3.17 <none> 8001/TCP 6s
配置WorkloadEntry等設施
asm.deploy.sh/asm_traffic_shift.sh
aliyun servicemesh AddVmAppToMesh \\ --ServiceMeshId "$MESH_ID" \\ --Namespace external-hello \\ --ServiceName hello2-svc \\ --Ips "$VM_PRI_1","$VM_PRI_2","$VM_PRI_3" \\ --Ports http:8001 \\ --Labels app=hello-workload m apply -f control_plane/http-reciprocal-hello-all.yamlm -n http-reciprocal-hello get serviceentry,workloadentryNAME AGEserviceentry.networking.istio.io/mesh-expansion-hello2-svc 73sNAME AGEworkloadentry.networking.istio.io/mesh-expansion-hello2-svc-1 73sworkloadentry.networking.istio.io/mesh-expansion-hello2-svc-2 73sworkloadentry.networking.istio.io/mesh-expansion-hello2-svc-3 73sm -n http-reciprocal-hello get gateway,virtualservice,destinationruleNAME AGEgateway.networking.istio.io/hello-gateway 2m25sNAME AGEvirtualservice.networking.istio.io/gateway-vs 2m25svirtualservice.networking.istio.io/hello2-vs 2m25sNAME AGEdestinationrule.networking.istio.io/hello2-dr 2m25s
為WorkloadEntry新增version標籤
分別為3個WorkloadEntry新增version:v1/version:v2/version:v3標籤,示意如下。
spec: address: 192.168.0.170 labels: app: hello-workload version: v1
為VM新增hello3 DNS
為虛擬機器節點新增hello3 service的clusterIp
hello3_svc_ip=$(k get svc hello3-svc -n hybrid-hello -o jsonpath='{.spec.clusterIP}')echo "$hello3_svc_ip hello3-svc.http-reciprocal-hello.svc.cluster.local" > dns_recordVMS=("$VM_PUB_1" "$VM_PUB_2" "$VM_PUB_3")for vm in "${VMS[@]}"; do ssh root@"$vm" "cat >> /etc/hosts" < dns_recorddonerm -rf dns_record
驗證虛擬機器對hello3 POD和hello3 service的訪問
for vm in "${VMS[@]}"; do hello3_pod_ip=$(k get pod -l app=hello3-deploy -n http-reciprocal-hello -o jsonpath={.items[*].status.podIP}) echo "Test access to hello3 pod on $vm" ssh root@"$vm" "curl -s $hello3_pod_ip:8001/hello/pod_testing_msg" echo echo "Test access to hello3 service on $vm" ssh root@"$vm" "curl -s hello3-svc.trace-hello.svc.cluster.local:8001/hello/svc_testing_msg" echodone
2 KUBE驗證-POD和VM互訪
完成實驗環境搭建後,我們首先驗證POD和VM互訪。
首先,我們從hello1 POD中分別直接向3個VM中的應用發起請求,驗證hello2 app是否可以訪問到hello3 POD
hello1_pod=$(k get pod -l app=hello1-deploy -n http-reciprocal-hello -o jsonpath={.items..metadata.name})echo "Test access vm ip directly"VMS=("$VM_PRI_1" "$VM_PRI_2" "$VM_PRI_3")for vm in "${VMS[@]}"; do k exec "$hello1_pod" -c hello-v1-deploy -n http-reciprocal-hello -- curl -s "$vm":8001/hello/eric echodone
驗證結果如下,我們期待3個hello2 app均能返回下游hello3的資訊,證明VM已經完全網格化。hello2 app通過sidecar找到了hello3 svc對應的POD ip,並完成請求。
Test access vm ip directlyHello eric(192.168.0.170)<-Hello eric(172.18.1.36)Bonjour eric(192.168.0.171)<-Hello eric(172.18.1.36)Hola eric(192.168.0.172)<-Hello eric(172.18.1.36)
直接訪問hello2 app所在ecs節點的ip驗證通過後,我們再從hello1 POD中向hello2 svc發起請求,驗證hello1 POD到hello2 VM是否生效。這其實是在迴歸前一篇介紹的WorkloadEntry是否生效。
echo "Test access hello2-svc"k exec "$hello1_pod" -c hello-v1-deploy -n http-reciprocal-hello \\-- curl -s hello2-svc.http-reciprocal-hello.svc.cluster.local:8001/hello/eric
驗證結果如下,我們期待hello1 POD通過ServiceEntry和WorkloadEntry找到相應的VM。
Test access hello2-svcHello eric(192.168.0.170)<-Hello eric(172.18.1.36)
最後我們在hello1 POD中驗證全鏈路:
echo "Test access hello1-svc"k exec "$hello1_pod" -c hello-v1-deploy -n http-reciprocal-hello \\-- curl -s hello1-svc.http-reciprocal-hello.svc.cluster.local:8004/hello/ericTest access hello1-svcHello eric(172.18.0.230)<-Bonjour eric(192.168.0.171)<-Hello eric(172.18.1.36)
3 MESH驗證-全鏈路流量轉移
物理鏈路驗證完畢,我們再來驗證邏輯鏈路,即流量轉移的配比是否生效。
這個驗證我們從本地向服務網格入口閘道器發起。首先確認本示例對外暴露的8004 埠已經在IstioGateway在配置好,然後獲取istio-ingressgateway的ip,並通過ip發起請求。指令碼示意如下:
m get IstioGateway -n istio-system -o jsonpath='{.items[0].spec.ports[?(@.name=="http-reciprocal")]}'map[name:http-reciprocal port:8004 targetPort:8004]
IP=$(k -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')for i in {1..100}; do resp=$(curl -s "$IP":8004/hello/eric) echo "$resp" >>test_traffic_shift_resultdoneecho "expected 30%(Hello eric)-60%(Bonjour eric)-10%(Hola eric):"sort test_traffic_shift_result | grep -v "^[[:space:]]*$"| uniq -c | sort -nrk1
驗證結果如下:
Test route n a loopexpected 30%(Hello eric)-60%(Bonjour eric)-10%(Hola eric): 61 Hello eric(172.18.0.230)<-Bonjour eric(192.168.0.171)<-Hello eric(172.18.1.36) 28 Hello eric(172.18.0.230)<-Hello eric(192.168.0.170)<-Hello eric(172.18.1.36) 11 Hello eric(172.18.0.230)<-Hola eric(192.168.0.172)<-Hello eric(172.18.1.36)
到此,POD和VM互訪的http協議流量管理驗證完畢。
本篇示例完整演示了非容器應用網格化過程中的一個比較經典的場景。覆蓋到從istio-ingressgateway到deployment、workloadentry等各種CRD的配置,較完整地展示了POD和VM互訪中遇到的各技術點的配置。在有了http協議的基礎後,下一篇我們使用grpc代替http再次驗證,並著重關注grpc的長連結特性。