1、收集容器中日誌檔案

大致思路：

在Pod中增加一個容器執行日誌採集器，使用emtyDir共享日誌目錄讓日誌採集器讀取到業務容器的日誌檔案

PS: 收集容器中日誌檔案所需要的Pod Yaml 檔案 在 https://github.com/fxkjnj/kubernetes/elk-for-kubernetes/es-single-node/app-tomcat-filebeat-log 目錄下

編寫dockerfile，建立 一個標準的tomcat8 映象

PS: 確保本機有docker 的環境, 如果沒有部署docker 可以參考我的另一篇文章

https://www.fxkjnj.com/?p=2732

當然如果不想自己製作映象，也可以使用我製作好的tomcat8 映象 docker pull feixiangkeji974907/tomcat-test:v8

建立軟體目錄，下載tomcat8, jdk1.8

[root@master-1 es-single-node]# mkdir app-tomcat-filebeat-log

[root@master-1 es-single-node]# cd app-tomcat-filebeat-log[root@master-1 app-tomcat-filebeat-log]# http://jpg.fxkjnj.com/ruanjian/apache-tomcat-8.5.39.tar.gz[root@master-1 app-tomcat-filebeat-log]# http://jpg.fxkjnj.com/ruanjian/jdk1.8.0_66.tar.gz

編寫dockerfile

cat >  Dockerfile  << EOFFROM centosMAINTAINER fxkjnj.com fxkjEXPOSE 8080WORKDIR /opt#ADD jdk1.8    COPY jdk1.8.0_66.tar.gz /opt    RUN tar zxf /opt/jdk1.8.0_66.tar.gz -C /usr/local/ && rm -rf /opt/jdk1.8.0_66.tar.gz    RUN ln -s /usr/local/jdk1.8.0_66 /usr/local/jdk#環境變數/etc/profile    ENV JAVA_HOME /usr/local/jdk    ENV CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar    ENV PATH $PATH:$JAVA_HOME/bin#ADD tomcat8    COPY apache-tomcat-8.5.39.tar.gz /opt    RUN tar zxf apache-tomcat-8.5.39.tar.gz -C /usr/local  && rm -rf apache-tomcat-8.5.39.tar.gz    RUN mv /usr/local/apache-tomcat-8.5.39 /usr/local/tomcat#CMDENTRYPOINT /usr/local/tomcat/bin/startup.sh && tail -f /usr/local/tomcat/logs/catalina.outEOF

構建映象

[root@master-1 app-tomcat-filebeat-log]# docker build -t feixiangkeji974907/tomcat-test:v8 /root/kubernetes/elk-for-kubernetes/es-single-node/app-tomcat-filebeat-log/Sending build context to Docker daemon  191.2MBStep 1/14 : FROM centoslatest: Pulling from library/centos7a0437f04f83: Pull complete Digest: sha256:5528e8b1b1719d34604c87e11dcd1c0a20bedf46e83b5632cdeac91b8c04efc1Status: Downloaded newer image for centos:latest ---> 300e315adb2fStep 2/14 : MAINTAINER fxkjnj.com fxkj ---> Running in c6960bcfe61fRemoving intermediate container c6960bcfe61f ---> 4d90c5f058e4Step 3/14 : EXPOSE 8080 ---> Running in 4b74564852a6Removing intermediate container 4b74564852a6 ---> 1d513bed4b8aStep 4/14 : WORKDIR /opt ---> Running in ba66ad1e1f2bRemoving intermediate container ba66ad1e1f2b ---> af3d2848cd2aStep 5/14 : COPY jdk1.8.0_66.tar.gz /opt ---> 5407bdfd840eStep 6/14 : RUN tar zxf /opt/jdk1.8.0_66.tar.gz -C /usr/local/ && rm -rf /opt/jdk1.8.0_66.tar.gz ---> Running in 969ef89b2a29Removing intermediate container 969ef89b2a29 ---> 84717736fc66Step 7/14 : RUN ln -s /usr/local/jdk1.8.0_66 /usr/local/jdk ---> Running in 3e2a24de56fdRemoving intermediate container 3e2a24de56fd ---> 807c98672e7fStep 8/14 : ENV JAVA_HOME /usr/local/jdk ---> Running in c1f21968d26cRemoving intermediate container c1f21968d26c ---> a24e93067d43Step 9/14 : ENV CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar ---> Running in 1bc184124271Removing intermediate container 1bc184124271 ---> 50e6aa9d66f9Step 10/14 : ENV PATH $PATH:$JAVA_HOME/bin ---> Running in 104d6ee96bfbRemoving intermediate container 104d6ee96bfb ---> 7ff4d81f456cStep 11/14 : COPY apache-tomcat-8.5.39.tar.gz /opt ---> 4815155b0c9fStep 12/14 : RUN tar zxf apache-tomcat-8.5.39.tar.gz -C /usr/local  && rm -rf /opt/apache-tomcat-8.5.39.zip ---> Running in b5d13adfbf93Removing intermediate container b5d13adfbf93 ---> 49413a5efaedStep 13/14 : RUN mv /usr/local/apache-tomcat-8.5.39 /usr/local/tomcat ---> Running in a2ea891bb8b2Removing intermediate container a2ea891bb8b2 ---> 6c71db7365e9Step 14/14 : ENTRYPOINT /usr/local/tomcat/bin/startup.sh && tail -f /usr/local/tomcat/logs/catalina.out ---> Running in f01fa6926b74Removing intermediate container f01fa6926b74 ---> 0686065360e3Successfully built 0686065360e3Successfully tagged feixiangkeji974907/tomcat-test:v8

測試下映象，啟動容器

 [root@master-1 app-tomcat-filebeat-log]# docker run --name tomcat -itd -p 80:8080 feixiangkeji974907/tomcat-test:v8 

訪問tomcat: http://192.168.31.61 可以看到首頁效果

如果需要替換war包操作。可以將上面製作的tomcat8 映象為基礎映象，在寫一個dockerfile。我這裡提供一下cat >  Dockerfile  << EOFFROM feixiangkeji974907/tomcat-test:v8MAINTAINER fxkjnj.com fxkjCOPY app.war /optRUN unzip /opt/app.war -d /usr/local/tomcat/webapps/ && rm -rf /opt/app.warEOF[root@master-1 app-tomcat-filebeat-log]# docker build -t tomcat-app:v1 .

建立 app-tomcat-log-logfile.yaml 檔案，並加入 Filebeat 來收集tomcat容器日誌

cat >  app-tomcat-log-logfile.yaml   << EOFapiVersion: apps/v1kind: Deploymentmetadata:  name: tomcat-logfilespec:  replicas: 3  selector:    matchLabels:      project: tomcat-app      app: tomcat-logfile  template:    metadata:      labels:        project: tomcat-app        app: tomcat-logfile    spec:      containers:      # 應用容器      - name: tomcat        image: feixiangkeji974907/tomcat-test:v8        # 將資料卷掛載到日誌目錄        volumeMounts:        - name: tomcat-logs           mountPath: /usr/local/tomcat/logs      # 日誌採集器容器      - name: filebeat        image: elastic/filebeat:7.9.2         args: [          "-c", "/etc/filebeat.yml",          "-e",        ]        resources:          requests:            cpu: 100m            memory: 100Mi          limits:            memory: 500Mi        securityContext:          runAsUser: 0        volumeMounts:        # 掛載filebeat配置檔案        - name: filebeat-config          mountPath: /etc/filebeat.yml          subPath: filebeat.yml        # 將資料卷掛載到日誌目錄        - name: tomcat-logs           mountPath: /usr/local/tomcat/logs      # 資料卷共享日誌目錄      volumes:      - name: tomcat-logs        emptyDir: {}      - name: filebeat-config        configMap:          name: filebeat-tomcat-config---apiVersion: v1kind: Servicemetadata:  name: app-log-logfilespec:  ports:  - port: 80    protocol: TCP    targetPort: 8080  selector:    project: tomcat-app    app: tomcat-logfile---apiVersion: v1kind: ConfigMapmetadata:  name: filebeat-tomcat-config  data:  # 配置檔案儲存在ConfigMap  filebeat.yml: |-    filebeat.inputs:      - type: log        paths:          - /usr/local/tomcat/logs/localhost_access_log.*        # tags: ["access-log"]        # fields_under_root，如果值為ture，那麼fields 欄位儲存在輸出文件的頂級位置，如果與filebeat中欄位衝突，自定義欄位會覆蓋其他欄位        fields_under_root: true        fields:          project: tomcat-app          app: tomcat-logfile        #自定義ES的索引需要把ilm設定為false        #定義模板的相關資訊    setup.ilm.enabled: false    setup.template.name: "tomcat-access"    setup.template.pattern: "tomcat-access-*"    output.elasticsearch:      hosts: ['elasticsearch.ops:9200']      index: "tomcat-access-%{+yyyy.MM.dd}"EOF[root@master-1 app-tomcat-filebeat-log]# kubectl apply -f app-tomcat-log-logfile.yaml deployment.apps/tomcat-logfile createdservice/app-log-logfile createdconfigmap/filebeat-tomcat-config created

檢視tomcat pod，service 狀態

[root@master-1 es-single-node]# kubectl get podsNAME                              READY   STATUS    RESTARTS   AGEtomcat-logfile-694d588b78-7k97g   2/2     Running   0          5m36stomcat-logfile-694d588b78-phnxt   2/2     Running   0          5m36stomcat-logfile-694d588b78-vmp25   2/2     Running   0          5m36s[root@master-1 es-single-node]# kubectl get svcNAME              TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGEapp-log-logfile   ClusterIP   10.0.0.194   <none>        80/TCP    5m40s

登陸kibana 管理索引， 新增索引模式

索引管理：

（一般只要有資料入到ES中就會有索引出現 ，如果沒有出現可以試著訪問下業務使其產生日誌輸出到ES中）

新增索引模式:

輸入索引模式名稱：tomcat-access-*

表示可以匹配到上面的索引 tomcat-access-2021.03.08

選擇@timestamp 時間欄位

訪問tomcat 的Pod 使其產生日誌

[root@node-1 ~]# curl -I 10.0.0.194HTTP/1.1 200 Content-Type: text/html;charset=UTF-8Transfer-Encoding: chunkedDate: Mon, 08 Mar 2021 10:11:17 GMT

登陸kibana dashboard 檢索tomcat 日誌

點選左邊的Discover,選擇正確的索引

檢索的語句： project : "tomcat-app"

可以看到有1個 日誌被命中了