fix issue #40 and update log-pilot capability and doc example

AliyunContainerService · Oct 16, 2018 · 9a3ec0e · 9a3ec0e
1 parent 7e47a45
commit 9a3ec0e
Show file tree

Hide file tree

Showing 27 changed files with 282 additions and 282 deletions.
diff --git a/README.md b/README.md
@@ -14,13 +14,12 @@ Prerequisites:
 - Docker Engine >= 1.10
 
 ```
+# download log-pilot project
 git clone git@github.com:AliyunContainerService/log-pilot.git
 # build log-pilot image
-cd log-pilot/
-./build.sh
+cd log-pilot/ && ./build-image.sh
 # quick start
-cd ../quickstart/
-./run
+cd quickstart/ && ./run
 ```
 
 Then access kibana under the tips. You will find that tomcat's has been collected and sended to kibana.
@@ -37,10 +36,12 @@ Quickstart
 ### Run pilot
 
 ```
-docker run --rm -it --privileged \
+docker run --rm -it \
     -v /var/run/docker.sock:/var/run/docker.sock \
+    -v /etc/localtime:/etc/localtime \
     -v /:/host:ro \
-    registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:latest
+    --cap-add SYS_ADMIN \
+    registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-filebeat
 ```
 
 ### Run applications whose logs need to be collected
@@ -79,7 +80,7 @@ Prerequisites:
 go get github.com/AliyunContainerService/log-pilot
 cd $GOPATH/github.com/AliyunContainerService/log-pilot
 # This will create a new docker image named log-pilot:latest
-./build.sh
+./build-image.sh
 ```
 
 Contribute

diff --git a/assets/entrypoint b/assets/entrypoint
@@ -47,14 +47,15 @@ def run():
 def config():
     pilot_type = os.environ.get(ENV_PILOT_TYPE)
     if pilot_filebeat == pilot_type:
-        print "enable pilot:", pilot_filebeat
+        print "start log-pilot:", pilot_filebeat
         subprocess.check_call(['/pilot/config.filebeat'])
     else:
-        print "enable pilot:", pilot_fluentd
+        print "start log-pilot:", pilot_fluentd
         subprocess.check_call(['/pilot/config.fluentd'])
 
 
 if __name__ == '__main__':
     config()
+    # FIXME need SYS_ADMIN capability
     cleanup()
     run()
diff --git a/build.sh → build-image.sh b/build.sh → build-image.sh
diff --git a/docs/filebeat/docs.md b/docs/filebeat/docs.md
@@ -15,9 +15,9 @@ You must set environment variable ```PILOT_TYPE=filebeat``` to enable filebeat p
 ```
 docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
+   -v /etc/localtime:/etc/localtime \
    -v /:/host:ro \
-   -e PILOT_TYPE=filebeat \
-   registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:latest
+   registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-filebeat
 ```
 
 By default, all the logs that log-pilot collect will write to log-pilot's stdout. 
@@ -29,12 +29,12 @@ The command below run pilot with elastichsearch output, this makes log-pilot sen
 ```
 docker run --rm -it \
     -v /var/run/docker.sock:/var/run/docker.sock \
+    -v /etc/localtime:/etc/localtime \
     -v /:/host:ro \
-    -e PILOT_TYPE=filebeat \
     -e LOGGING_OUTPUT=elasticsearch \
     -e ELASTICSEARCH_HOST=${ELASTICSEARCH_HOST} \
     -e ELASTICSEARCH_PORT=${ELASTICSEARCH_PORT} \
-    registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:latest
+    registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-filebeat
 ```
 
 Log output plugin configuration
@@ -123,10 +123,6 @@ There are many labels you can use to describe the log info.
 - `aliyun.logs.$name=$path`
     - Name is an identify, can be any string you want. The valid characters in name are `0-9a-zA-Z_-`
     - Path is the log file path, can contians wildcard. `stdout` is a special value which means stdout of the container.
-- `aliyun.logs.$name.format=none|json|csv|nginx|apache2|regexp` format of the log
-    - none: pure text.
-    - json: a json object per line.
-    - regexp: use regex parse log. The pattern is specified by `aliyun.logs.$name.format.pattern = $regex`
 - `aliyun.logs.$name.tags="k1=v1,k2=v2"`: tags will be appended to log. 
 - `aliyun.logs.$name.target=target-for-log-storage`: target is used by the output plugins, instruct the plugins to store
 logs in appropriate place. For elasticsearch output, target means the log index in elasticsearch. For aliyun_sls output,

diff --git a/docs/fluentd/docs.md b/docs/fluentd/docs.md
@@ -15,9 +15,10 @@ You must set environment variable ```PILOT_TYPE=fluentd``` to enable fluentd plu
 ```
 docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
+   -v /etc/localtime:/etc/localtime \
    -v /:/host:ro \
-   -e PILOT_TYPE=fluentd \
-   registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:latest
+   --cap-add SYS_ADMIN \
+   registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-fluentd
 ```
 
 By default, all the logs that log-pilot collect will write to log-pilot's stdout. 
@@ -30,11 +31,10 @@ The command below run pilot with elastichsearch output, this makes log-pilot sen
 docker run --rm -it \
     -v /var/run/docker.sock:/var/run/docker.sock \
     -v /:/host:ro \
-    -e PILOT_TYPE=fluentd \
     -e LOGGING_OUTPUT=elasticsearch \
     -e ELASTICSEARCH_HOST=${ELASTICSEARCH_HOST} \
     -e ELASTICSEARCH_PORT=${ELASTICSEARCH_PORT} \
-    registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:latest
+    registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-fluentd
 ```
 
 Log output plugin configuration

diff --git a/docs/fluentd/output/aliyun_sls.md b/docs/fluentd/output/aliyun_sls.md
@@ -30,17 +30,18 @@ swm2ft9bzaxyyi9umwbb0mdd6   aliyun_access_key   53 minutes ago      53 minutes a
 ````
 
 #### Example for non-swarm-mode setup
-* For `registry.cn-hangzhou.aliyuncs.com/acs-sample/fluentd-pilot:latest`
+* For `registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-fluentd`
 ````
 docker run --rm -it \
     -v /var/run/docker.sock:/var/run/docker.sock \
-    -v /:/host \
+    -v /etc/localtime:/etc/localtime \
+    -v /:/host:ro \
     -e LOGGING_OUTPUT=aliyun_sls \
     -e ALIYUNSLS_PROJECT="your-aliyun-sls-project-name"  \
     -e ALIYUNSLS_REGION_ENDPOINT=cn-hangzhou.log.aliyuncs.com \
     -e ALIYUNSLS_ACCESS_KEY_ID="your-access-key-id" \
     -e ALIYUNSLS_ACCESS_KEY_SECRET="your-access-key-secret"  \
-    registry.cn-hangzhou.aliyuncs.com/acs-sample/fluentd-pilot:latest
+    registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-fluentd
 ````
 
 
@@ -56,7 +57,7 @@ since there is no project name for `tomcat`, the stdout of `tomcat` will streame
 
 
 #### Example for swarm-mode setup with docker secrets
-* For `registry.cn-hangzhou.aliyuncs.com/acs-sample/fluentd-pilot:latest`
+* For `registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-fluentd`
 ````
 $ echo "your-access-key-id:your-access-key-secret" | docker secret create aliyun_access_key -
 $ docker service create    -t \
@@ -66,7 +67,7 @@ $ docker service create    -t \
     -e LOGGING_OUTPUT=aliyun_sls \
     -e ALIYUNSLS_PROJECT="your-aliyun-sls-project-name"  \
     -e ALIYUNSLS_REGION_ENDPOINT=cn-hangzhou.log.aliyuncs.com \
-    registry.cn-hangzhou.aliyuncs.com/acs-sample/fluentd-pilot:latest
+    registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-fluentd
 ````
 * For your app, suppose a tomcat
 ````

diff --git a/examples/pilot-elasticsearch-kubernetes.yml b/examples/pilot-elasticsearch-kubernetes.yml
@@ -5,6 +5,8 @@ metadata:
   labels:
     k8s-app: log-pilot
 spec:
+  updateStrategy:
+    type: RollingUpdate
   template:
     metadata:
       labels:
@@ -15,7 +17,7 @@ spec:
         effect: NoSchedule
       containers:
       - name: log-pilot
-        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:0.9.1-fluentd
+        image: registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-fluentd
         env:
           - name: "LOGGING_OUTPUT"
             value: "elasticsearch"
@@ -35,6 +37,8 @@ spec:
           readOnly: true
         - name: pos
           mountPath: /pilot/pos
+        - name: localtime
+          mountPath: /etc/localtime
         securityContext:
           capabilities:
             add:
@@ -49,4 +53,6 @@ spec:
           path: /
       - name: pos
         emptyDir: {}
-
+      - name: localtime
+        hostPath:
+          path: /etc/localtime
diff --git a/examples/pilot-elastisearch-kubernetes-2.yml b/examples/pilot-elastisearch-kubernetes-2.yml
@@ -5,6 +5,8 @@ metadata:
   labels:
     k8s-app: log-pilot
 spec:
+  updateStrategy:
+    type: RollingUpdate
   template:
     metadata:
       labels:
@@ -15,7 +17,7 @@ spec:
         effect: NoSchedule
       containers:
       - name: log-pilot
-        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:0.9-filebeat
+        image: registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-filebeat
         env:
           - name: "LOGGING_OUTPUT"
             value: "elasticsearch"
@@ -35,6 +37,8 @@ spec:
         - name: root
           mountPath: /host
           readOnly: true
+        - name: localtime
+          mountPath: /etc/localtime
         securityContext:
           capabilities:
             add:
@@ -53,3 +57,6 @@ spec:
       - name: root
         hostPath:
           path: /
+      - name: localtime
+        hostPath:
+          path: /etc/localtime
diff --git a/examples/pilot-kafka-kubernetes-2.yml b/examples/pilot-kafka-kubernetes-2.yml
@@ -5,6 +5,8 @@ metadata:
   labels:
     k8s-app: log-pilot
 spec:
+  updateStrategy:
+    type: RollingUpdate
   template:
     metadata:
       labels:
@@ -15,7 +17,7 @@ spec:
         effect: NoSchedule
       containers:
       - name: log-pilot
-        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:0.9.5-filebeat
+        image: registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-filebeat
         env:
           - name: "LOGGING_OUTPUT"
             value: "kafka"
@@ -35,6 +37,8 @@ spec:
         - name: root
           mountPath: /host
           readOnly: true
+        - name: localtime
+          mountPath: /etc/localtime
         securityContext:
           capabilities:
             add:
@@ -53,3 +57,6 @@ spec:
       - name: root
         hostPath:
           path: /
+      - name: localtime
+        hostPath:
+          path: /etc/localtime
diff --git a/examples/pilot-kafka-kubernetes.yml b/examples/pilot-kafka-kubernetes.yml
@@ -5,6 +5,8 @@ metadata:
   labels:
     k8s-app: log-pilot
 spec:
+  updateStrategy:
+    type: RollingUpdate
   template:
     metadata:
       labels:
@@ -15,7 +17,7 @@ spec:
         effect: NoSchedule
       containers:
       - name: log-pilot
-        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:0.9.1-fluentd
+        image: registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-fluentd
         env:
           - name: "LOGGING_OUTPUT"
             value: "kafka"
@@ -33,6 +35,8 @@ spec:
           readOnly: true
         - name: pos
           mountPath: /pilot/pos
+        - name: localtime
+          mountPath: /etc/localtime
         securityContext:
           capabilities:
             add:
@@ -47,4 +51,6 @@ spec:
           path: /
       - name: pos
         emptyDir: {}
-
+      - name: localtime
+        hostPath:
+          path: /etc/localtime