Fluentbit

Fluent Bit v3.1 Documentation | Fluent Bit: Official Manual

ENV

EKS_CLUSTER_NAME="<Cluster Name>"
Shell
복사

Create Policy & IRSA

cat << EOF > fluent-bit-cloudwatch-policy.json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:DescribeLogStreams",
                "logs:DescribeLogGroups"
            ],
            "Resource": [
                "arn:aws:logs:*:*:log-group:skills/app:*",
                "arn:aws:logs:*:*:log-group:skills/app"
            ]
        }
    ]
}
EOF
Shell
복사

aws iam create-policy --policy-name FluentBitCloudWatchLogsPolicy --policy-document file://fluent-bit-cloudwatch-policy.json
Shell
복사

POLICY_ARN=$(aws iam list-policies --query 'Policies[?PolicyName==`FluentBitCloudWatchLogsPolicy`].Arn' --output text)

eksctl create iamserviceaccount \
    --name fluent-bit \
    --region="ap-northeast-2" \
    --cluster "$EKS_CLUSTER_NAME" \
    --namespace=skills \
    --attach-policy-arn "$POLICY_ARN" \
    --override-existing-serviceaccounts \
    --approve
Shell
복사

Attach IAM Role for EKS NodeGroup

NODEGROUP_ROLE_NAME=$(aws eks describe-nodegroup --cluster-name $EKS_CLUSTER_NAME --nodegroup-name $EKS_NODE_GROUP_NAME --query "nodegroup.nodeRole" --output text | cut -d'/' -f2-)

aws iam attach-role-policy --role-name $NODEGROUP_ROLE_NAME --policy-arn arn:aws:iam::362708816803:policy/FluentBitCloudWatchLogsPolicy
Shell
복사

OIDC

eksctl utils associate-iam-oidc-provider --region=ap-northeast-2 --cluster=$EKS_CLUSTER_NAME --approve
Shell
복사

•

Delete IRSA

eksctl delete iamserviceaccount --region ap-northeast-2 \
    --name fluent-bit \
    --namespace skills \
    --cluster $EKS_CLUSTER_NAME
Shell
복사

•

fluentbit-cm.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-bit-config
  namespace: skills
data:
  fluent-bit.conf: |
    [SERVICE]
      Flush        1
      Log_Level    info
      Daemon       Off
      Parsers_File parsers.conf
      HTTP_Server  On
      HTTP_Listen  0.0.0.0
      HTTP_Port    2020

    [INPUT]
      Name              tail
      Tag               kube.*
      Path              /var/log/containers/*.log
      Parser            docker
      DB                /var/log/fluent-bit.db

    [OUTPUT]
      Name              cloudwatch_logs
      Match             kube.*
      region            ap-northeast-2
      log_group_name    skills/app
      log_stream_prefix fluent
YAML
복사

kubectl apply -f fluentbit-cm.yaml
Shell
복사

•

fluentbit-ds.yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluent-bit
  namespace: skills
spec:
  selector:
    matchLabels:
      app: fluent-bit
  template:
    metadata:
      labels:
        app: fluent-bit
    spec:
      containers:
        - name: fluent-bit
          image: fluent/fluent-bit:latest
          ports:
            - containerPort: 2020
          volumeMounts:
            - name: varlog
              mountPath: /var/log
            - name: varlibdockercontainers
              mountPath: /var/lib/docker/containers
              readOnly: true
            - name: fluent-bit-config
              mountPath: /fluent-bit/etc/
      volumes:
        - name: varlog
          hostPath:
            path: /var/log
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: fluent-bit-config
          configMap:
            name: fluent-bit-config
YAML
복사

kubectl apply -f fluentbit-ds.yaml
Shell
복사

•

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: skills-app-deployment
  namespace: skills
spec:
  selector:
    matchLabels:
      app: skills-app
  replicas: 2
  template:
    metadata:
      labels:
        app: skills-app
    spec:
      nodeSelector:
        eks.amazonaws.com/nodegroup: skills-app-nodegroup
      containers:
        - name: skills-app
          image: 362708816803.dkr.ecr.ap-northeast-2.amazonaws.com/skills-app:latest
          ports:
            - containerPort: 8080
          imagePullPolicy: Always
YAML
복사

kubectl apply -f deployment.yaml
YAML
복사