EKS_CLUSTER_NAME="skills-eks-cluster"
EKS_NODE_GROUP_NAME="skills-app-nodegroup"
EKS_NODE_ROLE_NAME=$(aws eks describe-nodegroup --cluster-name $EKS_CLUSTER_NAME --nodegroup-name $EKS_NODE_GROUP_NAME --query "nodegroup.nodeRole" --output text | cut -d'/' -f2-)
Shell
복사
cat << EOF > iam_role_policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:DescribeLogStreams"
],
"Resource": [
"arn:aws:logs:*:*:*"
]
}
]
}
EOF
Shell
복사
aws iam create-policy --policy-name EKS-CloudWatchLogs --policy-document file://iam_role_policy.json
Shell
복사
aws iam attach-role-policy --role-name $EKS_NODE_ROLE_NAME --policy-arn `aws iam list-policies | jq -r '.[][] | select(.PolicyName == "EKS-CloudWatchLogs") | .Arn'`
Shell
복사
apiVersion: v1
kind: ConfigMap
metadata:
name: fluent-bit-config
labels:
app.kubernetes.io/name: fluentbit
data:
fluent-bit.conf: |
[SERVICE]
Parsers_File parsers.conf
[INPUT]
Name tail
Tag falco.*
Path /var/log/containers/falco*.log
Parser falco
DB /var/log/flb_falco.db
Mem_Buf_Limit 5MB
Skip_Long_Lines On
Refresh_Interval 10
[OUTPUT]
Name cloudwatch
Match falco.**
region ap-northeast-2
log_group_name falco
log_stream_name alerts
auto_create_group true
parsers.conf: |
[PARSER]
Name falco
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep Off
# Command | Decoder | Field | Optional Action
# =============|==================|=================
Decode_Field_As json log
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentbit
labels:
app.kubernetes.io/name: fluentbit
spec:
selector:
matchLabels:
name: fluentbit
template:
metadata:
labels:
name: fluentbit
spec:
serviceAccountName: fluent-bit
containers:
- name: aws-for-fluent-bit
image: amazon/aws-for-fluent-bit:latest
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: fluent-bit-config
mountPath: /fluent-bit/etc/
- name: mnt
mountPath: /mnt
readOnly: true
resources:
limits:
memory: 500Mi
requests:
cpu: 500m
memory: 100Mi
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: fluent-bit-config
configMap:
name: fluent-bit-config
- name: mnt
hostPath:
path: /mnt
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluent-bit
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: pod-log-reader
rules:
- apiGroups: [""]
resources:
- namespaces
- pods
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: pod-log-crb
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: pod-log-reader
subjects:
- kind: ServiceAccount
name: fluent-bit
namespace: default
YAML
복사
kubectl apply -f fluent-bit-configuration.yaml
Shell
복사
cat << EOF > values.yaml
tty: true
falcosidekick:
enabled: true
webui:
enabled: true
user: "admin:admin"
service:
type: LoadBalancer
redis:
storageEnabled: false
# config:
# aws:
# cloudwatchlogs:
# loggroup: "falco-log"
# logstream: "detect-events"
# minimumpriority: ""
EOF
Shell
복사
Log 활성화 시 노드그룹에 CloudWatch 권한 필요
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update
helm upgrade --install falco --namespace falco --create-namespace -f values.yaml falcosecurity/falco
Shell
복사
•
Falco UI Dashboard (alb + 2802포트로 접근)
kubectl get svc -n falco
Shell
복사
kubectl get pods -n falco
Shell
복사
kubectl logs -n falco -l app.kubernetes.io/instance=falco -c falco
Shell
복사
Rule Condition line별 의미
•
spawned_process
새로운 프로세스가 생성되는 이벤트에 해당하는지 확인함
→ 프로세스가 새로 실행 됐을 때만 탐지
•
container
이벤트가 컨테이너 내부에서 발생했는지 확인함
•
proc.name = nc
실행된 프로세스의 이름이 nc 인지 확인
→ basename만 포함이기 때문에 /usr/bin/nc여도 nc로 작성
•
proc.args
실행된 명령어 이후에 넘겨진 모든 인자 목록
ex) nc -zv 10.0.0.1 22 이 명령어의 경우 proc.args는 -zv 10.0.0.1 22 이기 때문에 검사 시 contains로 -zv 또는 10.0.0.1, 22 이런식으로 검사 가능
proc.args = “” 로 인자가 없는 명령인지도 검사 가능
•
proc_name_exists
프로세스의 이름이 실제로 존재하는 이벤트인지 확인
→ 비정상 event, syscall등에 방지해서 null 오류가 나오는 경우를 방지 해서 안정성 향상
활용 가능한 kubernetes metadata field
k8s.ns.name | 네임스페이스 이름 |
k8s.pod.name | Pod 이름 |
k8s.pod.uid | Pod UID |
k8s.rc.name / k8s.rs.name / k8s.deployment.name | 소속된 컨트롤러 |
k8s.container.name | 컨테이너 이름 |
k8s.node.name | 실행 중인 노드 이름 |
터미널 1
•
지속적인 로그 탐지
kubectl logs -n falco -l app.kubernetes.io/instance=falco -c falco -f
Shell
복사
터미널 2 (default rule 중에 대표적인 Terminal shell in container 규칙 테스트)
•
"falco-demo" 라는 새로운 Pod 을 배포한 후 Shell 접근
kubectl run falco-demo -n falco --rm -ti --image busybox /bin/sh
Shell
복사
touch /etc/2
Shell
복사
default rule disable 하는 방법 (Contact K8S API Server From Container 라는 rule을 disable)
customRules:
disable-contacts-k8s.yaml: |-
- rule: Contact K8S API Server From Container
override:
enabled: replace
enabled: false
YAML
복사
helm upgrade falco falcosecurity/falco -n falco --reuse-values -f update-default-rule.yaml
Shell
복사
•
Allow Terminal Shell
customRules:
allow-terminal-shell.yaml: |-
- macro: user_expected_terminal_shell_in_container_conditions
condition: (k8s.ns.name in (default, kube-system))
YAML
복사
•
컨테이너 내에서 netcat(nc) 실행
customRules:
detect-nc-port-scan.yaml: |-
- rule: Netcat port scan attempt
desc: Detect use of netcat for port scanning inside a Kubernetes container
condition: >
spawned_process
and container
and proc.name = nc
and proc_name_exists
output: >
ALERT: Netcat port scan detected (command=%proc.cmdline user=%user.name pod=%k8s.pod.name ns=%k8s.ns.name image=%container.image.repository)
priority: WARNING
tags: [custom, container, network-scan, netcat, k8s]
enabled: true
YAML
복사
helm upgrade falco falcosecurity/falco -n falco --reuse-values -f custom-rule1.yaml
Shell
복사
•
/etc/shadow 접근 (default rule 중에 Read sensitive file untrusted 에서 감지
•
chmod 777로 실행 권한 변경 시도
customRules:
detect-chmod-all-open.yaml: |-
- rule: chmod 777 detect
desc: Detect file permission all open
condition: >
spawned_process
and container
and proc.name = chmod
and (proc.args contains "777")
and proc_name_exists
output: >
ALERT: chmod 777 detected (command=%proc.cmdline user=%user.name pod=%k8s.pod.name ns=%k8s.ns.name image=%container.image.repository)
priority: WARNING
tags: [custom, container, chmod, all-open, k8s]
enabled: true
YAML
복사
helm upgrade falco falcosecurity/falco -n falco --reuse-values -f custom-rule2.yaml
Shell
복사
•
curl로 외부 도메인 호출
customRules:
detect-outside-domain-call.yaml: |-
- rule: Curl to external domain
desc: Detect use of curl to call external public domains or IPs from a container
condition: >
spawned_process
and container
and proc.name = curl
and not proc.args contains ".svc"
and not proc.args contains "cluster.local"
and not proc.args contains "10."
and not proc.args contains "192.168."
and not proc.args contains "127."
and not proc.args contains "localhost"
and proc_name_exists
output: >
ALERT: External curl request detected (cmd=%proc.cmdline user=%user.name pod=%k8s.pod.name ns=%k8s.ns.name image=%container.image.repository)
priority: WARNING
tags: [curl, egress, public-access, container]
enabled: true
YAML
복사
helm upgrade falco falcosecurity/falco -n falco --reuse-values -f custom-rule3.yaml
Shell
복사
kubectl run -it --rm test-pod --image=curlimages/curl -- sh
curl http://example.com
Shell
복사
•
주요 바이너리 파일을 포함한 디렉토리에서의 쓰기 액션
customRules:
write_binary_dir.yaml: |-
- macro: open_write
condition: >
(evt.type=open or evt.type=openat) and
fd.typechar='f' and
(evt.arg.flags contains O_WRONLY or
evt.arg.flags contains O_RDWR or
evt.arg.flags contains O_CREAT or
evt.arg.flags contains O_TRUNC)
- macro: package_mgmt_binaries
condition: proc.name in (dpkg, dpkg-preconfigu, rpm, rpmkey, yum)
- macro: bin_dir
condition: fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
- rule: write_binary_dir
desc: an attempt to write to any file below a set of binary directories
condition: evt.dir = < and open_write and not proc.name in (package_mgmt_binaries) and bin_dir
output: "File below a known binary directory opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)"
priority: WARNING
YAML
복사
helm upgrade falco falcosecurity/falco -n falco --reuse-values -f custom-rule4.yaml
Shell
복사
kubectl run -it --rm test-write -n default --image=ubuntu -- bash
apt update && apt install -y bash
echo "test" > /usr/bin/testfile
Shell
복사