AWS Batch란?
AWS Batch는 대규모 배치 컴퓨팅 작업을 효율적으로 실행하고 관리할 수 있는 완전 관리형 서비스이다.
EKS_CLUSTER_NAME="<CLUSTER_NAME>"
ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
Shell
복사
apiVersion: v1
kind: Namespace
metadata:
name: batch
labels:
name: batch
YAML
복사
kubectl apply -f ns.yaml
Shell
복사
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: batch-cluster-role
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
resources: ["daemonsets", "deployments", "statefulsets", "replicasets"]
verbs: ["get", "list", "watch"]
- apiGroups: ["rbac.authorization.k8s.io"]
resources: ["clusterroles", "clusterrolebindings"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: batch-cluster-role-binding
subjects:
- kind: User
name: batch
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: batch-cluster-role
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: batch-compute-environment-role
namespace: batch
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["create", "get", "list", "watch", "delete", "patch"]
- apiGroups: [""]
resources: ["serviceaccounts"]
verbs: ["get", "list"]
- apiGroups: ["rbac.authorization.k8s.io"]
resources: ["roles", "rolebindings"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: batch-compute-environment-role-binding
namespace: batch
subjects:
- kind: User
name: batch
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: batch-compute-environment-role
apiGroup: rbac.authorization.k8s.io
YAML
복사
kubectl apply -f rbac.yaml
Shell
복사
eksctl create iamidentitymapping \
--cluster $EKS_CLUSTER_NAME \
--arn "arn:aws:iam::$ACCOUNT_ID:role/AWSServiceRoleForBatch" \
--username batch
Shell
복사
•
COMPUTE ENVIRONMENT (컴퓨팅 환경)
cat <<EOF > ./batch-eks-compute-environment.json
{
"computeEnvironmentName": "skills-batch-eks-ce",
"type": "MANAGED",
"state": "ENABLED",
"eksConfiguration": {
"eksClusterArn": "arn:aws:eks:ap-northeast-2:362708816803:cluster/skills-eks-cluster",
"kubernetesNamespace": "batch"
},
"computeResources": {
"type": "EC2",
"allocationStrategy": "BEST_FIT_PROGRESSIVE",
"minvCpus": 0,
"maxvCpus": 128,
"instanceTypes": [
"c5.large"
],
"subnets": [
"subnet-09b9ef37bb0c33961",
"subnet-0dc080f81d0456cbe"
],
"securityGroupIds": [
"sg-0e633a2a9006a4361"
],
"instanceRole": "arn:aws:iam::362708816803:instance-profile/skills-role"
}
}
EOF
aws batch create-compute-environment --cli-input-json file://./batch-eks-compute-environment.json
Shell
복사
•
JOB QUEUE (작업 대기열)
cat <<EOF > ./batch-eks-job-queue.json
{
"jobQueueName": "skills-batch-eks-jq",
"priority": 10,
"computeEnvironmentOrder": [
{
"order": 1,
"computeEnvironment": "skills-batch-eks-ce"
}
]
}
EOF
aws batch create-job-queue --cli-input-json file://./batch-eks-job-queue.json
Shell
복사
•
JOB DEFINITION (작업 정의)
cat <<EOF > ./batch-eks-jd.json
{
"jobDefinitionName": "skills-batch-eks-job-definition",
"type": "container",
"eksProperties": {
"podProperties": {
"hostNetwork": true,
"containers": [
{
"image": "nginx",
"name": "nginx-container",
"command": [
"nginx",
"-g",
"daemon off;"
],
"resources": {
"limits": {
"cpu": "1",
"memory": "512Mi"
},
"requests": {
"cpu": "1",
"memory": "512Mi"
}
},
"env": [],
"volumeMounts": []
}
],
"volumes": [],
"metadata": {
"labels": {
"environment": "test"
}
}
}
}
}
EOF
aws batch register-job-definition --cli-input-json file://batch-eks-jd.json
Shell
복사
•
JOB SUBMIT (작업)
JOB_QUEUE=$(aws batch describe-job-queues --query 'jobQueues[0].jobQueueName' --output text)
JOB_DEFINITION=$(aws batch describe-job-definitions --query 'jobDefinitions[0].jobDefinitionName' --output text)
JOB_NAME="skills-batch-eks-job"
aws batch submit-job --job-queue $JOB_QUEUE \
--job-definition $JOB_DEFINITION \
--job-name $JOB_NAME
Shell
복사
# Pod 확인
kubectl get po -n batch
Shell
복사