Container Orchestration


Container Orchestration ์ด๋ž€ ๋ณต์žกํ•œ ์ปจํ…Œ์ด๋„ˆ ํ™˜๊ฒฝ์„ ํšจ๊ณผ์ ์œผ๋กœ ๊ด€๋ฆฌํ•˜๊ธฐ ์œ„ํ•œ ๋„๊ตฌ๋กœ K8s ๊ฐ€ ์ฃผ๋กœ ์“ฐ์ธ๋‹ค.

  • Cluster = Master Node ๊ฐ€ ์ค‘์•™์ œ์–ด, ๋‹ค์ˆ˜์˜ Worker Node ๊ฐ€ ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ์ด๋ฃจ์–ด ์„œ๋กœ ํ†ต์‹ ํ•˜๋ฉฐ ์ž‘๋™
  • State = Desired State ์„ ์–ธ ์‹œ ๊ด€๋ฆฌ์ž์˜ ๊ฐœ์ž… ์—†์ด ์ž๋™์œผ๋กœ ์ƒํƒœ๋ฅผ ์œ ์ง€
  • Scheduling = Container ๋ฅผ ๋ฐฐ์น˜ํ•  ์ ํ•ฉํ•œ Worker Node ๋ฅผ ์ฐพ์•„ ๋ฐฐํฌ
  • Rollout Rollback = ๋ฐฐํฌ ๋ฒ„์ „๊ด€๋ฆฌ
  • Service Discovery = ์„œ๋น„์Šค ๋“ฑ๋ก ๋ฐ ์กฐํšŒ
  • Volume = NFS, EBS ๋“ฑ ๋‹ค์–‘ํ•œ ์Šคํ† ๋ฆฌ์ง€ ๋งˆ์šดํŒ…

K8s Architecture


K8s ๋Š” Master Node ์™€ Worker Node ์˜ ์ง‘ํ•ฉ์œผ๋กœ ์ด๋ฃจ์–ด์ง„๋‹ค.

Master Node ๋Š” Worker Node ๋“ค์„ Manage, Plan, Schedule, Monitor ํ•˜๋Š” ์—ญํ• ์„ ์ˆ˜ํ–‰ํ•˜๊ธฐ ์œ„ํ•ด ์•„๋ž˜์™€ ๊ฐ™์€ Component ๋“ค์„ ๊ฐ€์ง„๋‹ค.

  • kube-apiserver
  • controller manager
  • etcd cluster
  • kube-scheduler

Worker Node ๋Š” Containerized Application ์ด ์‹คํ–‰๋˜๋Š” Node ๋กœ, ์ด๋ฅผ ์œ„ํ•ด ์•„๋ž˜์™€ ๊ฐ™์€ Component ๋“ค์„ ๊ฐ€์ง„๋‹ค.

  • Container Runtime
  • kubelet
  • kube-proxy

Docker vs containerd


  • Back in time, K8s only supported Docker as its container runtime
  • CRI(Container Runtime Interface) was introduced as demand for various container runtime increases
  • CRI allowed any vendor to work as a container runtime as long as they follow OCI(Open Container Initiative) standards
    • OCI consists of imagespec and runtimespec
  • Docker consists of containerd and many other components such as CLI, API, etc.
  • containerd is compatible with CRI however other components need dockershim to be compatible with K8s
    • But now K8s no longer supports Docker engine through dockershim
  • We can use below 2 CLIs instead of docker to work with K8s
    • nerdctl = for general purpose from ContainerD community
    • crictl = for debugging from K8s community (works with all CRI compatible container runtimes)

ETCD


  • ETCD is a distributed reliable key-value store that is Simple, Secure & Fast
  • K8s ์˜ ๋ชจ๋“  ์ƒํƒœ์™€ ๋ฐ์ดํ„ฐ๋ฅผ ์ €์žฅ
  • Key-Value ํ˜•ํƒœ๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ์ €์žฅ
  • ๋ถ„์‚ฐ ์‹œ์Šคํ…œ์œผ๋กœ ๊ตฌ์„ฑํ•˜์—ฌ ๊ณ ๊ฐ€์šฉ์„ฑ ํ™•๋ณด
  • TTL, watch ๋“ฑ ๋ถ€๊ฐ€ ๊ธฐ๋Šฅ ์ œ๊ณต

Kube-API Server


  • ์ƒํƒœ๋ฅผ ๋ฐ”๊พธ๊ฑฐ๋‚˜ ์กฐํšŒ
  • etcd ์™€ ์œ ์ผํ•˜๊ฒŒ ํ†ต์‹ ํ•˜๋Š” ๋ชจ๋“ˆ
  • REST API ํ˜•ํƒœ๋กœ ์ œ๊ณต
  • ์š”์ฒญ์— ๋Œ€ํ•œ ๊ถŒํ•œ ์ฒดํฌ
  • ์ˆ˜ํ‰์  ํ™•์žฅ ๊ฐ€๋Šฅ
  • kubeadm ์œผ๋กœ ์„ค์น˜ํ•  ๊ฒฝ์šฐ Pod ์˜ ํ˜•ํƒœ๋กœ ์‹คํ–‰๋œ๋‹ค.
  1. Authenticate User
  2. Validate Request
  3. Retrieve data
  4. Update ETCD
  5. Scheduler
  6. Kubelet

Kube Controller Manager


  • ๋‹ค์–‘ํ•œ Controller ๊ฐ€ ์กด์žฌ
    • A controller is a process that continuously monitors the state of the components within the system and works towards bringing the whole system to the desired functioning state
    • Replication Controller, Node Controller, Endpoint Controller, โ€ฆ
    • ๋Š์ž„ ์—†์ด ์ƒํƒœ๋ฅผ ์ฒดํฌํ•˜๊ณ  ์›ํ•˜๋Š” ์ƒํƒœ๋ฅผ ์œ ์ง€
    • ๋ณต์žก์„ฑ์„ ๋‚ฎ์ถ”๊ธฐ ์œ„ํ•ด ํ•˜๋‚˜์˜ ํ”„๋กœ์„ธ์Šค๋กœ ์‹คํ–‰

Installing Kube Controller Manager

wget https://storage.googleapis.com/kubernetes-release/release/v1.13.0/bin/linux/amd64/kube-controller-manager
  • kubeadm ์œผ๋กœ ์„ค์น˜ํ•  ๊ฒฝ์šฐ Pod ์˜ ํ˜•ํƒœ๋กœ ์‹คํ–‰๋œ๋‹ค.

Kube Scheduler


  • kube-scheduler ๋Š” ์ƒ์„ฑ ์š”์ฒญ๋œ Pod ๊ฐ€ ์–ด๋Š Node ์— ๋ฐฐํฌ๋˜์–ด์•ผ ํ•˜๋Š”์ง€ ํ™•์ธํ•˜๋Š” ์—ญํ• ์„ ์ˆ˜ํ–‰ํ•œ๋‹ค. Node ์˜ ํ˜„์žฌ ์ƒํƒœ์™€ Pod ์˜ ์š”๊ตฌ์‚ฌํ•ญ์„ ์ฒดํฌํ•˜์—ฌ ์ ์ ˆํ•œ Node ๋ฅผ ์ฐพ๋Š” ์ž‘์—…๋งŒ ์ˆ˜ํ–‰ํ•  ๋ฟ Pod ๋ฅผ ์ƒ์„ฑํ•˜๋Š” ์—ญํ• ์€ kubelet ์ด ์ˆ˜ํ–‰ํ•œ๋‹ค.

How does it work?

  • Pod ์— ํ•„์š”ํ•œ ๋ฆฌ์†Œ์Šค๋ฅผ ์—ฌ๋ถ„์œผ๋กœ ๊ฐ€์ง€๊ณ  ์žˆ๋Š” Node ๋ฅผ ํ™•์ธํ•˜๊ณ  Pod ๊ฐ€ ๋ฐฐ์น˜๋œ ํ›„ ๋‚จ์€ ๋ฆฌ์†Œ์Šค๋Ÿ‰์„ ๊ธฐ์ค€์œผ๋กœ ์ˆœ์œ„๋ฅผ ๋งค๊ฒจ ์Šค์ผ€์ฅด๋ง์„ ์ˆ˜ํ–‰ํ•œ๋‹ค.

Installing Kube Scheduler

wget https://storage.googleapis.com/kubernetes-release/release/v1.13.0/bin/linux/amd64/kube-scheduler
  • kubeadm ์œผ๋กœ ์„ค์น˜ํ•  ๊ฒฝ์šฐ Pod ์˜ ํ˜•ํƒœ๋กœ ์‹คํ–‰๋œ๋‹ค.

Kubelet


kubelet ์€ Master Node ์˜ kube-apiserver ๋กœ ๋ถ€ํ„ฐ Container ์ƒ์„ฑ ์š”์ฒญ์„ ๋ฐ›์•„ Worker Node ์— ์„ค์น˜๋œ Container Runtime ์„ ์ด์šฉํ•ด Container ๋ฅผ ์ƒ์„ฑํ•˜๋Š” ์—ญํ• ์„ ํ•œ๋‹ค. kubelet ์€ ์ž์‹ ์ด ์œ„์น˜ํ•œ Node ์™€ ์ƒ์„ฑํ•œ Pod ๋“ค์„ ๋ชจ๋‹ˆํ„ฐ๋งํ•˜์—ฌ ์ฃผ๊ธฐ์ ์œผ๋กœ kube-apiserver ์™€ ์†Œํ†ตํ•œ๋‹ค.

Installing Kubelet

wget https://storage.googleapis.com/kubernetes-release/release/v1.13.0/bin/linux/amd64/kubelet

kubeadm ์œผ๋กœ K8s Cluster ๋ฅผ ๊ตฌ์ถ•ํ•  ๋•Œ kubelet ์€ ์„ค์น˜๋˜์ง€ ์•Š์œผ๋‹ˆ ์ˆ˜๋™์œผ๋กœ Worker Node ์— kubelet ์„ ์„ค์น˜ํ•ด์ฃผ์–ด์•ผ ํ•œ๋‹ค. ๋•Œ๋ฌธ์— kubelet ์€ ๋‹ค๋ฅธ Component ๋“ค๊ณผ ๋‹ค๋ฅด๊ฒŒ Pod ์ด ์•„๋‹Œ Node ์˜ ํ”„๋กœ์„ธ์Šค๋กœ์„œ ์‹คํ–‰๋œ๋‹ค.

Kube Proxy


  • ๋„คํŠธ์›Œํฌ ํ”„๋ก์‹œ์™€ ๋ถ€ํ•˜ ๋ถ„์‚ฐ ์—ญํ• ์„ ํ•˜๋ฉฐ K8s Cluster ์— ๋ฐฐํฌ๋œ ๋ชจ๋“  Pod ๊ฐ„์˜ ํ†ต์‹ ์„ ๋‹ด๋‹นํ•œ๋‹ค.
  • Pod ํ˜•ํƒœ๋กœ ์‹คํ–‰๋˜๊ธฐ ๋•Œ๋ฌธ์— kubectl get pods ๋กœ ์กฐํšŒ ๊ฐ€๋Šฅํ•˜๋‹ค.
  • ์„ฑ๋Šฅ์ƒ์˜ ์ด์œ ๋กœ ๋ณ„๋„์˜ ํ”„๋ก์‹œ ํ”„๋กœ๊ทธ๋žจ ๋Œ€์‹  iptables ๋˜๋Š” IPVS ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์„ค์ •๋งŒ ๊ด€๋ฆฌํ•œ๋‹ค.

References