IRSA (IAM role for service account) is a very useful tool which permits us to provide AWS permissions to the containers in any pod that uses that service account. This enables us to give a very granular permission setting for specific service accounts and remove most of the headache related to authentication and persisting of secrets.
Below a quick snippet of how to create such mechanism.
Prerequisites
I am going to assume that we have following datasources and locals in place:
1data "aws_eks_cluster" "cluster" {
2 name = var.cluster_name
3}
4locals {
5 openid_url = trimprefix(data.aws_eks_cluster.cluster.identity[0].oidc[0].issuer, "https://")
6}
this will give us required information about the oidc provider used for our cluster.
Iam roles
First, create an iam policy
1resource "aws_iam_policy" "cluster_autoscaler" {
2 count = var.app_autoscaler_enable ? 1 : 0
3 name_prefix = "cluster-autoscaler-${var.cluster_name}"
4 description = "EKS cluster-autoscaler policy for cluster ${var.cluster_name}"
5 policy = data.aws_iam_policy_document.cluster_autoscaler.json
6}
7
8data "aws_iam_policy_document" "cluster_autoscaler" {
9 statement {
10 sid = "clusterAutoscalerAll"
11
12 effect = "Allow"
13
14 actions = [
15 "autoscaling:DescribeAutoScalingGroups",
16 "autoscaling:DescribeAutoScalingInstances",
17 "autoscaling:DescribeLaunchConfigurations",
18 "autoscaling:DescribeTags",
19 "ec2:DescribeLaunchTemplateVersions",
20 "ec2:DescribeInstanceTypes"
21 ]
22
23 resources = ["*"]
24 }
25
26 statement {
27 sid = "clusterAutoscalerOwn"
28 effect = "Allow"
29
30 actions = [
31 "autoscaling:SetDesiredCapacity",
32 "autoscaling:TerminateInstanceInAutoScalingGroup",
33 "autoscaling:UpdateAutoScalingGroup",
34 ]
35
36 resources = ["*"]
37
38 condition {
39 test = "StringEquals"
40 variable = "autoscaling:ResourceTag/kubernetes.io/cluster/${var.cluster_name}"
41 values = ["owned"]
42 }
43
44 condition {
45 test = "StringEquals"
46 variable = "autoscaling:ResourceTag/k8s.io/cluster-autoscaler/enabled"
47 values = ["true"]
48 }
49 }
50}
now create the iam role with trust policy. For the ease of implementation we are going to use a terraform module
1module "cluster_autoscaler" {
2 source = "terraform-aws-modules/iam/aws//modules/iam-assumable-role-with-oidc"
3 version = "4.1.0"
4 create_role = true
5 role_name_prefix = "cluster-autoscaler-${var.cluster_name}"
6 provider_url = local.openid_url
7 role_policy_arns = [aws_iam_policy.cluster_autoscaler[0].arn]
8 oidc_fully_qualified_subjects = ["system:serviceaccount:kube-system:cluster-autoscaler"]
9}
you will need to change oidc_fully_qualified_subjects and replace the namespace (kube-system) and your service account role (cluster-autoscaler)
Bind service account to the role
Now you can use the previously created iam role and bind it to your service account, i.e.
1resource "kubectl_manifest" "argocd-app-autoscaler" {
2 yaml_body = templatefile(
3 "${path.module}/templates/argo-helm-app.yaml", merge(local.argocd_default_app_values, {
4 name = var.app_autoscaler_helm_chart_name
5 namespace = "kube-system"
6 project = "runtime"
7 helm_chart = var.app_autoscaler_helm_chart_name
8 helm_chart_repoUrl = var.app_autoscaler_helm_chart
9 helm_chart_version = var.app_autoscaler_helm_chart_version
10 helm_values = <<-EOT
11awsRegion: ${var.aws_region}
12fullnameOverride: cluster-autoscaler
13priorityClassName: system-cluster-critical
14serviceMonitor:
15 enabled: true
16prometheusRule:
17 enabled: true
18replicaCount: 2
19rbac:
20 create: true
21 serviceAccount:
22 name: cluster-autoscaler
23 annotations:
24 # This value should match the ARN of the role created by module.iam_assumable_role_admin in irsa.tf
25 eks.amazonaws.com/role-arn: "${module.cluster_autoscaler.iam_role_arn}"
26autoDiscovery:
27 clusterName: ${var.cluster_name}
28 enabled: true
29EOT
30 })
31 )
32}
Comments