TargetGroupBinding¶

TargetGroupBinding is a custom resource (CR) that can expose your pods using an existing ALB TargetGroup or NLB TargetGroup.

This will allow you to provision the load balancer infrastructure completely outside of Kubernetes but still manage the targets with Kubernetes Service.

usage to support Ingress and Service

The AWS LoadBalancer controller internally used TargetGroupBinding to support the functionality for Ingress and Service resource as well. It automatically creates TargetGroupBinding in the same namespace of the Service used.

You can view all TargetGroupBindings in a namespace by kubectl get targetgroupbindings -n <your-namespace> -o wide

TargetType¶

TargetGroupBinding CR supports TargetGroups of either instance or ip TargetType.

If TargetType is not explicitly specified, a mutating webhook will automatically call AWS API to find the TargetType for your TargetGroup and set it to correct value.

Choosing the Target Group¶

One can either use targetGroupARN of targetGroupName to identify a Target Group. Although both are unique and immutable in an AWS region, one only has control of the targetGroupName, for targetGroupARN is generated by AWS and contain random characters.

If you provide both targetGroupARN and targetGroupName, beware that targetGroupARN prevails.

Sample YAMLs¶

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  serviceRef:
    name: awesome-service # route traffic to the awesome-service
    port: 80
  targetGroupName: <name-of-the-targetGroup>

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  serviceRef:
    name: awesome-service # route traffic to the awesome-service
    port: 80
  targetGroupARN: <arn-to-targetGroup>

VpcID¶

TargetGroupBinding CR supports the explicit definition of the Virtual Private Cloud (VPC) of your TargetGroup.

If the VpcID is not explicitly specified, a mutating webhook will automatically call AWS API to find the VpcID for your TargetGroup and set it to correct value.

Sample YAML¶

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  serviceRef:
    name: awesome-service # route traffic to the awesome-service
    port: 80
  targetGroupARN: <arn-to-targetGroup>
  vpcID: <vpcID>

NodeSelector¶

Default Node Selector¶

For TargetType: instance, all nodes of a cluster that match the following selector are added to the target group by default:

matchExpressions:
  - key: node-role.kubernetes.io/master
    operator: DoesNotExist
  - key: node.kubernetes.io/exclude-from-external-load-balancers
    operator: DoesNotExist
  - key: alpha.service-controller.kubernetes.io/exclude-balancer
    operator: DoesNotExist
  - key: eks.amazonaws.com/compute-type
    operator: NotIn
    values: ["fargate"]

Custom Node Selector¶

TargetGroupBinding CR supports NodeSelector which is a LabelSelector. This will select nodes to attach to the instance TargetType target group and is merged with the default node selector above.

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  nodeSelector:
    matchLabels:
      foo: bar
  ...

AssumeRole¶

Sometimes the AWS LoadBalancer controller needs to manipulate target groups from different AWS accounts. The way to do that is assuming a role from such an account. The following spec fields help you with that.

iamRoleArnToAssume: the ARN that you need to assume
assumeRoleExternalId: the external ID for the assume role operation. Optional, but recommended. It helps you to prevent the confused deputy problem ( https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html )

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: peered-tg
  namespace: nlb-game-2048-1
spec:
  assumeRoleExternalId: very-secret-string-2
  iamRoleArnToAssume: arn:aws:iam::155642222660:role/tg-management-role
  networking:
    ingress:
    - from:
      - securityGroup:
          groupID: sg-0b6a41a2fd959623f
      ports:
      - port: 80
        protocol: TCP
  serviceRef:
    name: service-2048
    port: 80
  targetGroupARN: arn:aws:elasticloadbalancing:us-west-2:155642222660:targetgroup/peered-tg/6a4ecf7bfae473c1

In the following examples, we will refer to Cluster Owner (CO) and Target Group Owner (TGO) accounts.

First, in the TGO account creates a role that will allow the AWS LBC in the CO account to assume it. For improved security, we only allow the AWS LBC role in CO account to assume the role.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::565768096483:role/eksctl-awslbc-loadtest-addon-iamserviceaccoun-Role1-13RdJCMqV6p2"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": "very-secret-string"
                }
            }
        }
    ]
}

Next, still in the TGO account we need to add the following permissions to the Role created in the first step.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:RegisterTargets",
        "elasticloadbalancing:DeregisterTargets"
      ],
      "Resource": [
        "arn:aws:elasticloadbalancing:us-west-2:155642222660:targetgroup/tg1/*",
        "arn:aws:elasticloadbalancing:us-west-2:155642222660:targetgroup/tg2/*"
        // add more here //
      ]
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:DescribeTargetGroups",
        "elasticloadbalancing:DescribeTargetHealth"
      ],
      "Resource": "*"
    }
  ]
}

Next, in the CO account, we need to allow the AWS LBC to perform the AssumeRole call. By default, this permission is not a part of the standard IAM policy that is vended with the LBC installation scripts. For improved security, it is possible to scope the AssumeRole permissions down to only roles that you know ahead of time the LBC will need to Assume.

        {
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole"
            ],
            "Resource": "*"
        }

Sample YAML¶

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  iamRoleArnToAssume: "arn:aws:iam::999999999999:role/alb-controller-policy-to-assume"
  assumeRoleExternalId: "some-magic-string"
  ...

MultiCluster Target Group¶

TargetGroupBinding CRD supports sharing the same target group ARN among multiple clusters. Setting this flag will ensure the controller only operates on targets within the cluster.

The default value is false, meaning that the controller assumes full control over the target group ARN and will deregister any targets that are not found within the cluster. To set this flag for TGBs managed by the controller use either: ALB: alb.ingress.kubernetes.io/multi-cluster-target-group: "true" NLB: service.beta.kubernetes.io/aws-load-balancer-multi-cluster-target-group: "true"

It is not recommended to change this value after TGB creation. Changing between shared / not shared might lead to leaked targets.

Only use this flag if you intend to share the target group ARN in multiple clusters. This flag will slow down reconciles and put a small additonal load on the kubernetes control plane.

Sample YAML¶

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  serviceRef:
    name: awesome-service # route traffic to the awesome-service
    port: 80
  targetGroupARN: <arn-to-targetGroup>
  multiClusterTargetGroup: true

Reference¶

See the reference for TargetGroupBinding CR