TargetGroupBinding¶

TargetGroupBinding is a custom resource (CR) that can expose your pods using an existing ALB TargetGroup or NLB TargetGroup.

This allows you to provision the load balancer infrastructure completely outside of Kubernetes but still manage the targets with Kubernetes Service.

Usage in multi-tenant clusters

TargetGroupBinding allows users to reference any TargetGroup in the AWS account where the EKS cluster resides. In multi-tenant environments, this means tenants can potentially import arbitrary TargetGroups from the account into their namespace, subject to the controller's IAM permissions.

Security risk: Tenants could route traffic to TargetGroups belonging to other workloads or services causing unintended traffic routing.

Recommendation: Cluster administrators should restrict TargetGroupBinding creation using Kubernetes RBAC: Use Kubernetes RBAC to deny create and update permissions on TargetGroupBinding resources for untrusted users.

usage to support Ingress and Service

The AWS LoadBalancer controller internally uses TargetGroupBinding to support the functionality for Ingress, Service, and Gateway resources as well. It automatically creates TargetGroupBinding in the same namespace of the Service used.

You can view all TargetGroupBindings in a namespace by `kubectl get targetgroupbindings -n <your-namespace> -o wide`

Do not manually delete controller-generated TargetGroupBindings

Deleting a TargetGroupBinding that the controller generated for an Ingress, Service, or Gateway resource:

deregisters every target from the associated AWS target group, and
is not immediately reconciled by the controller.

The controller watches the Ingress, Service, and Gateway resources that own the binding, but it does not watch the generated TargetGroupBinding itself. As a result, a deleted binding is only recreated when the owning resource is updated (for example, editing any annotation) or during the periodic resync (--sync-period, which defaults to 10h). Until then, the load balancer forwards to zero registered targets.

If you delete a controller-generated binding while troubleshooting, force an immediate reconcile by editing any annotation on the owning Ingress, Service, or Gateway resource, and verify that the binding and its targets are restored.

EKS Auto Mode users

If you are using EKS Auto Mode, please see the EKS Auto Mode documentation for key differences between the load balancing capability of EKS Auto Mode and the open source load balancer controller.

TargetType¶

TargetGroupBinding CR supports TargetGroups of either instance or ip TargetType.

If TargetType is not explicitly specified, a mutating webhook will automatically call AWS API to find the TargetType for your TargetGroup and set it to the correct value.

Choosing the Target Group¶

You can use either targetGroupARN or targetGroupName to identify a TargetGroup. Although both are unique and immutable in an AWS region, you only have control of the targetGroupName, as targetGroupARN is generated by AWS and contains random characters.

If you provide both targetGroupARN and targetGroupName, the targetGroupARN takes precedence.

Sample YAML¶

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  serviceRef:
    name: awesome-service # route traffic to the awesome-service
    port: 80
  targetGroupName: <name-of-the-targetGroup>

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  serviceRef:
    name: awesome-service # route traffic to the awesome-service
    port: 80
  targetGroupARN: <arn-to-targetGroup>

VpcID¶

TargetGroupBinding CR supports the explicit definition of the Virtual Private Cloud (VPC) of your TargetGroup.

If the VpcID is not explicitly specified, a mutating webhook will automatically call AWS API to find the VpcID for your TargetGroup and set it to the correct value.

Sample YAML with VpcID¶

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  serviceRef:
    name: awesome-service # route traffic to the awesome-service
    port: 80
  targetGroupARN: <arn-to-targetGroup>
  vpcID: <vpcID>

NodeSelector¶

Default Node Selector¶

For TargetType: instance, all nodes of a cluster that match the following selector are added to the TargetGroup by default:

matchExpressions:
  - key: node-role.kubernetes.io/master
    operator: DoesNotExist
  - key: node.kubernetes.io/exclude-from-external-load-balancers
    operator: DoesNotExist
  - key: alpha.service-controller.kubernetes.io/exclude-balancer
    operator: DoesNotExist
  - key: eks.amazonaws.com/compute-type
    operator: NotIn
    values: ["fargate"]

Custom Node Selector¶

TargetGroupBinding CR supports NodeSelector which is a LabelSelector. This will select nodes to attach to the instance TargetType TargetGroup and is merged with the default node selector above.

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  nodeSelector:
    matchLabels:
      foo: bar
  ...

AssumeRole (Cross-Account TargetGroups)¶

Use this feature when you need to manage TargetGroups in a different AWS account than where your EKS cluster runs. This is common in multi-account architectures where load balancers are centrally managed.

Terminology: - Cluster Owner (CO): The AWS account where your EKS cluster and AWS Load Balancer Controller run - TargetGroup Owner (TGO): The AWS account where the TargetGroup exists

Spec fields: * iamRoleArnToAssume: The ARN of the role in the TGO account that the controller will assume * assumeRoleExternalId: External ID for the assume role operation (optional but recommended to prevent the confused deputy problem) * registerTargetsWithPodAvailabilityZone: Register cross-account IP targets with the availability zone of the pod's node instead of all. See Availability zones for cross-account targets. Requires iamRoleArnToAssume.

Sample YAML¶

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: peered-tg
  namespace: nlb-game-2048-1
spec:
  assumeRoleExternalId: very-secret-string-2
  iamRoleArnToAssume: arn:aws:iam::155642222660:role/tg-management-role
  networking:
    ingress:
    - from:
      - securityGroup:
          groupID: sg-0b6a41a2fd959623f
      ports:
      - port: 80
        protocol: TCP
  serviceRef:
    name: service-2048
    port: 80
  targetGroupARN: arn:aws:elasticloadbalancing:us-west-2:155642222660:targetgroup/peered-tg/6a4ecf7bfae473c1

Setup Instructions¶

Step 1: Create IAM role in the TargetGroup Owner (TGO) account

Create a role that allows the AWS Load Balancer Controller role from the Cluster Owner (CO) account to assume it.

Trust policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::565768096483:role/eksctl-awslbc-loadtest-addon-iamserviceaccoun-Role1-13RdJCMqV6p2"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": "very-secret-string"
                }
            }
        }
    ]
}

Step 2: Add permissions to the role in the TGO account

Attach the following policy to the role created in Step 1:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:RegisterTargets",
        "elasticloadbalancing:DeregisterTargets"
      ],
      "Resource": [
        "arn:aws:elasticloadbalancing:us-west-2:155642222660:targetgroup/tg1/*",
        "arn:aws:elasticloadbalancing:us-west-2:155642222660:targetgroup/tg2/*"
      ]
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:DescribeTargetGroups",
        "elasticloadbalancing:DescribeTargetHealth"
      ],
      "Resource": "*"
    }
  ]
}

Step 3: Add AssumeRole permission in the Cluster Owner (CO) account

Add the following permission to the AWS Load Balancer Controller's IAM role. This is not included in the standard IAM policy. For improved security, scope the permission to specific role ARNs:

{
    "Effect": "Allow",
    "Action": [
        "sts:AssumeRole"
    ],
    "Resource": "*"
}

Availability zones for cross-account targets¶

By default, cross-account IP targets are registered with AvailabilityZone: all. Every zonal node of the load balancer can then forward to every target, so turning off cross-zone load balancing has no effect and inter-AZ data transfer charges are unavoidable.

Set registerTargetsWithPodAvailabilityZone: true to register each target with the availability zone of the node its pod runs on. With cross-zone load balancing disabled, a zonal load balancer node then only forwards to targets in its own zone.

Availability zone names are randomized per AWS account — us-west-2a in the CO account is usually a different physical zone than us-west-2a in the TGO account. Availability zone IDs (for example usw2-az1) are stable, so the controller resolves the pod's zone name to a zone ID in the CO account and then back to the matching zone name in the TGO account. Both lookups use ec2:DescribeAvailabilityZones, so the TGO role from Step 2 needs it:

{
  "Effect": "Allow",
  "Action": [
    "ec2:DescribeAvailabilityZones"
  ],
  "Resource": "*"
}

The zone must be one the load balancer has a subnet in. If a cluster runs nodes in a zone the load balancer does not use, registration is rejected and the TargetGroupBinding reports an error.

If the zone cannot be translated — for instance when the TGO role is missing ec2:DescribeAvailabilityZones — the controller logs the failure and falls back to AvailabilityZone: all rather than failing registration.

The setting only affects targets registered after it is set. Targets that are already registered keep their current availability zone (all or a zone name) until they are deregistered and registered again, for example when the pod is replaced. This avoids disrupting live traffic when the setting is toggled on an existing TargetGroupBinding.

MultiCluster TargetGroup¶

TargetGroupBinding CR supports sharing the same TargetGroup ARN among multiple TargetGroupBindings. Setting this flag allows TargetGroup ARNs among multiple clusters or services within the same cluster.

The default value is false, meaning that the controller assumes full control over the TargetGroup ARN and will deregister any targets that are not found within the cluster. To set this flag for TGBs managed by the controller use either: ALB: alb.ingress.kubernetes.io/multi-cluster-target-group: "true" NLB: service.beta.kubernetes.io/aws-load-balancer-multi-cluster-target-group: "true" Gateway: Set enableMultiCluster in the TargetGroupConfiguration attached to your service.

It is not recommended to change this value after TGB creation. Changing between shared / not shared might lead to leaked targets.

Only use this flag if you intend to share the TargetGroup ARN in multiple clusters. This flag will slow down reconciles and put a small additional load on the Kubernetes control plane.

Sample YAML with MultiCluster¶

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: my-tgb
spec:
  serviceRef:
    name: awesome-service # route traffic to the awesome-service
    port: 80
  targetGroupARN: <arn-to-targetGroup>
  multiClusterTargetGroup: true

Reference¶

See the reference for TargetGroupBinding CR