TargetGroupBinding¶
TargetGroupBinding is a custom resource (CR) that can expose your pods using an existing ALB TargetGroup or NLB TargetGroup.
This allows you to provision the load balancer infrastructure completely outside of Kubernetes but still manage the targets with Kubernetes Service.
Usage in multi-tenant clusters
TargetGroupBinding allows users to reference any TargetGroup in the AWS account where the EKS cluster resides. In multi-tenant environments, this means tenants can potentially import arbitrary TargetGroups from the account into their namespace, subject to the controller's IAM permissions.
Security risk: Tenants could route traffic to TargetGroups belonging to other workloads or services causing unintended traffic routing.
Recommendation: Cluster administrators should restrict TargetGroupBinding creation using Kubernetes RBAC: Use Kubernetes RBAC to deny create and update permissions on TargetGroupBinding resources for untrusted users.
usage to support Ingress and Service
The AWS LoadBalancer controller internally uses TargetGroupBinding to support the functionality for Ingress, Service, and Gateway resources as well. It automatically creates TargetGroupBinding in the same namespace of the Service used.
You can view all TargetGroupBindings in a namespace by `kubectl get targetgroupbindings -n <your-namespace> -o wide`
EKS Auto Mode users
If you are using EKS Auto Mode, please see the EKS Auto Mode documentation for key differences between the load balancing capability of EKS Auto Mode and the open source load balancer controller.
TargetType¶
TargetGroupBinding CR supports TargetGroups of either instance or ip TargetType.
If TargetType is not explicitly specified, a mutating webhook will automatically call AWS API to find the TargetType for your TargetGroup and set it to the correct value.
Choosing the Target Group¶
You can use either targetGroupARN or targetGroupName to identify a TargetGroup. Although both are unique and immutable in an AWS region, you only have control of the targetGroupName, as targetGroupARN is generated by AWS and contains random characters.
If you provide both targetGroupARN and targetGroupName, the targetGroupARN takes precedence.
Sample YAML¶
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
name: my-tgb
spec:
serviceRef:
name: awesome-service # route traffic to the awesome-service
port: 80
targetGroupName: <name-of-the-targetGroup>
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
name: my-tgb
spec:
serviceRef:
name: awesome-service # route traffic to the awesome-service
port: 80
targetGroupARN: <arn-to-targetGroup>
VpcID¶
TargetGroupBinding CR supports the explicit definition of the Virtual Private Cloud (VPC) of your TargetGroup.
If the VpcID is not explicitly specified, a mutating webhook will automatically call AWS API to find the VpcID for your TargetGroup and set it to the correct value.
Sample YAML with VpcID¶
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
name: my-tgb
spec:
serviceRef:
name: awesome-service # route traffic to the awesome-service
port: 80
targetGroupARN: <arn-to-targetGroup>
vpcID: <vpcID>
NodeSelector¶
Default Node Selector¶
For TargetType: instance, all nodes of a cluster that match the following
selector are added to the TargetGroup by default:
matchExpressions:
- key: node-role.kubernetes.io/master
operator: DoesNotExist
- key: node.kubernetes.io/exclude-from-external-load-balancers
operator: DoesNotExist
- key: alpha.service-controller.kubernetes.io/exclude-balancer
operator: DoesNotExist
- key: eks.amazonaws.com/compute-type
operator: NotIn
values: ["fargate"]
Custom Node Selector¶
TargetGroupBinding CR supports NodeSelector which is a
LabelSelector. This will select nodes to attach to the
instance TargetType TargetGroup and is merged with the default node
selector above.
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
name: my-tgb
spec:
nodeSelector:
matchLabels:
foo: bar
...
AssumeRole (Cross-Account TargetGroups)¶
Use this feature when you need to manage TargetGroups in a different AWS account than where your EKS cluster runs. This is common in multi-account architectures where load balancers are centrally managed.
Terminology: - Cluster Owner (CO): The AWS account where your EKS cluster and AWS Load Balancer Controller run - TargetGroup Owner (TGO): The AWS account where the TargetGroup exists
Spec fields:
* iamRoleArnToAssume: The ARN of the role in the TGO account that the controller will assume
* assumeRoleExternalId: External ID for the assume role operation (optional but recommended to prevent the confused deputy problem)
Sample YAML¶
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
name: peered-tg
namespace: nlb-game-2048-1
spec:
assumeRoleExternalId: very-secret-string-2
iamRoleArnToAssume: arn:aws:iam::155642222660:role/tg-management-role
networking:
ingress:
- from:
- securityGroup:
groupID: sg-0b6a41a2fd959623f
ports:
- port: 80
protocol: TCP
serviceRef:
name: service-2048
port: 80
targetGroupARN: arn:aws:elasticloadbalancing:us-west-2:155642222660:targetgroup/peered-tg/6a4ecf7bfae473c1
Setup Instructions¶
Step 1: Create IAM role in the TargetGroup Owner (TGO) account
Create a role that allows the AWS Load Balancer Controller role from the Cluster Owner (CO) account to assume it.
Trust policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::565768096483:role/eksctl-awslbc-loadtest-addon-iamserviceaccoun-Role1-13RdJCMqV6p2"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "very-secret-string"
}
}
}
]
}
Step 2: Add permissions to the role in the TGO account
Attach the following policy to the role created in Step 1:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:DeregisterTargets"
],
"Resource": [
"arn:aws:elasticloadbalancing:us-west-2:155642222660:targetgroup/tg1/*",
"arn:aws:elasticloadbalancing:us-west-2:155642222660:targetgroup/tg2/*"
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetHealth"
],
"Resource": "*"
}
]
}
Step 3: Add AssumeRole permission in the Cluster Owner (CO) account
Add the following permission to the AWS Load Balancer Controller's IAM role. This is not included in the standard IAM policy. For improved security, scope the permission to specific role ARNs:
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": "*"
}
MultiCluster TargetGroup¶
TargetGroupBinding CR supports sharing the same TargetGroup ARN among multiple TargetGroupBindings. Setting this flag allows TargetGroup ARNs among multiple clusters or services within the same cluster.
The default value is false, meaning that the controller assumes full control over the TargetGroup ARN and will deregister any targets that are not found within the cluster.
To set this flag for TGBs managed by the controller use either:
ALB: alb.ingress.kubernetes.io/multi-cluster-target-group: "true"
NLB: service.beta.kubernetes.io/aws-load-balancer-multi-cluster-target-group: "true"
Gateway: Set enableMultiCluster in the TargetGroupConfiguration attached to your service.
It is not recommended to change this value after TGB creation. Changing between shared / not shared might lead to leaked targets.
Only use this flag if you intend to share the TargetGroup ARN in multiple clusters. This flag will slow down reconciles and put a small additional load on the Kubernetes control plane.
Sample YAML with MultiCluster¶
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
name: my-tgb
spec:
serviceRef:
name: awesome-service # route traffic to the awesome-service
port: 80
targetGroupARN: <arn-to-targetGroup>
multiClusterTargetGroup: true
Reference¶
See the reference for TargetGroupBinding CR