Efficient EKS cluster management with AWS Cloud9

To address this challenge, our team turned to AWS Cloud9. AWS Cloud9 is a cloud-based integrated development environment (IDE) that lets you write, run, and debug your code with a browser. We employed AWS Cloud9 as a management environment for our EKS clusters. This environment could be securely shared with other IAM roles or IAM users within our AWS account, facilitating collaborative work while maintaining secure access.

Our focus was on optimising for security. To achieve this, we selected the AWS Cloud9 IDE service. This service exists within our AWS account, which means it inherits the existing access controls of our AWS console. This integration provides a secure environment for managing our EKS clusters.

Within the Cloud9 environment, we deploy and manage our EKS clusters. We've also detailed how to leverage EKS's role-based access control (RBAC) to segment tenants by namespace. This segmentation allows for fine-grained control over access and resources, ensuring each tenant can operate independently and securely.

Our customers have derived several benefits from this solution. It includes an EKS management environment optimised for secure access via the AWS console, providing a safe, controlled environment for managing EKS clusters. Additionally, the solution leverages AWS Cloud9's shared access management capabilities for IAM roles and IAM users within an AWS account, enabling collaborative work while maintaining secure access. Using EKS's unique IAM role-based RBAC capabilities for multi-tenant EKS clusters offers fine-grained access control, guaranteeing secure, independent operations for each tenant. The solution also incorporates reusable Infrastructure as Code (IaC) templates for rapid deployment and adaptation, facilitating quick and effortless setup and modification of EKS clusters. Lastly, an EKS solution optimised for security is a part of the package, ensuring all operations are conducted securely, protecting our customers and their data.

Prerequisite:

1. Deploy VPC via CloudFormation template located in this repo: cfn/vpc-2az.yaml
2. CloudFormation stack name must be vpc-2az

Note: VPC subnet selected for Cloud9 must be publicly available.

1. Login AWS Console
2. Create new Cloud9 environment named "eks-management-env"
3. Allow Cloud9 environment setup to complete

Update Cloud9 IAM Role permissions to allow AdministratorAccess

1. Use AWS Console and navigate to IAM -> Role -> AWSCloud9SSMAccessRole
2. Update existing role AWSCloud9SSMAccessRole by adding IAM permission policy AdministratorAccess

Setup - (CloudFormation)

1. Open Cloud9 environment named "eks-management-env"
2. In Cloud9, open Preferences -> AWS Settings. Disable the "AWS Manage Temporary Credentials" toggle
3. In Cloud9, open a terminal session Clone the GitHub repo for this project:
   
   Unset
   git clone https://github.com/ha-king/eks-cfn.git
   
4. cd eks-cfn/cfn
5. /bin/sh deploy.sh EKS-DEV us-east-1
6. Visit CloudFormation service to view the stack status, until status value is CREATE_COMPLETE
7. cd ../cloud9
8. /bin/sh install_kubectl.sh
9. /bin/sh update-kubeconfig.sh EKS-DEV us-east-1
10. Run the kubectl get all - A command to view all Kubernetes resources
    
    Clean up:
    Unset
    awscloudformationdelete-stack--stack-nameEKS-DEV

Setup - (Terraform)

1. Open Cloud9 environment named "eks-management-env"
2. In Cloud9, open Preferences -> AWS Settings. Disable the "AWS Manage Temporary Credentials" toggle
3. In Cloud9, open a terminal session Clone the GitHub repo for this project:
   
   Unset
   git clone https://github.com/ha-king/eks-cfn.git
   
4. cd eks-cfn/tf
5. /bin/sh deploy.sh
6. cd ../cloud9
7. /bin/sh install_kubectl.sh
8. /bin/sh update-kubeconfig.sh EKS-DEV-TF us-east-1
9. Run the kubectl get all - A command to view all Kubernetes resources
   
   Clean up:
   Unset
   cd tf
   terraformdestroy--auto-approve

Sharing Cloud9 Environment

1. To invite an IAM user, enter arn:aws:iam::123456789012:user/MyUser. Replace 123456789012 with your AWS account ID and replace MyUser with the name of the user.
2. To invite a user with an assumed role or a federated user with an assumed role, enter arn:aws:sts::123456789012:assumed-role/MyAssumedRole/MyAssumedRoleSession.
3. To invite the AWS account root user, enter arn:aws:iam::123456789012:root. Replace 123456789012 with your AWS account ID.

Example:
Unset
awscloud9create-environment-membership--environment-id1234567890987654321--user-arnarn:aws:iam::123456789098:root--permissionsread-write

Setup - EKS Admin IAM entities

1. kubectl edit cm/aws-auth -n kube-system
2. Reference the aws-auth configuration map below:
   
   Unset
   # Please edit the object below. Lines beginning with a '#' will be ignored,
   # and an empty file will abort the edit. If an error occurs while saving this file will be
   # reopened with the relevant failures.
   #
   apiVersion: v1
   data:
     mapRoles: |
       - groups:
         - system:bootstrappers
         - system:nodes
         rolearn: arn:aws:iam::123456789098:role/eks-dev-EksNodeWorkerRole-123456789
         username: system:node:{{EC2PrivateDNSName}}
       - groups:
         - system:masters
         rolearn: arn:aws:iam::123456789098:role/eks-dev-EksRbacAdminRole-123456789
         username: ec2-admin
       - groups:
         - system:masters
         rolearn: arn:aws:iam::123456789098:role/my-role-name
         username: my-admin
     mapUsers: |
       - groups:
         - system:masters
         userarn: arn:aws:iam::123456789098:user/my-user
         username: my-user
   kind: ConfigMap
   metadata:
     creationTimestamp: "2023-05-25T17:32:16Z"
     name: aws-auth
     namespace: kube-system
     resourceVersion: "21552"
   uid:13e8d1d3-c6a1-4369-aff8-e58e94572ad3

EKS RBAC Setup

Prerequisite: Create an IAM Role for this purpose

Notes: This Cloudformation deployment for EKS cluster also creates an EC2 Instance profile, see the Resources tab of CloudFormation

1. cd ../cloud9
2. /bin/sh install_eksctl.sh
3. cd ../rbac
4. /bin/sh create-rolebindings.sh NAMESPACE EKSCLUSTER NAMESPACE_ROLE_ARN

EKS RBAC Cleanup

1. cd rbac
2. /bin/sh delete-rolebindings.sh NAMESPACE EKSCLUSTER ROLE_YAML BINDING_YAML NAMESPACE_ROLE_ARN

References

1. https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html#create-service-role
2. https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/deploy-an-amazon-eks[…]-cluster-from-aws-cloud9-using-an-ec2-instance-profile.html
3. https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html
4. https://docs.aws.amazon.com/cloud9/latest/user-guide/share-environment.html#share-environment-admin-user
5. https://repost.aws/knowledge-center/eks-iam-permissions-namespaces