As more developers adopt microservice design patterns, containers and microservices have begun to be deployed at scale in production environments. In this process, more and more challenges are being faced. For example, many microservices are interdependent, requiring more methods and approaches for microservice planning, scaling, and resource management. Additionally, microservices have less isolation from each other—they typically share kernels or networks—which raises higher security requirements.

AWS is the preferred platform for running container workloads. Third-party data shows that 80% of container workloads in the cloud and 82% of Kubernetes workloads are built on the AWS cloud platform. When running containers on AWS, we provide more choices. First, you can choose orchestration tools—AWS-native Amazon ECS or Kubernetes-supporting Amazon EKS. Second, you can choose launch types—whether you want to manage servers. If you want serverless computing for containers, you can choose AWS Fargate mode; if you want to control the installation, configuration, and management of the compute environment, you can choose Amazon EC2 mode. We provide more choices to help you migrate container workloads to the cloud faster, better, and more securely in a more flexible way.

Security and compliance are shared responsibilities between AWS and customers. Based on this, AWS has proposed a shared responsibility model for cloud security. This responsibility is divided into security of the cloud and security in the cloud. AWS is responsible for security of the cloud, including protecting all infrastructure running AWS cloud services, including regions, availability zones, edge sites, compute, storage, networking, databases, etc. Customers are responsible for security in the cloud. Customer responsibilities are determined by the AWS services they use. Generally, customers are responsible for operating system security, network and firewall configuration, identity and access management, application, platform, and customer data security. At the same time, for customer-responsible security in the cloud, AWS provides many tools to help users improve security.

Let's look at specific practices for applying the above shared responsibility model for security and compliance to AWS container services Amazon ECS and Amazon EKS.

First, let's look at identity and access management. When talking about identity and access management, we easily think of AWS IAM service, which can securely manage access to AWS services and resources. You can use IAM to create and manage AWS users and groups, and use various permissions to allow or deny access to AWS resources for these users and groups. For ECS, since it is an AWS-native container solution, IAM can completely manage identity and access control. For EKS, however, you need to understand and configure both IAM and Kubernetes RBAC (Role-Based Access Control). IAM is responsible for assigning permissions to AWS services, while RBAC is responsible for controlling resource permissions.

Let's look at how kubectl, the Kubernetes management tool, performs authentication on EKS. For example, when running the command kubectl get pods, we access the Kubernetes API through kubectl, passing AWS-related identity information. Kubernetes verifies the identity information with IAM, using an IAM authentication plugin called aws-iam-authenticator, which is AWS's official tool for connecting and verifying identity information. After verification returns, the Kubernetes API authorizes AWS identity resource access permissions through RBAC, and finally returns to kubectl whether the execution result is allowed or denied.

For Kubernetes, RBAC is very important. We won't elaborate here—please refer to Kubernetes documentation if needed. Here we'll just provide a basic introduction. In RBAC, a role contains a set of rules for related permissions. In RBAC, permissions are purely cumulative; there are no rules that deny certain operations. Roles can be defined with Role to a specific namespace, or with ClusterRole to the entire cluster. In RBAC, you can define resources to describe objects like pods and nodes; allow verbs on resources like get, update, and delete. At the same time, RBAC has a series of built-in default ClusterRoles, including cluster-admin, admin, edit, view, granting different users different permissions for different operations on different resources.

Additionally, through IAM roles for service accounts on Amazon EKS clusters, you can associate IAM roles with Kubernetes service accounts. Then, this service account can provide AWS permissions to containers in any pod using it. You can scope IAM permissions to service accounts, and only pods using that service account can access those permissions.

Second, let's look at platform security. Here, we're considering logging and auditing of control plane security. Control plane logging, especially audit logs around API actions, is an important part of platform security. For ECS, since it is an AWS-native container service, control plane logs go into AWS CloudTrail for recording cloud resource calls, just like other AWS products. CloudTrail is a service that supports governance, compliance checking, operational auditing, and risk auditing for AWS accounts. With CloudTrail, you can log, continuously monitor, and retain account activity related to operations across your entire AWS infrastructure. For Kubernetes, its control plane includes audit trails, but these logs are not exposed by default. EKS has a feature to enable these logs, and we recommend enabling them and sending them to Amazon CloudWatch for further processing and insights.

Third, let's look at network and firewall configuration, which is also the most important part of container security practices. For ECS, since it is an AWS-native service, you only need to understand and configure Amazon VPC and AWS security groups. For EKS, in addition to managing VPC and security groups, you also need to install and configure Kubernetes network plugins and network policies.

Let's first look at ECS network configuration. When we use ECS with VPC, each task has its own dedicated Elastic Network Interface (ENI). Since each task corresponds to each ENI one-to-one, and each ENI corresponds to each security group one-to-one, any communication in and out of each task goes through the security group, thus achieving network security simply and conveniently.

For EKS, when creating a new Kubernetes cluster, EKS creates an endpoint for the managed Kubernetes API server to communicate with the cluster. By default, this API endpoint is public to the Internet. Access to the API endpoint is protected using a combination of AWS IAM and Kubernetes RBAC. However, we recommend that you enable private access to the Kubernetes API endpoint so that all communication between worker nodes and the API endpoint stays within the VPC. You can restrict IP addresses that can access the API endpoint from the Internet, or completely disable Internet access to the API endpoint.

Amazon VPC CNI is currently the default network plugin for Amazon EKS clusters. In Amazon VPC CNI, for each Kubernetes node, we create multiple ENIs and assign secondary IP addresses. For each pod, we select an available secondary IP address to assign. Through Amazon VPC CNI, EKS gets native Amazon VPC network support, enabling direct pod-to-pod communication, including communication between pods on the same host, between pods on different hosts, and even between pods and other AWS services, pods and on-premises systems, pods and the Internet, providing faster network latency. Amazon VPC CNI is an open source project maintained on GitHub.

For Kubernetes, network policies are a specification of allowed communication rules between pods and between pods and other networks. If you want to manage network policies on EKS, you first need to add a network policy provider to EKS. Calico is a mainstream approach introduced in the official EKS documentation.

One approach that allows both assigning EC2 instance-level IAM roles and fully trusting security group-based approaches is to use different worker node clusters for different pods, or even completely separate clusters. EKS has the concept of NodeGroup, which is an independent auto-scaling worker node group that can be tagged, so you can limit which pods/services can run on it.

Additionally, a service mesh is another method for configuring and managing networks. You can use a service mesh to encrypt and authenticate all services rather than imposing network-level restrictions like AWS security groups or Kubernetes network policies, thus allowing a flatter underlying network while maintaining security. A service mesh is typically implemented through a set of lightweight network proxies deployed alongside application code. Network proxies are included in each microservice, mainly handling communication between microservices, monitoring, and some security-related work. We can use service meshes to enhance security. Taking transport authentication as an example, transport authentication can be understood as service-to-service authentication, which service meshes implement through mutual TLS functionality. When mutual TLS is enabled, traffic between services is encrypted, and mutual access based on certificates and keys ensures communication security between services.

AWS App Mesh is AWS's service mesh offering. App Mesh can integrate with AWS services for monitoring and tracing, and can also work with many popular third-party tools. App Mesh can be used with various containers running on AWS, including ECS, EKS, Fargate, and self-built Kubernetes clusters. Additionally, Istio also supports deployment on EKS well.

Fourth, let's look at operating system security. In container EC2 mode, customers have more security responsibilities. For example, choosing instance types and quantities, CPU-to-RAM ratios, scaling capabilities and availability; choosing which operating system, when to perform OS hardening, when to patch OS, Docker, ECS agent, or kubelet, etc.—these are all customer responsibilities. In Fargate mode, AWS takes on more security responsibilities, and customers do less. AWS is responsible for scaling, patching, securing, and managing servers, performing patching operations for OS, Docker, ECS agent, etc. Fargate needs to run in a VPC network, and Fargate has no privileged container mode. Each ECS task or EKS pod runs in its own dedicated kernel runtime environment and does not share CPU, memory, storage, or network resources with other tasks and pods. This ensures workload isolation for each task or pod and improves security.

For EKS, Kubernetes updates are also an important topic. Typically, Kubernetes has a new major version every quarter, and also regularly releases new minor versions. Sometimes Kubernetes updates are security-related. EKS has APIs for triggering control plane updates. After triggering, you need to update worker nodes—for example, Kubernetes, Docker, and OS. Usually worker nodes are in an auto-scaling group, so we need to rebuild or update the AMI. AWS provides regularly auto-updated AMIs for worker nodes and scripts for manual updates.

Fifth, let's look at customer data security in containers. AWS has both Parameter Store and Secrets Manager to store your secrets. They are integrated into ECS, but for EKS, you need to call them in Kubernetes pods through CLI or SDK. Kubernetes' built-in Secrets functionality stores secrets in its control plane and puts them into running pods through environment variables or files in the file system, but they cannot be used outside the Kubernetes cluster.

We recently released a new feature. You can use keys generated by AWS Key Management Service (KMS) to perform envelope encryption on Kubernetes Secrets stored in EKS; or, you can import keys generated elsewhere into KMS and use them in EKS clusters. Implementing envelope encryption is considered a best security practice for storing sensitive data. We use the open-source AWS Encryption Provider to provide KMS secret envelope encryption in EKS. This project is supported by the Kubernetes community and Kubernetes interest groups.

Finally, let's look at container image security. Best practices for container image security include: not storing secrets inside container images; having one container per service, using sidecar proxies within tasks/pods; minimizing container size, only including what's needed at runtime, etc. At the same time, we should use known and trusted base images, including using official images from Docker Hub, carefully reading Dockerfiles, scanning images for CVEs. We need to specify USER and minimum permissions in Dockerfiles, and establish unique and informative tags for container images to quickly identify container image versions. Container image scanning includes image scanning in registries, images in build pipelines, and runtime container image scanning. Image scanning in registries is provided by Docker Hub and Amazon ECR. There are also third-party open-source or commercial software that can scan images in build pipelines or runtime images. Also, we need to ensure runtime container security. We can limit operations that can be executed in containers through rule engines—for example, "do not run content not included in the container" or "do not run content not in this whitelist" to ensure only trusted images can be deployed/run in the cluster. We need to understand the runtime behavior of the entire environment at any time, and immediately detect running vulnerable containers once a CVE is disclosed.

Summary:

When securely running containers in AWS, customers should assume many responsibilities—running EKS even more so than ECS. Key areas to study in depth include: identity and access management, network topology and firewalls, logging and auditing, encryption and mutual authentication between tasks/pods, container images, patching of container hosts and Kubernetes control planes, secrets management, runtime security of container images, etc.