The security of Kubernetes clusters is crucial and should not be treated as an afterthought. Unfortunately, it appears to be widely accepted that developers cannot be expected to write secure code. A Gitlab’s 2021 Global DevSecOps Survey reveals that an overwhelming majority of security professionals do not have confidence in the ability of developers to produce secure programs or applications from the get-go.
More than three-quarters of the security teams that responded to the survey say that “devs find too few bugs too late in the process.” This needs to change, and it should not be that much of a hassle for developers who are learning container orchestration to also learn Kubernetes with security in mind. It is a good time to start becoming more mindful of security.
Go over the following discussions on getting accustomed to Kubectl and learning how to be security-aware with container orchestration in the Kubernetes environment right from the start.
Aim to master Kubectl
Kubectl is the standard command line tool used to communicate and interact with Kubernetes (K8s) clusters. It is essential in creating, deleting, and managing resources on the K8s platform. This tool is designed to be intuitive and easy to use, but just like other software tools, getting acquainted varies from person to person. There are no effective shortcuts in learning how to use Kubectl. Having a Kubectl commands cheat sheet for the most crucial functions would be great, but it is crucial to actually spend time using it to perform various functions.
Becoming proficient with Kubectl flags and commands is important in cluster security because it allows you to proceed to the critical steps in securing clusters more quickly. You can control access to the K8s API and the Kubelet seamlessly. Also, you get to control workload or user (at runtime) capabilities with greater ease instead of having to constantly consult a reference for the appropriate commands.
The Kubernetes website provides useful Kubectl resources including a comprehensive listing of the different options and environmental variables, overrides, operations, resource types, output options, and usage conventions. These references are presented in a simplified way complete with command guides and examples. These should serve as an excellent starting point for anyone interested to learn Kubectl.
First line of defense: Regulate K8s API access
Kubernetes is fully API-driven, so it logically follows that the first “security control” lies in the access to the API. There are three vital steps to secure the K8s API.
- Implementing Transport Layer Security (TLS) for API traffic – All API traffic should be encrypted, and TLS is the most suitable default encryption technology to use for this. However, some local ports may be enabled over HTTP when doing certain installation methods. In such cases, administrators must be familiar with the settings used for each component installation to keep track of traffic that is potentially not secure.
- API authentication – There has to be an authentication mechanism for API servers to ensure secure access. A static token approach, for example, can be used in small single-user clusters. For bigger ones, it would be advisable to integrate an LDAP server, for example, to be able to authenticate with the users divided into sub-groups. API authentication should cover all client connections, including nodes, schedulers, volume plugins, and proxies.
- API authorization – In addition to authentication, it is also important to employ an authorization system, particularly an integrated Role-Based Access Control (RBAC) mechanism, wherein sets of permissions are associated with specific roles and users or user groups. The authorization scheme may be cluster-scoped or namespace-scoped.
Control Kubelet access
Kubelet is the primary node agent running on each node. It is the underlying technology in the creation, application, updating, and destroying of containers on K8s nodes. It is crucial to regulate Kubelet access because it can expose HTTPS endpoints, which may allow threat actors to gain control over nodes and containers. Also, Kubelets, by default, permits unauthenticated access to the Kubernetes API.
Check out the nifty Kubelet authentication and authorization guide on the official Kubernetes website. The process is relatively long, so it would be difficult to squeeze in all the details here.
Protect cluster components
There are different ways to secure cluster components from cyber threats. The most common and important ones are described below.
- Etcd access restriction – Free access to the etcd backend of the K8s API is just like allowing threat actors to gain root access on the entire cluster. As such, it should be restricted with solid authentication and authorization methods.
- Audit logger activation – Audit logging and archiving should always be enabled to have useful data to work on if ever security breaches happen.
- Secrets encryption and infrastructure credentials rotation – The secrets used in Kubernetes should always be properly encrypted both at rest and in motion. Also, they must be short-lived to significantly reduce the opportunities threat actors may have to steal and use these secrets in their attacks.
- Third-party integration evaluation – Kubernetes has extensive third-party integration support, which is generally good but can also become a possible vulnerability. As such, all integrations should be meticulously examined before they are enabled. Untrusted components or those that cannot be verified should not be allowed to generate pods in any system namespace. There are cases when third-party integrations require the ability to view all secrets on a cluster. Such instances are not inherently harmful, that’s why it is crucial to meticulously review them. As much as possible, the access granted should be limited to functioning in a single namespace.
- Limiting access to alpha and beta features – As the names suggest, alpha and beta features are not the most polished releases. They can contain bugs or weaknesses that can be exploited by threat actors, so it is advisable to restrict access to them.
Regulate workload or user capabilities at runtime
Workloads and users are cyber attack targets, especially at runtime. They can be vulnerable, so it is crucial to control what they can do to limit their capabilities based on what a certain task or process only requires. Regulation can be through the following:
- Resource usage limitations on a cluster – It is possible to impose resource quotas to control the amount of resources such as CPU and memory available to a namespace.
- Container privilege restriction – Pod security standards can be enforced through pod security admission configuration. This configuration can also be used to detect security breaches.
- Stopping container from loading harmful kernel modules – The Linux kernel has the weakness of allowing automatic kernel module loading. This can be prevented by uninstalling unwanted or suspicious modules from the node or writing rules to block them.
- Network access restriction – Network policies can be established for a namespace to make it possible for app authors to limit pods in other namespaces from accessing pods and ports in their (authors’) namespaces.
- Cloud metadata API access control – When Kubernetes is used on a cloud platform, it is advisable to control the permissions granted to instance credentials. This can be done by using network policies to prevent pod access to the metadata API and prevent secrets from being shared unnecessarily.
- Regulating pod access to nodes – Without restricting policies, any node may run a pod. To protect K8s workloads and users, it is recommended to control the placement of pods onto nodes.
Takeaway
Container orchestration is relatively new and many are likely taking their first steps in becoming familiar with or mastering it. Instead of going about learning Kubernetes (and other related topics) with little or no regard for security, it helps to be more mindful of container and cluster security. The pointers discussed above should offer some relevant insights.
